claudetools/clients/birth-biologic/session-logs/2026-06/2026-06-30-mike-birthbio-quality-sync-complete.md

Birth Biologic — Quality Systems Department -> Datto SharePoint sync (COMPLETED)

User

• User: Mike Swanson (mike)
• Machine: GURU-5070
• Role: admin

Session Summary

Picked up the Birth Biologic Quality Systems Department -> Datto SharePoint migration after a Mac session (Mikes-MacBook-Air) worked it all day and failed to complete. The goal: make the SharePoint "Quality Systems Department" document library contain every file from the Datto Workplace source (3,768 files) on agent ACG-DWP-X-BB. Started by loading the pulled-in continuation docs and reconciling two contradictory records — an earlier one claiming a surgical 5-file delete "COMPLETE," and a later CONTINUE-QUALITY-SYNC.md describing a wipe-and-reupload stuck at 3,249/3,768.

Rather than trust the stale docs, pulled live ground truth via the Graph delta endpoint: SharePoint held 3,337 files (only 17 >=4MB), gap of 431. Re-enumerated the Datto source via RMM, which revealed the actual scope the Mac missed: 3,768 files, of which 301 are >=4MB (~29.7 GB total, largest a 3.94 GB .mov). The Mac's upload script skipped every file >=4MB, so it could never converge — the entire large-file corpus (the bulk of the gap) was being ignored.

Wrote a correct uploader (upload-quality-final.ps1): idempotent (skip a file if SharePoint already has it at matching size), simple PUT for <4MB and Graph chunked upload sessions for

=4MB, \\?\ long-path handling, Graph token auto-refresh for long runs, an internal time budget, and a tailable progress log. Validated the risky path on the single largest file (3.94 GB) before the full run. That test also uncovered the second root cause: the RMM agent ignores the timeout field (caps ~300 s, producing the Mac's zombie "running" commands) and honors timeout_seconds instead.

Cancelled the hung command 9e0fcfe8, dispatched the full uploader with timeout_seconds: 10800, and monitored to completion in the background. Result: 1,137 uploaded (283 large + 854 small), 2,628 already-matching, 3 errors. The idempotent size-check also silently repaired ~700 files that existed but had a mismatched size (partial/corrupt residue from the earlier failed attempts).

Verified the final state with a full Datto-vs-SharePoint diff: 0 Datto files missing from SharePoint (all 3,768 present), and exactly 4 differences — all live current work edited today by named client staff. Mike chose to preserve them. Documented the outcome in an authoritative completion doc, retired the contradictory continuation doc, logged the two lessons to errorlog, and saved two reference memories.

Key Decisions

• Trusted live Graph/RMM enumeration over the committed docs — the docs contradicted each other and were hours stale. Ground truth drove every decision.
• Did not undo the Mac's wipe-and-reupload (already executed before pickup); finished the rebuild additively (upload-only). Performed no deletions.
• Fixed the >=4MB gap with proper chunked upload sessions rather than continuing to skip large files (the actual reason the job never finished).
• Validated the chunked-upload/long-path/Content-Range path on one real 3.94 GB file before committing to a ~30 GB run.
• Preserved the 4 live-work files (1 new xlsx + 3 open/locked docs edited today by staff) rather than forcing a byte-identical match — overwriting would have destroyed today's work. Confirmed with Mike.
• Kept the client secret out of the repo (scripts carry SECRET_PLACEHOLDER; injected at dispatch time from the vault into a scratch payload that was deleted).

Problems Encountered

• Upload could never reach 3,768: prior script skipped all 301 files >=4MB (~29.7 GB). Fixed with Graph chunked upload sessions (10 MB chunks, Content-Range).
• Zombie/hung RMM commands: agent ignores timeout, capping at ~300 s. Fixed by using timeout_seconds (both fields sent). Cancelled the stuck 9e0fcfe8 first.
• Recursive /children count timed out (one HTTP call per folder). Switched to the Graph /root/delta endpoint — whole drive in ~9 paged calls.
• requests not installed on GURU-5070 Python. Rewrote checks with stdlib urllib.
• Long paths (>260) in Datto tree: used the \\?\ prefix for [IO.File] reads.
• 3 upload errors (423 Locked / 409 Conflict): files were locked because staff had them open — live current work, correctly left untouched.
• SharePoint 3,769 vs Datto 3,768 (one extra): diffed to Temperature Excursion Log.xlsx, created today by client staff; preserved, not deleted.

Configuration Changes

Created:

• clients/birth-biologic/scripts/enumerate-datto.ps1 — Datto enumeration + manifest writer.
• clients/birth-biologic/scripts/upload-quality-final.ps1 — idempotent chunked uploader (carries SECRET_PLACEHOLDER, not a real secret).
• clients/birth-biologic/docs/migration/2026-06-30-quality-sync-COMPLETE.md — authoritative final record.
• .claude/memory/reference_gururmm_command_timeout_seconds.md
• .claude/memory/reference_sharepoint_graph_large_file_upload.md

Modified:

• clients/birth-biologic/CONTINUE-QUALITY-SYNC.md — replaced with a RESOLVED pointer.
• .claude/memory/MEMORY.md — two new reference index lines.
• errorlog.md — one --friction (timeout_seconds) + one --correction (>=4MB skip).

Credentials & Secrets

• No new credentials created or discovered. Used existing vault entries only.
• Graph app (client-credentials): vault msp-tools/computerguru-tenant-admin, credentials.client_secret. Tenant 19a568e8-9e88-413b-9341-cbc224b39145, client 709e6eed-0711-4875-9c44-2d3518c47063.
• GuruRMM API: via .claude/scripts/rmm-auth.sh (vault infrastructure/gururmm-server.sops.yaml).
• Note: dispatching PowerShell that embeds the Graph secret leaves that secret in the RMM command-history record (same as the prior Mac scripts). Acceptable here (internal RMM), but a future hardening would have the agent fetch the secret itself.

Infrastructure & Servers

• Agent: ACG-DWP-X-BB, id a4524e85-8a07-45d0-91b1-51ce7e2ca74a.
• Datto source: C:\Users\Public\Desktop\Datto Workplace Server Projects\Quality Department (3,768 files, 301 >=4MB, ~29.7 GB).
• SharePoint drive: b!F8BzMb1YakCIWCyWlmczb09LHqtxDxVMpLT6kAwYmsM7NUY4oPLSRq7ng3tJq-E9.
• RMM API (internal): http://172.16.3.30:3001 (JWT via rmm-auth.sh).
• VM work files: C:\Windows\Temp\quality-manifest.txt, C:\Windows\Temp\quality-upload.log.

Commands & Outputs

• Count via delta: GET /drives/{drive}/root/delta?$select=...&$top=500 — final live count 3,769.
• Dispatch: POST {RMM}/api/agents/{id}/command with {command_type:"powershell", command:<script>, timeout_seconds:10800}.
• Poll: GET {RMM}/api/commands/{id} (status completed/failed/cancelled).
• Cancel: POST {RMM}/api/commands/{id}/cancel.
• Full run result: up=1137 skip=2628 err=3 big=283 total=3768 reachedEnd=True.
• Errors: 409 Conflict DEV39.Exhibit B.docx; 423 Locked on both ...Current.docx files.
• Diff: 0 Datto files missing from SP; 1 extra in SP (LOGS/Equipment/3. Temp Excursions/Temperature Excursion Log.xlsx, created 2026-06-30 by Mary Ster, edited by Kristin Steen).

Pending / Incomplete Tasks

• None functional. Migration complete and verified.
• Optional future: parallel-stream uploader (multiple concurrent files + ~60 MiB chunks + Expect100Continue=$false) to beat the ~40 Mbps single-session SharePoint Online ceiling on large migrations.
• Optional cleanup: remove C:\Windows\Temp\quality-manifest.txt / quality-upload.log from ACG-DWP-X-BB (harmless, no secrets).

Reference Information

• Authoritative doc: clients/birth-biologic/docs/migration/2026-06-30-quality-sync-COMPLETE.md
• Command IDs: hung/cancelled 9e0fcfe8-0619-4a39-bd9c-6f5fd75c9b55; full run 4c978424-03cf-401c-805a-45162ff52be2; big-file test fb3a6c4b-7158-4b77-9988-4326503753d8.
• Memories: gururmm-command-timeout-seconds, sharepoint-graph-large-file-upload.