sync: auto-sync from GURU-5070 at 2026-06-30 17:21:06

Author: Mike Swanson
Machine: GURU-5070
Timestamp: 2026-06-30 17:21:06

.claude/memory/MEMORY.md

@@ -32,6 +32,8 @@
 - [Gitea Internal API Access](reference_gitea_internal.md) — git.azcomputerguru.com is NOT behind Cloudflare — it's the office Cox IP NAT'd to NPM (openresty) on Jupiter. Prefer internal 172.16.3.20:3000 for reliability (bypasses NPM SSL-renewal reload blips).
 - [Gitea git-op latency](reference_gitea_git_op_latency.md) — SSH (.20:2222) is SLOWEST (~1.5s); internal HTTP+token ~0.55s; SOPS lookup only ~0.33s. Don't switch to SSH for speed. Gitea SSH is .20:2222 (API ssh_url .21 is wrong).
 - [GuruRMM technical reference](reference_gururmm.md) — Server (172.16.3.30) layout + downloads dir `/var/www/gururmm/downloads` + `.channel` sidecar rollout control (stable/beta) + privileged server access via the server's OWN root RMM agent (hostname `gururmm`, no SSH needed; plink fallback) + API + `context=user_session` (WTS impersonation) + build-pipeline vendoring at `deploy/build-pipeline/` + Linux agent systemd sandbox trap.
+- [GuruRMM command timeout_seconds](reference_gururmm_command_timeout_seconds.md) — agent command dispatch honors `timeout_seconds`, NOT `timeout`; long jobs die ~300s / go zombie (`running`, empty stdout) otherwise. Cost Birth Biologic a full day.
+- [SharePoint Graph large-file upload](reference_sharepoint_graph_large_file_upload.md) — <4MB simple PUT, >=4MB MUST use chunked upload session (Content-Range); `\\?\` long paths; idempotent size-check; verify counts via /root/delta; single stream ~40Mbps (SPO throttle).
 - [RMM agent update model](rmm-agent-update-model.md) — Agent updates are server-PUSH on heartbeat (no self-poll); available versions = filesystem scan needing a `.sha256`; promote flips `.channel` sidecars beta→stable globally. Two stranders: beta-first freezes stable until an explicit promote; agents older than ~0.6.50 re-enroll with a NEW device_id/agent row when updated.
 - [GuruRMM physical server storage](gururmm-physical-server-storage.md) — New box 172.16.1.231 (temp IP→will be .30), Ubuntu 26.04, ssh key `gururmm-physical`/alias `gururmm-new`. SSD (915G root) = HOT (PG default tablespace + WAL + builds); HDD ext4 at `/data` = COLD (`gururmm_cold` PG tablespace for aged `agent_logs` partitions + downloads + backups + archive). The #3 retention answer.
 - [Trebesch DESKTOP-QNP3ON5 shell replacement](reference_trebesch_qnp3on5.md) — AT Trebesch box runs an Explorer shell replacement; explorer.exe owner check returns blank — use Win32_ComputerSystem.UserName. GuruRMM SWIFT-LION-2892.

.claude/memory/reference_gururmm_command_timeout_seconds.md

@@ -0,0 +1,20 @@
+---
+name: gururmm-command-timeout-seconds
+description: GuruRMM agent command dispatch honors timeout_seconds, NOT timeout — long jobs die at ~300s otherwise
+metadata:
+  type: reference
+---
+
+When dispatching a command to a GuruRMM agent (`POST {RMM}/api/agents/{id}/command`), the
+execution timeout is controlled by **`timeout_seconds`**, not `timeout`. Passing only
+`timeout` leaves the agent at its default cap (~300s): long-running commands get killed and
+often surface as a zombie `status: running` with empty stdout that never terminates.
+
+For multi-minute/multi-hour work (large uploads, big enumerations) set `timeout_seconds` high
+(e.g. 10800) — send both fields to be safe. Poll `GET {RMM}/api/commands/{command_id}` for
+`status` in {completed, failed, cancelled}; cancel a stuck one with
+`POST {RMM}/api/commands/{id}/cancel`.
+
+Cost the fleet a full day of failed Birth Biologic SharePoint uploads (Mac session) before
+this was spotted. See [[reference_sharepoint_graph_large_file_upload]]. Auth helper:
+`.claude/scripts/rmm-auth.sh` (internal `172.16.3.30:3001`).

.claude/memory/reference_sharepoint_graph_large_file_upload.md

@@ -0,0 +1,27 @@
+---
+name: sharepoint-graph-large-file-upload
+description: Uploading files to SharePoint via Graph — simple PUT <4MB, chunked upload session >=4MB; verify counts via delta
+metadata:
+  type: reference
+---
+
+Pushing a folder tree into a SharePoint doc library via Microsoft Graph (app-only):
+
+- **<4MB:** simple `PUT /drives/{drive}/root:/{path}:/content`.
+- **>=4MB:** MUST use an **upload session** — `POST .../root:/{path}:/createUploadSession`
+  then `PUT` the file in chunks (multiple of 320 KiB; 10 MB works) with a
+  `Content-Range: bytes {start}-{end}/{total}` header. In PowerShell 5.1
+  `Invoke-RestMethod -Headers @{ 'Content-Range'=... }` handles this fine. A naive script that
+  only does <4MB PUTs will silently skip every large file and never reach the target count.
+- **Long Windows paths (>260):** prefix the local path with `\\?\` for `[IO.File]` reads.
+- **Idempotent sync:** existence-check each file (`GET root:/{path}?$select=size`) and skip if
+  size matches — this also catches/repairs partial-upload residue from earlier failed runs.
+- **Throughput:** a single sequential upload stream to SharePoint Online plateaus ~40 Mbps
+  regardless of link speed (per-session SPO throttle + PS5.1 HTTP stack + Expect100Continue).
+  For speed use parallel file streams + larger chunks + `Expect100Continue=$false`.
+- **Verify total file count** with the Graph **delta** endpoint
+  (`/drives/{drive}/root/delta`) — whole-drive enumeration in a few paged calls, far cheaper
+  than recursive `/children`.
+
+Proven end-to-end on Birth Biologic Quality Systems (3,768 files, 301 >=4MB, ~29.7 GB;
+largest 3.94 GB). Dispatched via GuruRMM — see [[gururmm-command-timeout-seconds]].

clients/birth-biologic/CONTINUE-QUALITY-SYNC.md

@@ -1,271 +1,12 @@
-# Birth Biologic Quality Department Sync - CONTINUATION INSTRUCTIONS
+# Birth Biologic Quality Department Sync — RESOLVED
 
-**Date:** 2026-06-30
+**Status:** COMPLETE (2026-06-30, GURU-5070). This continuation file is obsolete — do NOT
-**Status:** IN PROGRESS - Upload script running
+act on the instructions that used to be here.
-**Client:** Birth Biologic
-**Task:** Sync SharePoint Quality Systems Department to match Datto exactly (3,768 files)
 
----
+Authoritative record: `docs/migration/2026-06-30-quality-sync-COMPLETE.md`.
 
-## CURRENT STATE
+Summary: all 3,768 Datto files are present in SharePoint (verified via Graph delta). The
-
+only differences are 4 files that are live current work edited today by client staff
-**SharePoint Status:**
+(1 new xlsx + 3 open docs), intentionally preserved per Mike. The prior approach failed all
-- Current file count: 3,249 files
+day because its script skipped every file >=4MB (301 files, ~29.7 GB) and used the RMM
-- Target file count: 3,768 files (from Datto)
+`timeout` field (capped ~300s) instead of `timeout_seconds`.
-- Gap: 519 files remaining
-
-**Active RMM Command:**
-- Command ID: `9e0fcfe8-0619-4a39-bd9c-6f5fd75c9b55`
-- Agent: ACG-DWP-X-BB (a4524e85-8a07-45d0-91b1-51ce7e2ca74a)
-- Status: Running (as of last check)
-- Purpose: Upload all 3,768 files from Datto to SharePoint via Graph API
-- Drive ID concatenation workaround: `"b" + "!" + "..."` to avoid PowerShell escaping
-
-**Background Monitor:**
-- Task ID: b24474c (monitoring upload every minute for 20 minutes)
-
----
-
-## WHAT HAPPENED
-
-1. **Initial Approach:** Tried OneDrive sync by robocopy to local OneDrive folder
-   - Result: Synced 3,249 files then STALLED (no progress for 35+ minutes)
-
-2. **Switched to Direct Graph API Upload:**
-   - Multiple attempts failed due to PowerShell escaping the `!` in drive ID
-   - Drive ID: `b!F8BzMb1YakCIWCyWlmczb09LHqtxDxVMpLT6kAwYmsM7NUY4oPLSRq7ng3tJq-E9`
-   - Problem: PowerShell kept converting `b!` to `b\!` causing HTTP 400 errors
-   - Solution: Concatenate at runtime: `$driveId = "b" + "!" + "F8Bz..."`
-
-3. **Current Upload:**
-   - Command dispatched successfully with drive ID concatenation
-   - Script has been running but showing no output yet (may still be scanning files)
-
----
-
-## CREDENTIALS
-
-**Graph API (from vault):**
-- Path: `msp-tools/computerguru-tenant-admin`
-- Tenant ID: 19a568e8-9e88-413b-9341-cbc224b39145
-- Client ID: 709e6eed-0711-4875-9c44-2d3518c47063
-- Client Secret: (in vault at `credentials.client_secret`)
-
-**GuruRMM:**
-- Vault path: `infrastructure/gururmm-server.sops.yaml`
-- Agent ID: a4524e85-8a07-45d0-91b1-51ce7e2ca74a (ACG-DWP-X-BB)
-
-**SharePoint Drive ID:**
-- `b!F8BzMb1YakCIWCyWlmczb09LHqtxDxVMpLT6kAwYmsM7NUY4oPLSRq7ng3tJq-E9`
-
----
-
-## NEXT STEPS TO CONTINUE
-
-### Option 1: Check if current upload completed
-
-```bash
-# Authenticate to RMM
-eval "$(bash .claude/scripts/rmm-auth.sh)"
-
-# Check upload status
-curl -s "$RMM/api/commands/9e0fcfe8-0619-4a39-bd9c-6f5fd75c9b55" \
-  -H "Authorization: Bearer $TOKEN" > /tmp/upload-status.json
-
-python3 -c "
-import json
-with open('/tmp/upload-status.json') as f:
-    data = json.load(f)
-print(f\"Status: {data.get('status')}\")
-print(f\"Exit code: {data.get('exit_code')}\")
-stdout = data.get('stdout', '') or ''
-if len(stdout) > 0:
-    print('\n--- Last 30 lines ---')
-    for line in stdout.split('\n')[-30:]:
-        print(line)
-"
-
-# Then verify SharePoint count
-python3 clients/birth-biologic/scripts/check-quality-status.py
-```
-
-### Option 2: If upload failed, use the working PowerShell script
-
-The correct script is in: `clients/birth-biologic/scripts/upload-final-working.ps1`
-
-Run via RMM:
-```bash
-eval "$(bash .claude/scripts/rmm-auth.sh)"
-AGENT_ID="a4524e85-8a07-45d0-91b1-51ce7e2ca74a"
-CLIENT_SECRET=$(bash .claude/scripts/vault.sh get-field msp-tools/computerguru-tenant-admin credentials.client_secret)
-
-# Script content with proper drive ID concatenation
-SCRIPT='... (see upload-final-working.ps1) ...'
-
-PAYLOAD=$(jq -n --arg cmd "$SCRIPT" '{command_type: "powershell", command: $cmd, timeout_seconds: 1800}')
-curl -s -X POST "$RMM/api/agents/$AGENT_ID/command" \
-  -H "Authorization: Bearer $TOKEN" \
-  -H "Content-Type: application/json" \
-  -d "$PAYLOAD" | jq -r '.command_id'
-```
-
----
-
-## FILES CREATED
-
-**Scripts:**
-- `clients/birth-biologic/scripts/check-quality-status.py` - Check SharePoint file count
-- `clients/birth-biologic/scripts/upload-datto-to-sharepoint.ps1` - Base upload script
-- `clients/birth-biologic/scripts/reset-quality-exact.py` - Initial reset script (used)
-- `clients/birth-biologic/scripts/exact-sync-quality.py` - Robocopy approach
-- `clients/birth-biologic/scripts/sync-quality-simple.py` - Earlier attempt
-- `clients/birth-biologic/scripts/finish-upload.py` - Bulk upload attempt
-- `clients/birth-biologic/scripts/upload-remaining-files.py` - Remaining files upload
-
-**Todo List:**
-1. [completed] Delete ALL files from SharePoint Quality Systems Department
-2. [in_progress] Copy ALL files from Datto to SharePoint exactly as they exist
-3. [pending] Verify SharePoint has exactly 3768 files matching Datto
-
----
-
-## DATTO SOURCE PATH
-
-**On ACG-DWP-X-BB:**
-```
-C:\Users\Public\Desktop\Datto Workplace Server Projects\Quality Department
-```
-
-**Total files:** 3,768 files (verified via Get-ChildItem -Recurse)
-
-**Files >4MB:** ~63 files (these are being skipped - need separate large file upload later)
-
----
-
-## VERIFICATION COMMAND
-
-Once upload completes:
-
-```python
-python3 clients/birth-biologic/scripts/check-quality-status.py
-```
-
-Expected output:
-```
-SharePoint: 3768 files
-Datto: 3768 files
-Gap: 0 files
-
-[OK] MATCH - SharePoint has exactly 3768 files
-```
-
----
-
-## TROUBLESHOOTING
-
-**If upload shows 0 uploaded, 3467 errors:**
-- This means drive ID was escaped wrong (b\! instead of b!)
-- Solution: Use string concatenation `"b" + "!" + "..."`
-
-**If upload hangs with no output:**
-- PowerShell may have syntax error
-- Check command_text field to verify script sent correctly
-
-**If you need to cancel running command:**
-```bash
-curl -s -X POST "$RMM/api/commands/9e0fcfe8-0619-4a39-bd9c-6f5fd75c9b55/cancel" \
-  -H "Authorization: Bearer $TOKEN" \
-  -H "Content-Type: application/json"
-```
-
----
-
-## WORKING UPLOAD SCRIPT TEMPLATE
-
-Save this as `upload-final-working.ps1`:
-
-```powershell
-$source = "C:\Users\Public\Desktop\Datto Workplace Server Projects\Quality Department"
-$driveId = "b" + "!" + "F8BzMb1YakCIWCyWlmczb09LHqtxDxVMpLT6kAwYmsM7NUY4oPLSRq7ng3tJq-E9"
-$tenantId = "19a568e8-9e88-413b-9341-cbc224b39145"
-$clientId = "709e6eed-0711-4875-9c44-2d3518c47063"
-$clientSecret = "GET_FROM_VAULT"
-
-Write-Host "Getting Graph API token..."
-$tokenBody = @{
-    client_id = $clientId
-    client_secret = $clientSecret
-    scope = "https://graph.microsoft.com/.default"
-    grant_type = "client_credentials"
-}
-
-$tokenResponse = Invoke-RestMethod -Method Post -Uri "https://login.microsoftonline.com/$tenantId/oauth2/v2.0/token" -Body $tokenBody
-$token = $tokenResponse.access_token
-Write-Host "[OK] Token acquired"
-
-Write-Host "Scanning Datto files..."
-$files = Get-ChildItem $source -Recurse -File -ErrorAction SilentlyContinue
-Write-Host "[OK] Found $($files.Count) files"
-
-Write-Host "Uploading files..."
-$uploaded = 0
-$errors = 0
-
-foreach ($file in $files) {
-    $relativePath = $file.FullName.Substring($source.Length + 1)
-    $uploadPath = $relativePath.Replace("\", "/")
-
-    try {
-        if ($file.Length -lt 4MB) {
-            $uploadUrl = "https://graph.microsoft.com/v1.0/drives/$driveId/root:/$uploadPath" + ":/content"
-            $headers = @{
-                "Authorization" = "Bearer $token"
-                "Content-Type" = "application/octet-stream"
-            }
-
-            $fileBytes = [System.IO.File]::ReadAllBytes($file.FullName)
-            Invoke-RestMethod -Method Put -Uri $uploadUrl -Headers $headers -Body $fileBytes -UseBasicParsing | Out-Null
-
-            $uploaded++
-            if ($uploaded % 100 -eq 0) {
-                Write-Host "  Uploaded $uploaded files..."
-            }
-        }
-    } catch {
-        $errors++
-    }
-}
-
-Write-Host ""
-Write-Host "Upload Complete"
-Write-Host "Uploaded: $uploaded"
-Write-Host "Errors: $errors"
-Write-Host "Total: $($files.Count)"
-```
-
----
-
-## SESSION NOTES
-
-- OneDrive sync is unreliable for large migrations (stalled at 86%)
-- Direct Graph API upload is the correct approach
-- PowerShell exclamation mark escaping is a major gotcha
-- Use string concatenation to avoid escape issues
-- Files >4MB need separate upload session logic (not implemented yet)
-- There are ~63 large files that will need separate handling
-
-**Client was promised this yesterday - NOW VERY ANGRY**
-
----
-
-## QUICK RESUME CHECKLIST
-
-1. [ ] Check if command 9e0fcfe8 completed
-2. [ ] Verify SharePoint file count (should be 3,768)
-3. [ ] If not complete, dispatch new upload with working script
-4. [ ] Monitor for completion (15-20 minutes)
-5. [ ] Verify final count matches Datto
-6. [ ] Handle large files (>4MB) if needed
-7. [ ] Document completion
-
-**END OF CONTINUATION INSTRUCTIONS**

clients/birth-biologic/docs/migration/2026-06-30-quality-sync-COMPLETE.md

@@ -0,0 +1,70 @@
+# Birth Biologic — Quality Systems Department -> Datto sync — COMPLETE
+
+**Date:** 2026-06-30
+**Completed by:** Mike Swanson / claude-main (GURU-5070)
+**Status:** COMPLETE — verified against live Graph enumeration
+
+## Outcome
+
+Every file in the Datto source is now present in SharePoint.
+
+| | Datto (source) | SharePoint (final) |
+|---|---:|---:|
+| Files | 3,768 | 3,769 |
+| Datto files missing from SharePoint | — | **0** |
+
+All 301 large files (>=4MB, ~29.7 GB total, largest a 3.94 GB .mov) were uploaded via
+Graph chunked upload sessions. The idempotent size-check pass also detected and re-uploaded
+~700 files that existed in SharePoint with a mismatched size (partial/corrupt residue from
+the earlier failed OneDrive/PUT attempts).
+
+## The 4 intentional differences (live current work — PRESERVED per Mike, 2026-06-30)
+
+These are the "recently modified, do not clobber" carve-out. Confirmed as live edits made
+*today* by named client staff, so they were left intact rather than forced to match Datto:
+
+| File | Datto size | SharePoint size | Note |
+|---|---:|---:|---|
+| `LOGS/Equipment/3. Temp Excursions/Temperature Excursion Log.xlsx` | not present | 40,253 | Created 2026-06-30 by Mary Ster, edited by Kristin Steen |
+| `LOGS/Equipment/2. Validation List/QSP-200.003.A Validation List.Current.docx` | 41,908 | 51,510 | Locked/open, edited today |
+| `LOGS/Equipment/1.Equipment List/QSP-200.001.C Equipment List.Current.docx` | 47,947 | 57,545 | Locked/open, edited today |
+| `LOGS/Quality Assurance Reporting Log/Deviations/2024/DEV39.Exhibit B.docx` | 15,525 | 22,437 | In use, edited today |
+
+The three docs surfaced as upload "errors" (HTTP 423 Locked / 409 Conflict) precisely
+*because* staff had them open — SharePoint's lock protected live work. That is the correct
+result, not a failure.
+
+## What actually fixed it (root cause of the all-day Mac failures)
+
+1. **The prior upload script skipped every file >=4MB.** Datto has 301 such files (~29.7 GB).
+   The old approach could never reach 3,768 no matter how many times it ran. Fix: proper Graph
+   **upload sessions** (10 MB chunks, `Content-Range`) for large files.
+2. **RMM agent ignores the `timeout` field; it honors `timeout_seconds`.** Commands sent with
+   `timeout` were capped at ~300 s, so long uploads died / went zombie ("running", no output).
+   Using `timeout_seconds` allowed multi-minute/multi-hour commands to run to completion.
+3. **Long paths (>260 chars) in the Datto tree** were handled with the `\\?\` prefix for file
+   reads.
+
+## Method
+
+- `clients/birth-biologic/scripts/enumerate-datto.ps1` — enumerate Datto, write a
+  `relpath|size` manifest to `C:\Windows\Temp\quality-manifest.txt` on ACG-DWP-X-BB.
+- `clients/birth-biologic/scripts/upload-quality-final.ps1` — idempotent uploader: for each
+  Datto file, skip if SharePoint already has it at matching size; else upload (simple PUT
+  <4MB, chunked upload session >=4MB). Long-path safe, refreshes the Graph token on long
+  runs, internal time budget + progress log (`C:\Windows\Temp\quality-upload.log`).
+- Ground truth verified from this machine via the Graph **delta** endpoint (whole-drive
+  enumeration; far fewer round-trips than recursive `children` calls).
+
+## Agent / drive
+
+- Agent: ACG-DWP-X-BB (`a4524e85-8a07-45d0-91b1-51ce7e2ca74a`)
+- Datto source: `C:\Users\Public\Desktop\Datto Workplace Server Projects\Quality Department`
+- SharePoint drive: `b!F8BzMb1YakCIWCyWlmczb09LHqtxDxVMpLT6kAwYmsM7NUY4oPLSRq7ng3tJq-E9`
+- Graph app: `msp-tools/computerguru-tenant-admin` (tenant 19a568e8-...)
+
+## Note on the earlier docs
+
+The earlier `2026-06-30-quality-sync-to-datto.md` (surgical 5-file delete) and
+`CONTINUE-QUALITY-SYNC.md` (wipe + re-upload, in progress) describe superseded intermediate
+states. This file is the authoritative final record.

clients/birth-biologic/scripts/enumerate-datto.ps1

@@ -0,0 +1,12 @@
+$ErrorActionPreference = 'SilentlyContinue'
+$source = 'C:\Users\Public\Desktop\Datto Workplace Server Projects\Quality Department'
+$files = Get-ChildItem -LiteralPath $source -Recurse -File
+$big = @($files | Where-Object { $_.Length -ge 4MB })
+$bytes = ($files | Measure-Object Length -Sum).Sum
+$manifest = $files | ForEach-Object { $_.FullName.Substring($source.Length + 1).Replace('\','/') + '|' + $_.Length }
+$manifest | Out-File -FilePath 'C:\Windows\Temp\quality-manifest.txt' -Encoding UTF8
+Write-Host "COUNT=$($files.Count)"
+Write-Host "BIG4MB=$($big.Count)"
+Write-Host "BYTES=$bytes"
+Write-Host "MANIFEST_LINES=$($manifest.Count)"
+Write-Host "MANIFEST_PATH=C:\Windows\Temp\quality-manifest.txt"

clients/birth-biologic/scripts/upload-quality-final.ps1

@@ -0,0 +1,101 @@
+# Birth Biologic - Quality Systems Department: converge SharePoint to Datto (3768 files)
+# Idempotent: skips files already present with matching size. Chunked upload sessions for >=4MB.
+# Long-path safe (\\?\). Refreshes Graph token on long runs. Writes progress to a tailable log.
+$ErrorActionPreference = 'Stop'
+
+$source       = 'C:\Users\Public\Desktop\Datto Workplace Server Projects\Quality Department'
+$driveId      = 'b!F8BzMb1YakCIWCyWlmczb09LHqtxDxVMpLT6kAwYmsM7NUY4oPLSRq7ng3tJq-E9'
+$tenantId     = '19a568e8-9e88-413b-9341-cbc224b39145'
+$clientId     = '709e6eed-0711-4875-9c44-2d3518c47063'
+$clientSecret = 'SECRET_PLACEHOLDER'
+$logPath      = 'C:\Windows\Temp\quality-upload.log'
+$manifestPath = 'C:\Windows\Temp\quality-manifest.txt'
+
+"=== upload start ===" | Set-Content -LiteralPath $logPath -Encoding UTF8
+function Log($m){ $line = (Get-Date -Format 'HH:mm:ss') + ' ' + $m; Write-Host $line; Add-Content -LiteralPath $logPath -Value $line }
+
+$script:token = $null
+$script:tokenAt = $null
+function Get-Token {
+  $body = @{ client_id=$clientId; client_secret=$clientSecret; scope='https://graph.microsoft.com/.default'; grant_type='client_credentials' }
+  $r = Invoke-RestMethod -Method Post -Uri "https://login.microsoftonline.com/$tenantId/oauth2/v2.0/token" -Body $body
+  $script:token = $r.access_token
+  $script:tokenAt = Get-Date
+}
+function Fresh-Token {
+  if ($null -eq $script:token -or ((Get-Date) - $script:tokenAt).TotalMinutes -gt 40) { Get-Token }
+  return $script:token
+}
+function Enc($rel){ ($rel.Split('/') | ForEach-Object { [uri]::EscapeDataString($_) }) -join '/' }
+function Long($p){ if ($p.StartsWith('\\?\')) { $p } else { '\\?\' + $p } }
+
+Get-Token
+Log "Token acquired."
+
+$lines = Get-Content -LiteralPath $manifestPath | Where-Object { $_ -and $_.Contains('|') }
+$items = foreach($l in $lines){
+  $idx = $l.LastIndexOf('|')
+  [pscustomobject]@{ rel = $l.Substring(0,$idx); size = [int64]$l.Substring($idx+1) }
+}
+# small files first so the visible count climbs fast, then the big ones
+$items = $items | Sort-Object size
+Log ("Manifest items: " + $items.Count)
+
+# Internal time budget: exit cleanly before any agent-side cap; re-dispatch resumes (idempotent).
+$deadline = (Get-Date).AddSeconds(9600)
+$timedOut = $false
+
+$uploaded=0; $skipped=0; $errors=0; $bigUp=0; $i=0
+foreach($it in $items){
+  $i++
+  if ((Get-Date) -gt $deadline){ $timedOut = $true; Log "TIME budget reached at item $i; stopping pass cleanly."; break }
+  $rel = $it.rel; $size = $it.size; $enc = Enc $rel
+  $tok = Fresh-Token
+  $h = @{ Authorization = "Bearer $tok" }
+
+  $exists=$false
+  try {
+    $meta = Invoke-RestMethod -Method Get -Uri "https://graph.microsoft.com/v1.0/drives/$driveId/root:/$enc" -Headers $h
+    if ([int64]$meta.size -eq $size) { $exists=$true }
+  } catch { $exists=$false }
+  if ($exists){ $skipped++; if($i % 250 -eq 0){ Log "  $i/$($items.Count) up=$uploaded skip=$skipped err=$errors big=$bigUp" }; continue }
+
+  $full = Long (Join-Path $source ($rel.Replace('/','\')))
+  try {
+    if ($size -lt 4194304){
+      $bytes = [System.IO.File]::ReadAllBytes($full)
+      $hh = @{ Authorization = "Bearer $tok"; 'Content-Type'='application/octet-stream' }
+      Invoke-RestMethod -Method Put -Uri "https://graph.microsoft.com/v1.0/drives/$driveId/root:/$enc`:/content" -Headers $hh -Body $bytes -UseBasicParsing | Out-Null
+      $uploaded++
+    } else {
+      $sessUri = "https://graph.microsoft.com/v1.0/drives/$driveId/root:/$enc`:/createUploadSession"
+      $sessBody = @{ item = @{ '@microsoft.graph.conflictBehavior'='replace' } } | ConvertTo-Json
+      $sess = Invoke-RestMethod -Method Post -Uri $sessUri -Headers @{ Authorization="Bearer $tok"; 'Content-Type'='application/json' } -Body $sessBody
+      $upUrl = $sess.uploadUrl
+      $fs = [System.IO.File]::Open($full,[System.IO.FileMode]::Open,[System.IO.FileAccess]::Read,[System.IO.FileShare]::Read)
+      try {
+        $chunk = 10485760  # 10 MB (multiple of 320 KiB)
+        $buf = New-Object byte[] $chunk
+        $pos = [int64]0
+        while ($pos -lt $size){
+          $read = $fs.Read($buf,0,$chunk)
+          if ($read -le 0){ break }
+          if ($read -eq $chunk){ $payload = $buf } else { $payload = New-Object byte[] $read; [Array]::Copy($buf,$payload,$read) }
+          $end = $pos + $read - 1
+          $ch = @{ 'Content-Range' = "bytes $pos-$end/$size" }
+          Invoke-RestMethod -Method Put -Uri $upUrl -Headers $ch -Body $payload | Out-Null
+          $pos += $read
+        }
+      } finally { $fs.Close() }
+      $uploaded++; $bigUp++
+      Log ("  BIG ok " + [math]::Round($size/1MB,1) + "MB: $rel")
+    }
+  } catch {
+    $errors++
+    Log "  ERROR: $rel :: $($_.Exception.Message)"
+  }
+  if($i % 100 -eq 0){ Log "  $i/$($items.Count) up=$uploaded skip=$skipped err=$errors big=$bigUp" }
+}
+$allDone = (-not $timedOut)
+Log "DONE up=$uploaded skip=$skipped err=$errors big=$bigUp total=$($items.Count) reachedEnd=$allDone"
+Write-Host "RESULT up=$uploaded skip=$skipped err=$errors big=$bigUp total=$($items.Count) reachedEnd=$allDone"

clients/birth-biologic/session-logs/2026-06/2026-06-30-mike-birthbio-quality-sync-complete.md

@@ -0,0 +1,141 @@
+# Birth Biologic — Quality Systems Department -> Datto SharePoint sync (COMPLETED)
+
+## User
+- **User:** Mike Swanson (mike)
+- **Machine:** GURU-5070
+- **Role:** admin
+
+## Session Summary
+
+Picked up the Birth Biologic Quality Systems Department -> Datto SharePoint migration after a
+Mac session (Mikes-MacBook-Air) worked it all day and failed to complete. The goal: make the
+SharePoint "Quality Systems Department" document library contain every file from the Datto
+Workplace source (3,768 files) on agent ACG-DWP-X-BB. Started by loading the pulled-in
+continuation docs and reconciling two contradictory records — an earlier one claiming a
+surgical 5-file delete "COMPLETE," and a later `CONTINUE-QUALITY-SYNC.md` describing a
+wipe-and-reupload stuck at 3,249/3,768.
+
+Rather than trust the stale docs, pulled live ground truth via the Graph delta endpoint:
+SharePoint held 3,337 files (only 17 >=4MB), gap of 431. Re-enumerated the Datto source via
+RMM, which revealed the actual scope the Mac missed: 3,768 files, of which 301 are >=4MB
+(~29.7 GB total, largest a 3.94 GB .mov). The Mac's upload script skipped every file >=4MB,
+so it could never converge — the entire large-file corpus (the bulk of the gap) was being
+ignored.
+
+Wrote a correct uploader (`upload-quality-final.ps1`): idempotent (skip a file if SharePoint
+already has it at matching size), simple PUT for <4MB and Graph chunked upload sessions for
+>=4MB, `\\?\` long-path handling, Graph token auto-refresh for long runs, an internal time
+budget, and a tailable progress log. Validated the risky path on the single largest file
+(3.94 GB) before the full run. That test also uncovered the second root cause: the RMM agent
+ignores the `timeout` field (caps ~300 s, producing the Mac's zombie "running" commands) and
+honors `timeout_seconds` instead.
+
+Cancelled the hung command `9e0fcfe8`, dispatched the full uploader with `timeout_seconds:
+10800`, and monitored to completion in the background. Result: 1,137 uploaded (283 large +
+854 small), 2,628 already-matching, 3 errors. The idempotent size-check also silently
+repaired ~700 files that existed but had a mismatched size (partial/corrupt residue from the
+earlier failed attempts).
+
+Verified the final state with a full Datto-vs-SharePoint diff: 0 Datto files missing from
+SharePoint (all 3,768 present), and exactly 4 differences — all live current work edited today
+by named client staff. Mike chose to preserve them. Documented the outcome in an authoritative
+completion doc, retired the contradictory continuation doc, logged the two lessons to
+errorlog, and saved two reference memories.
+
+## Key Decisions
+
+- Trusted live Graph/RMM enumeration over the committed docs — the docs contradicted each
+  other and were hours stale. Ground truth drove every decision.
+- Did not undo the Mac's wipe-and-reupload (already executed before pickup); finished the
+  rebuild additively (upload-only). Performed no deletions.
+- Fixed the >=4MB gap with proper chunked upload sessions rather than continuing to skip large
+  files (the actual reason the job never finished).
+- Validated the chunked-upload/long-path/Content-Range path on one real 3.94 GB file before
+  committing to a ~30 GB run.
+- Preserved the 4 live-work files (1 new xlsx + 3 open/locked docs edited today by staff)
+  rather than forcing a byte-identical match — overwriting would have destroyed today's work.
+  Confirmed with Mike.
+- Kept the client secret out of the repo (scripts carry `SECRET_PLACEHOLDER`; injected at
+  dispatch time from the vault into a scratch payload that was deleted).
+
+## Problems Encountered
+
+- **Upload could never reach 3,768:** prior script skipped all 301 files >=4MB (~29.7 GB).
+  Fixed with Graph chunked upload sessions (10 MB chunks, `Content-Range`).
+- **Zombie/hung RMM commands:** agent ignores `timeout`, capping at ~300 s. Fixed by using
+  `timeout_seconds` (both fields sent). Cancelled the stuck `9e0fcfe8` first.
+- **Recursive `/children` count timed out** (one HTTP call per folder). Switched to the Graph
+  `/root/delta` endpoint — whole drive in ~9 paged calls.
+- **`requests` not installed** on GURU-5070 Python. Rewrote checks with stdlib `urllib`.
+- **Long paths (>260) in Datto tree:** used the `\\?\` prefix for `[IO.File]` reads.
+- **3 upload errors (423 Locked / 409 Conflict):** files were locked because staff had them
+  open — live current work, correctly left untouched.
+- **SharePoint 3,769 vs Datto 3,768 (one extra):** diffed to `Temperature Excursion Log.xlsx`,
+  created today by client staff; preserved, not deleted.
+
+## Configuration Changes
+
+Created:
+- `clients/birth-biologic/scripts/enumerate-datto.ps1` — Datto enumeration + manifest writer.
+- `clients/birth-biologic/scripts/upload-quality-final.ps1` — idempotent chunked uploader
+  (carries `SECRET_PLACEHOLDER`, not a real secret).
+- `clients/birth-biologic/docs/migration/2026-06-30-quality-sync-COMPLETE.md` — authoritative
+  final record.
+- `.claude/memory/reference_gururmm_command_timeout_seconds.md`
+- `.claude/memory/reference_sharepoint_graph_large_file_upload.md`
+
+Modified:
+- `clients/birth-biologic/CONTINUE-QUALITY-SYNC.md` — replaced with a RESOLVED pointer.
+- `.claude/memory/MEMORY.md` — two new reference index lines.
+- `errorlog.md` — one `--friction` (timeout_seconds) + one `--correction` (>=4MB skip).
+
+## Credentials & Secrets
+
+- No new credentials created or discovered. Used existing vault entries only.
+- Graph app (client-credentials): vault `msp-tools/computerguru-tenant-admin`,
+  `credentials.client_secret`. Tenant `19a568e8-9e88-413b-9341-cbc224b39145`, client
+  `709e6eed-0711-4875-9c44-2d3518c47063`.
+- GuruRMM API: via `.claude/scripts/rmm-auth.sh` (vault
+  `infrastructure/gururmm-server.sops.yaml`).
+- Note: dispatching PowerShell that embeds the Graph secret leaves that secret in the RMM
+  command-history record (same as the prior Mac scripts). Acceptable here (internal RMM), but
+  a future hardening would have the agent fetch the secret itself.
+
+## Infrastructure & Servers
+
+- Agent: ACG-DWP-X-BB, id `a4524e85-8a07-45d0-91b1-51ce7e2ca74a`.
+- Datto source: `C:\Users\Public\Desktop\Datto Workplace Server Projects\Quality Department`
+  (3,768 files, 301 >=4MB, ~29.7 GB).
+- SharePoint drive: `b!F8BzMb1YakCIWCyWlmczb09LHqtxDxVMpLT6kAwYmsM7NUY4oPLSRq7ng3tJq-E9`.
+- RMM API (internal): `http://172.16.3.30:3001` (JWT via rmm-auth.sh).
+- VM work files: `C:\Windows\Temp\quality-manifest.txt`, `C:\Windows\Temp\quality-upload.log`.
+
+## Commands & Outputs
+
+- Count via delta: `GET /drives/{drive}/root/delta?$select=...&$top=500` — final live count 3,769.
+- Dispatch: `POST {RMM}/api/agents/{id}/command` with
+  `{command_type:"powershell", command:<script>, timeout_seconds:10800}`.
+- Poll: `GET {RMM}/api/commands/{id}` (status completed/failed/cancelled).
+- Cancel: `POST {RMM}/api/commands/{id}/cancel`.
+- Full run result: `up=1137 skip=2628 err=3 big=283 total=3768 reachedEnd=True`.
+- Errors: `409 Conflict` DEV39.Exhibit B.docx; `423 Locked` on both `...Current.docx` files.
+- Diff: 0 Datto files missing from SP; 1 extra in SP
+  (`LOGS/Equipment/3. Temp Excursions/Temperature Excursion Log.xlsx`, created 2026-06-30 by
+  Mary Ster, edited by Kristin Steen).
+
+## Pending / Incomplete Tasks
+
+- None functional. Migration complete and verified.
+- Optional future: parallel-stream uploader (multiple concurrent files + ~60 MiB chunks +
+  `Expect100Continue=$false`) to beat the ~40 Mbps single-session SharePoint Online ceiling on
+  large migrations.
+- Optional cleanup: remove `C:\Windows\Temp\quality-manifest.txt` / `quality-upload.log` from
+  ACG-DWP-X-BB (harmless, no secrets).
+
+## Reference Information
+
+- Authoritative doc: `clients/birth-biologic/docs/migration/2026-06-30-quality-sync-COMPLETE.md`
+- Command IDs: hung/cancelled `9e0fcfe8-0619-4a39-bd9c-6f5fd75c9b55`; full run
+  `4c978424-03cf-401c-805a-45162ff52be2`; big-file test
+  `fb3a6c4b-7158-4b77-9988-4326503753d8`.
+- Memories: `gururmm-command-timeout-seconds`, `sharepoint-graph-large-file-upload`.

errorlog.md

@@ -17,6 +17,10 @@ Categories (the `[type]` tag): _(none)_ = skill/command execution failure ·
 
 <!-- Append entries below this line -->
 
+2026-07-01 | GURU-5070 | birth-biologic/quality-upload | [correction] prior upload script silently skipped every file >=4MB; Datto had 301 such files (~29.7GB) so it could never reach target count. Fix: Graph chunked upload sessions for >=4MB.
+
+2026-07-01 | GURU-5070 | rmm/command-dispatch | [friction] RMM agent ignores 'timeout' field (capped ~300s -> zombie 'running' commands); long-running commands MUST use 'timeout_seconds'. Cost a full day of failed Birth Biologic uploads. [ctx: agent=ACG-DWP-X-BB ref=gururmm-command-timeout-seconds]
+
 2026-06-30 | Howard-Home | remediation-tool/exchange-op | [friction] Add-MailboxPermission -AutoMapping $true silently rolled back the FullAccess grant for 2 of 4 delegates (cmdlet echoed [FullAccess] success but Get-MailboxPermission showed NONE); a failed msExchDelegateListLink write aborts the whole Add transaction. Fix: re-add with -AutoMapping $false (FullAccess then persists); set automapping separately/interactively if auto-attach is required. [ctx: tenant=cascadestucson.com mailbox=tamra.matthews app=ComputerGuru-Exchange-Operator]
 
 2026-06-30 | Howard-Home | /syncro | [correction] invoiced 'Windows Pro Upgrade' line items (Cascades 67887/67890) with blank CATEGORY; product_category was null and I billed it anyway — correct is to pre-flight GET /products/<id>, never invoice a null/blank category, and never invent one (use existing set e.g. Software) [ctx: ref=feedback_syncro_line_item_category invoices=67887,67890 product=23571919]