azcomputerguru/claudetools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

sync: auto-sync from GURU-5070 at 2026-06-30 17:21:06

Browse Source 
Author: Mike Swanson
Machine: GURU-5070
Timestamp: 2026-06-30 17:21:06
This commit is contained in:
Mike Swanson
2026-06-30 17:21:47 -07:00
parent 1b0b313896
commit 01613697c6
9 changed files with 386 additions and 268 deletions
Download Patch File Download Diff File Expand all files Collapse all files

277
clients/birth-biologic/CONTINUE-QUALITY-SYNC.md
View File

@@ -1,271 +1,12 @@
# Birth Biologic Quality Department Sync - CONTINUATION INSTRUCTIONS
# Birth Biologic Quality Department Sync — RESOLVED
**Date:** 2026-06-30
**Status:** IN PROGRESS - Upload script running
**Client:** Birth Biologic
**Task:** Sync SharePoint Quality Systems Department to match Datto exactly (3,768 files)
**Status:** COMPLETE (2026-06-30, GURU-5070). This continuation file is obsolete — do NOT
act on the instructions that used to be here.
---
Authoritative record: `docs/migration/2026-06-30-quality-sync-COMPLETE.md`.
## CURRENT STATE
**SharePoint Status:**
- Current file count: 3,249 files
- Target file count: 3,768 files (from Datto)
- Gap: 519 files remaining
**Active RMM Command:**
- Command ID: `9e0fcfe8-0619-4a39-bd9c-6f5fd75c9b55`
- Agent: ACG-DWP-X-BB (a4524e85-8a07-45d0-91b1-51ce7e2ca74a)
- Status: Running (as of last check)
- Purpose: Upload all 3,768 files from Datto to SharePoint via Graph API
- Drive ID concatenation workaround: `"b" + "!" + "..."` to avoid PowerShell escaping
**Background Monitor:**
- Task ID: b24474c (monitoring upload every minute for 20 minutes)
---
## WHAT HAPPENED
1. **Initial Approach:** Tried OneDrive sync by robocopy to local OneDrive folder
- Result: Synced 3,249 files then STALLED (no progress for 35+ minutes)
2. **Switched to Direct Graph API Upload:**
- Multiple attempts failed due to PowerShell escaping the `!` in drive ID
- Drive ID: `b!F8BzMb1YakCIWCyWlmczb09LHqtxDxVMpLT6kAwYmsM7NUY4oPLSRq7ng3tJq-E9`
- Problem: PowerShell kept converting `b!` to `b\!` causing HTTP 400 errors
- Solution: Concatenate at runtime: `$driveId = "b" + "!" + "F8Bz..."`
3. **Current Upload:**
- Command dispatched successfully with drive ID concatenation
- Script has been running but showing no output yet (may still be scanning files)
---
## CREDENTIALS
**Graph API (from vault):**
- Path: `msp-tools/computerguru-tenant-admin`
- Tenant ID: 19a568e8-9e88-413b-9341-cbc224b39145
- Client ID: 709e6eed-0711-4875-9c44-2d3518c47063
- Client Secret: (in vault at `credentials.client_secret`)
**GuruRMM:**
- Vault path: `infrastructure/gururmm-server.sops.yaml`
- Agent ID: a4524e85-8a07-45d0-91b1-51ce7e2ca74a (ACG-DWP-X-BB)
**SharePoint Drive ID:**
- `b!F8BzMb1YakCIWCyWlmczb09LHqtxDxVMpLT6kAwYmsM7NUY4oPLSRq7ng3tJq-E9`
---
## NEXT STEPS TO CONTINUE
### Option 1: Check if current upload completed
```bash
# Authenticate to RMM
eval "$(bash .claude/scripts/rmm-auth.sh)"
# Check upload status
curl -s "$RMM/api/commands/9e0fcfe8-0619-4a39-bd9c-6f5fd75c9b55" \
-H "Authorization: Bearer $TOKEN" > /tmp/upload-status.json
python3 -c "
import json
with open('/tmp/upload-status.json') as f:
data = json.load(f)
print(f\"Status: {data.get('status')}\")
print(f\"Exit code: {data.get('exit_code')}\")
stdout = data.get('stdout', '') or ''
if len(stdout) > 0:
print('\n--- Last 30 lines ---')
for line in stdout.split('\n')[-30:]:
print(line)
"
# Then verify SharePoint count
python3 clients/birth-biologic/scripts/check-quality-status.py
```
### Option 2: If upload failed, use the working PowerShell script
The correct script is in: `clients/birth-biologic/scripts/upload-final-working.ps1`
Run via RMM:
```bash
eval "$(bash .claude/scripts/rmm-auth.sh)"
AGENT_ID="a4524e85-8a07-45d0-91b1-51ce7e2ca74a"
CLIENT_SECRET=$(bash .claude/scripts/vault.sh get-field msp-tools/computerguru-tenant-admin credentials.client_secret)
# Script content with proper drive ID concatenation
SCRIPT='... (see upload-final-working.ps1) ...'
PAYLOAD=$(jq -n --arg cmd "$SCRIPT" '{command_type: "powershell", command: $cmd, timeout_seconds: 1800}')
curl -s -X POST "$RMM/api/agents/$AGENT_ID/command" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d "$PAYLOAD" | jq -r '.command_id'
```
---
## FILES CREATED
**Scripts:**
- `clients/birth-biologic/scripts/check-quality-status.py` - Check SharePoint file count
- `clients/birth-biologic/scripts/upload-datto-to-sharepoint.ps1` - Base upload script
- `clients/birth-biologic/scripts/reset-quality-exact.py` - Initial reset script (used)
- `clients/birth-biologic/scripts/exact-sync-quality.py` - Robocopy approach
- `clients/birth-biologic/scripts/sync-quality-simple.py` - Earlier attempt
- `clients/birth-biologic/scripts/finish-upload.py` - Bulk upload attempt
- `clients/birth-biologic/scripts/upload-remaining-files.py` - Remaining files upload
**Todo List:**
1. [completed] Delete ALL files from SharePoint Quality Systems Department
2. [in_progress] Copy ALL files from Datto to SharePoint exactly as they exist
3. [pending] Verify SharePoint has exactly 3768 files matching Datto
---
## DATTO SOURCE PATH
**On ACG-DWP-X-BB:**
```
C:\Users\Public\Desktop\Datto Workplace Server Projects\Quality Department
```
**Total files:** 3,768 files (verified via Get-ChildItem -Recurse)
**Files >4MB:** ~63 files (these are being skipped - need separate large file upload later)
---
## VERIFICATION COMMAND
Once upload completes:
```python
python3 clients/birth-biologic/scripts/check-quality-status.py
```
Expected output:
```
SharePoint: 3768 files
Datto: 3768 files
Gap: 0 files
[OK] MATCH - SharePoint has exactly 3768 files
```
---
## TROUBLESHOOTING
**If upload shows 0 uploaded, 3467 errors:**
- This means drive ID was escaped wrong (b\! instead of b!)
- Solution: Use string concatenation `"b" + "!" + "..."`
**If upload hangs with no output:**
- PowerShell may have syntax error
- Check command_text field to verify script sent correctly
**If you need to cancel running command:**
```bash
curl -s -X POST "$RMM/api/commands/9e0fcfe8-0619-4a39-bd9c-6f5fd75c9b55/cancel" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json"
```
---
## WORKING UPLOAD SCRIPT TEMPLATE
Save this as `upload-final-working.ps1`:
```powershell
$source = "C:\Users\Public\Desktop\Datto Workplace Server Projects\Quality Department"
$driveId = "b" + "!" + "F8BzMb1YakCIWCyWlmczb09LHqtxDxVMpLT6kAwYmsM7NUY4oPLSRq7ng3tJq-E9"
$tenantId = "19a568e8-9e88-413b-9341-cbc224b39145"
$clientId = "709e6eed-0711-4875-9c44-2d3518c47063"
$clientSecret = "GET_FROM_VAULT"
Write-Host "Getting Graph API token..."
$tokenBody = @{
client_id = $clientId
client_secret = $clientSecret
scope = "https://graph.microsoft.com/.default"
grant_type = "client_credentials"
}
$tokenResponse = Invoke-RestMethod -Method Post -Uri "https://login.microsoftonline.com/$tenantId/oauth2/v2.0/token" -Body $tokenBody
$token = $tokenResponse.access_token
Write-Host "[OK] Token acquired"
Write-Host "Scanning Datto files..."
$files = Get-ChildItem $source -Recurse -File -ErrorAction SilentlyContinue
Write-Host "[OK] Found $($files.Count) files"
Write-Host "Uploading files..."
$uploaded = 0
$errors = 0
foreach ($file in $files) {
$relativePath = $file.FullName.Substring($source.Length + 1)
$uploadPath = $relativePath.Replace("\", "/")
try {
if ($file.Length -lt 4MB) {
$uploadUrl = "https://graph.microsoft.com/v1.0/drives/$driveId/root:/$uploadPath" + ":/content"
$headers = @{
"Authorization" = "Bearer $token"
"Content-Type" = "application/octet-stream"
}
$fileBytes = [System.IO.File]::ReadAllBytes($file.FullName)
Invoke-RestMethod -Method Put -Uri $uploadUrl -Headers $headers -Body $fileBytes -UseBasicParsing | Out-Null
$uploaded++
if ($uploaded % 100 -eq 0) {
Write-Host " Uploaded $uploaded files..."
}
}
} catch {
$errors++
}
}
Write-Host ""
Write-Host "Upload Complete"
Write-Host "Uploaded: $uploaded"
Write-Host "Errors: $errors"
Write-Host "Total: $($files.Count)"
```
---
## SESSION NOTES
- OneDrive sync is unreliable for large migrations (stalled at 86%)
- Direct Graph API upload is the correct approach
- PowerShell exclamation mark escaping is a major gotcha
- Use string concatenation to avoid escape issues
- Files >4MB need separate upload session logic (not implemented yet)
- There are ~63 large files that will need separate handling
**Client was promised this yesterday - NOW VERY ANGRY**
---
## QUICK RESUME CHECKLIST
1. [ ] Check if command 9e0fcfe8 completed
2. [ ] Verify SharePoint file count (should be 3,768)
3. [ ] If not complete, dispatch new upload with working script
4. [ ] Monitor for completion (15-20 minutes)
5. [ ] Verify final count matches Datto
6. [ ] Handle large files (>4MB) if needed
7. [ ] Document completion
**END OF CONTINUATION INSTRUCTIONS**
Summary: all 3,768 Datto files are present in SharePoint (verified via Graph delta). The
only differences are 4 files that are live current work edited today by client staff
(1 new xlsx + 3 open docs), intentionally preserved per Mike. The prior approach failed all
day because its script skipped every file >=4MB (301 files, ~29.7 GB) and used the RMM
`timeout` field (capped ~300s) instead of `timeout_seconds`.

70
clients/birth-biologic/docs/migration/2026-06-30-quality-sync-COMPLETE.md Normal file
View File

@@ -0,0 +1,70 @@
# Birth Biologic — Quality Systems Department -> Datto sync — COMPLETE
**Date:** 2026-06-30
**Completed by:** Mike Swanson / claude-main (GURU-5070)
**Status:** COMPLETE — verified against live Graph enumeration
## Outcome
Every file in the Datto source is now present in SharePoint.
| | Datto (source) | SharePoint (final) |
|---|---:|---:|
| Files | 3,768 | 3,769 |
| Datto files missing from SharePoint | — | **0** |
All 301 large files (>=4MB, ~29.7 GB total, largest a 3.94 GB .mov) were uploaded via
Graph chunked upload sessions. The idempotent size-check pass also detected and re-uploaded
~700 files that existed in SharePoint with a mismatched size (partial/corrupt residue from
the earlier failed OneDrive/PUT attempts).
## The 4 intentional differences (live current work — PRESERVED per Mike, 2026-06-30)
These are the "recently modified, do not clobber" carve-out. Confirmed as live edits made
*today* by named client staff, so they were left intact rather than forced to match Datto:
| File | Datto size | SharePoint size | Note |
|---|---:|---:|---|
| `LOGS/Equipment/3. Temp Excursions/Temperature Excursion Log.xlsx` | not present | 40,253 | Created 2026-06-30 by Mary Ster, edited by Kristin Steen |
| `LOGS/Equipment/2. Validation List/QSP-200.003.A Validation List.Current.docx` | 41,908 | 51,510 | Locked/open, edited today |
| `LOGS/Equipment/1.Equipment List/QSP-200.001.C Equipment List.Current.docx` | 47,947 | 57,545 | Locked/open, edited today |
| `LOGS/Quality Assurance Reporting Log/Deviations/2024/DEV39.Exhibit B.docx` | 15,525 | 22,437 | In use, edited today |
The three docs surfaced as upload "errors" (HTTP 423 Locked / 409 Conflict) precisely
*because* staff had them open — SharePoint's lock protected live work. That is the correct
result, not a failure.
## What actually fixed it (root cause of the all-day Mac failures)
1. **The prior upload script skipped every file >=4MB.** Datto has 301 such files (~29.7 GB).
The old approach could never reach 3,768 no matter how many times it ran. Fix: proper Graph
**upload sessions** (10 MB chunks, `Content-Range`) for large files.
2. **RMM agent ignores the `timeout` field; it honors `timeout_seconds`.** Commands sent with
`timeout` were capped at ~300 s, so long uploads died / went zombie ("running", no output).
Using `timeout_seconds` allowed multi-minute/multi-hour commands to run to completion.
3. **Long paths (>260 chars) in the Datto tree** were handled with the `\\?\` prefix for file
reads.
## Method
- `clients/birth-biologic/scripts/enumerate-datto.ps1` — enumerate Datto, write a
`relpath|size` manifest to `C:\Windows\Temp\quality-manifest.txt` on ACG-DWP-X-BB.
- `clients/birth-biologic/scripts/upload-quality-final.ps1` — idempotent uploader: for each
Datto file, skip if SharePoint already has it at matching size; else upload (simple PUT
<4MB, chunked upload session >=4MB). Long-path safe, refreshes the Graph token on long
runs, internal time budget + progress log (`C:\Windows\Temp\quality-upload.log`).
- Ground truth verified from this machine via the Graph **delta** endpoint (whole-drive
enumeration; far fewer round-trips than recursive `children` calls).
## Agent / drive
- Agent: ACG-DWP-X-BB (`a4524e85-8a07-45d0-91b1-51ce7e2ca74a`)
- Datto source: `C:\Users\Public\Desktop\Datto Workplace Server Projects\Quality Department`
- SharePoint drive: `b!F8BzMb1YakCIWCyWlmczb09LHqtxDxVMpLT6kAwYmsM7NUY4oPLSRq7ng3tJq-E9`
- Graph app: `msp-tools/computerguru-tenant-admin` (tenant 19a568e8-...)
## Note on the earlier docs
The earlier `2026-06-30-quality-sync-to-datto.md` (surgical 5-file delete) and
`CONTINUE-QUALITY-SYNC.md` (wipe + re-upload, in progress) describe superseded intermediate
states. This file is the authoritative final record.

12
clients/birth-biologic/scripts/enumerate-datto.ps1 Normal file
View File

@@ -0,0 +1,12 @@
$ErrorActionPreference = 'SilentlyContinue'
$source = 'C:\Users\Public\Desktop\Datto Workplace Server Projects\Quality Department'
$files = Get-ChildItem -LiteralPath $source -Recurse -File
$big = @($files | Where-Object { $_.Length -ge 4MB })
$bytes = ($files | Measure-Object Length -Sum).Sum
$manifest = $files | ForEach-Object { $_.FullName.Substring($source.Length + 1).Replace('\','/') + '|' + $_.Length }
$manifest | Out-File -FilePath 'C:\Windows\Temp\quality-manifest.txt' -Encoding UTF8
Write-Host "COUNT=$($files.Count)"
Write-Host "BIG4MB=$($big.Count)"
Write-Host "BYTES=$bytes"
Write-Host "MANIFEST_LINES=$($manifest.Count)"
Write-Host "MANIFEST_PATH=C:\Windows\Temp\quality-manifest.txt"

101
clients/birth-biologic/scripts/upload-quality-final.ps1 Normal file
View File

@@ -0,0 +1,101 @@
# Birth Biologic - Quality Systems Department: converge SharePoint to Datto (3768 files)
# Idempotent: skips files already present with matching size. Chunked upload sessions for >=4MB.
# Long-path safe (\\?\). Refreshes Graph token on long runs. Writes progress to a tailable log.
$ErrorActionPreference = 'Stop'
$source = 'C:\Users\Public\Desktop\Datto Workplace Server Projects\Quality Department'
$driveId = 'b!F8BzMb1YakCIWCyWlmczb09LHqtxDxVMpLT6kAwYmsM7NUY4oPLSRq7ng3tJq-E9'
$tenantId = '19a568e8-9e88-413b-9341-cbc224b39145'
$clientId = '709e6eed-0711-4875-9c44-2d3518c47063'
$clientSecret = 'SECRET_PLACEHOLDER'
$logPath = 'C:\Windows\Temp\quality-upload.log'
$manifestPath = 'C:\Windows\Temp\quality-manifest.txt'
"=== upload start ===" | Set-Content -LiteralPath $logPath -Encoding UTF8
function Log($m){ $line = (Get-Date -Format 'HH:mm:ss') + ' ' + $m; Write-Host $line; Add-Content -LiteralPath $logPath -Value $line }
$script:token = $null
$script:tokenAt = $null
function Get-Token {
$body = @{ client_id=$clientId; client_secret=$clientSecret; scope='https://graph.microsoft.com/.default'; grant_type='client_credentials' }
$r = Invoke-RestMethod -Method Post -Uri "https://login.microsoftonline.com/$tenantId/oauth2/v2.0/token" -Body $body
$script:token = $r.access_token
$script:tokenAt = Get-Date
}
function Fresh-Token {
if ($null -eq $script:token -or ((Get-Date) - $script:tokenAt).TotalMinutes -gt 40) { Get-Token }
return $script:token
}
function Enc($rel){ ($rel.Split('/') | ForEach-Object { [uri]::EscapeDataString($_) }) -join '/' }
function Long($p){ if ($p.StartsWith('\\?\')) { $p } else { '\\?\' + $p } }
Get-Token
Log "Token acquired."
$lines = Get-Content -LiteralPath $manifestPath | Where-Object { $_ -and $_.Contains('|') }
$items = foreach($l in $lines){
$idx = $l.LastIndexOf('|')
[pscustomobject]@{ rel = $l.Substring(0,$idx); size = [int64]$l.Substring($idx+1) }
}
# small files first so the visible count climbs fast, then the big ones
$items = $items | Sort-Object size
Log ("Manifest items: " + $items.Count)
# Internal time budget: exit cleanly before any agent-side cap; re-dispatch resumes (idempotent).
$deadline = (Get-Date).AddSeconds(9600)
$timedOut = $false
$uploaded=0; $skipped=0; $errors=0; $bigUp=0; $i=0
foreach($it in $items){
$i++
if ((Get-Date) -gt $deadline){ $timedOut = $true; Log "TIME budget reached at item $i; stopping pass cleanly."; break }
$rel = $it.rel; $size = $it.size; $enc = Enc $rel
$tok = Fresh-Token
$h = @{ Authorization = "Bearer $tok" }
$exists=$false
try {
$meta = Invoke-RestMethod -Method Get -Uri "https://graph.microsoft.com/v1.0/drives/$driveId/root:/$enc" -Headers $h
if ([int64]$meta.size -eq $size) { $exists=$true }
} catch { $exists=$false }
if ($exists){ $skipped++; if($i % 250 -eq 0){ Log " $i/$($items.Count) up=$uploaded skip=$skipped err=$errors big=$bigUp" }; continue }
$full = Long (Join-Path $source ($rel.Replace('/','\')))
try {
if ($size -lt 4194304){
$bytes = [System.IO.File]::ReadAllBytes($full)
$hh = @{ Authorization = "Bearer $tok"; 'Content-Type'='application/octet-stream' }
Invoke-RestMethod -Method Put -Uri "https://graph.microsoft.com/v1.0/drives/$driveId/root:/$enc`:/content" -Headers $hh -Body $bytes -UseBasicParsing | Out-Null
$uploaded++
} else {
$sessUri = "https://graph.microsoft.com/v1.0/drives/$driveId/root:/$enc`:/createUploadSession"
$sessBody = @{ item = @{ '@microsoft.graph.conflictBehavior'='replace' } } | ConvertTo-Json
$sess = Invoke-RestMethod -Method Post -Uri $sessUri -Headers @{ Authorization="Bearer $tok"; 'Content-Type'='application/json' } -Body $sessBody
$upUrl = $sess.uploadUrl
$fs = [System.IO.File]::Open($full,[System.IO.FileMode]::Open,[System.IO.FileAccess]::Read,[System.IO.FileShare]::Read)
try {
$chunk = 10485760 # 10 MB (multiple of 320 KiB)
$buf = New-Object byte[] $chunk
$pos = [int64]0
while ($pos -lt $size){
$read = $fs.Read($buf,0,$chunk)
if ($read -le 0){ break }
if ($read -eq $chunk){ $payload = $buf } else { $payload = New-Object byte[] $read; [Array]::Copy($buf,$payload,$read) }
$end = $pos + $read - 1
$ch = @{ 'Content-Range' = "bytes $pos-$end/$size" }
Invoke-RestMethod -Method Put -Uri $upUrl -Headers $ch -Body $payload | Out-Null
$pos += $read
}
} finally { $fs.Close() }
$uploaded++; $bigUp++
Log (" BIG ok " + [math]::Round($size/1MB,1) + "MB: $rel")
}
} catch {
$errors++
Log " ERROR: $rel :: $($_.Exception.Message)"
}
if($i % 100 -eq 0){ Log " $i/$($items.Count) up=$uploaded skip=$skipped err=$errors big=$bigUp" }
}
$allDone = (-not $timedOut)
Log "DONE up=$uploaded skip=$skipped err=$errors big=$bigUp total=$($items.Count) reachedEnd=$allDone"
Write-Host "RESULT up=$uploaded skip=$skipped err=$errors big=$bigUp total=$($items.Count) reachedEnd=$allDone"

141
clients/birth-biologic/session-logs/2026-06/2026-06-30-mike-birthbio-quality-sync-complete.md Normal file
View File

@@ -0,0 +1,141 @@
# Birth Biologic — Quality Systems Department -> Datto SharePoint sync (COMPLETED)
## User
- **User:** Mike Swanson (mike)
- **Machine:** GURU-5070
- **Role:** admin
## Session Summary
Picked up the Birth Biologic Quality Systems Department -> Datto SharePoint migration after a
Mac session (Mikes-MacBook-Air) worked it all day and failed to complete. The goal: make the
SharePoint "Quality Systems Department" document library contain every file from the Datto
Workplace source (3,768 files) on agent ACG-DWP-X-BB. Started by loading the pulled-in
continuation docs and reconciling two contradictory records — an earlier one claiming a
surgical 5-file delete "COMPLETE," and a later `CONTINUE-QUALITY-SYNC.md` describing a
wipe-and-reupload stuck at 3,249/3,768.
Rather than trust the stale docs, pulled live ground truth via the Graph delta endpoint:
SharePoint held 3,337 files (only 17 >=4MB), gap of 431. Re-enumerated the Datto source via
RMM, which revealed the actual scope the Mac missed: 3,768 files, of which 301 are >=4MB
(~29.7 GB total, largest a 3.94 GB .mov). The Mac's upload script skipped every file >=4MB,
so it could never converge — the entire large-file corpus (the bulk of the gap) was being
ignored.
Wrote a correct uploader (`upload-quality-final.ps1`): idempotent (skip a file if SharePoint
already has it at matching size), simple PUT for <4MB and Graph chunked upload sessions for
>=4MB, `\\?\` long-path handling, Graph token auto-refresh for long runs, an internal time
budget, and a tailable progress log. Validated the risky path on the single largest file
(3.94 GB) before the full run. That test also uncovered the second root cause: the RMM agent
ignores the `timeout` field (caps ~300 s, producing the Mac's zombie "running" commands) and
honors `timeout_seconds` instead.
Cancelled the hung command `9e0fcfe8`, dispatched the full uploader with `timeout_seconds:
10800`, and monitored to completion in the background. Result: 1,137 uploaded (283 large +
854 small), 2,628 already-matching, 3 errors. The idempotent size-check also silently
repaired ~700 files that existed but had a mismatched size (partial/corrupt residue from the
earlier failed attempts).
Verified the final state with a full Datto-vs-SharePoint diff: 0 Datto files missing from
SharePoint (all 3,768 present), and exactly 4 differences — all live current work edited today
by named client staff. Mike chose to preserve them. Documented the outcome in an authoritative
completion doc, retired the contradictory continuation doc, logged the two lessons to
errorlog, and saved two reference memories.
## Key Decisions
- Trusted live Graph/RMM enumeration over the committed docs — the docs contradicted each
other and were hours stale. Ground truth drove every decision.
- Did not undo the Mac's wipe-and-reupload (already executed before pickup); finished the
rebuild additively (upload-only). Performed no deletions.
- Fixed the >=4MB gap with proper chunked upload sessions rather than continuing to skip large
files (the actual reason the job never finished).
- Validated the chunked-upload/long-path/Content-Range path on one real 3.94 GB file before
committing to a ~30 GB run.
- Preserved the 4 live-work files (1 new xlsx + 3 open/locked docs edited today by staff)
rather than forcing a byte-identical match — overwriting would have destroyed today's work.
Confirmed with Mike.
- Kept the client secret out of the repo (scripts carry `SECRET_PLACEHOLDER`; injected at
dispatch time from the vault into a scratch payload that was deleted).
## Problems Encountered
- **Upload could never reach 3,768:** prior script skipped all 301 files >=4MB (~29.7 GB).
Fixed with Graph chunked upload sessions (10 MB chunks, `Content-Range`).
- **Zombie/hung RMM commands:** agent ignores `timeout`, capping at ~300 s. Fixed by using
`timeout_seconds` (both fields sent). Cancelled the stuck `9e0fcfe8` first.
- **Recursive `/children` count timed out** (one HTTP call per folder). Switched to the Graph
`/root/delta` endpoint — whole drive in ~9 paged calls.
- **`requests` not installed** on GURU-5070 Python. Rewrote checks with stdlib `urllib`.
- **Long paths (>260) in Datto tree:** used the `\\?\` prefix for `[IO.File]` reads.
- **3 upload errors (423 Locked / 409 Conflict):** files were locked because staff had them
open — live current work, correctly left untouched.
- **SharePoint 3,769 vs Datto 3,768 (one extra):** diffed to `Temperature Excursion Log.xlsx`,
created today by client staff; preserved, not deleted.
## Configuration Changes
Created:
- `clients/birth-biologic/scripts/enumerate-datto.ps1` — Datto enumeration + manifest writer.
- `clients/birth-biologic/scripts/upload-quality-final.ps1` — idempotent chunked uploader
(carries `SECRET_PLACEHOLDER`, not a real secret).
- `clients/birth-biologic/docs/migration/2026-06-30-quality-sync-COMPLETE.md` — authoritative
final record.
- `.claude/memory/reference_gururmm_command_timeout_seconds.md`
- `.claude/memory/reference_sharepoint_graph_large_file_upload.md`
Modified:
- `clients/birth-biologic/CONTINUE-QUALITY-SYNC.md` — replaced with a RESOLVED pointer.
- `.claude/memory/MEMORY.md` — two new reference index lines.
- `errorlog.md` — one `--friction` (timeout_seconds) + one `--correction` (>=4MB skip).
## Credentials & Secrets
- No new credentials created or discovered. Used existing vault entries only.
- Graph app (client-credentials): vault `msp-tools/computerguru-tenant-admin`,
`credentials.client_secret`. Tenant `19a568e8-9e88-413b-9341-cbc224b39145`, client
`709e6eed-0711-4875-9c44-2d3518c47063`.
- GuruRMM API: via `.claude/scripts/rmm-auth.sh` (vault
`infrastructure/gururmm-server.sops.yaml`).
- Note: dispatching PowerShell that embeds the Graph secret leaves that secret in the RMM
command-history record (same as the prior Mac scripts). Acceptable here (internal RMM), but
a future hardening would have the agent fetch the secret itself.
## Infrastructure & Servers
- Agent: ACG-DWP-X-BB, id `a4524e85-8a07-45d0-91b1-51ce7e2ca74a`.
- Datto source: `C:\Users\Public\Desktop\Datto Workplace Server Projects\Quality Department`
(3,768 files, 301 >=4MB, ~29.7 GB).
- SharePoint drive: `b!F8BzMb1YakCIWCyWlmczb09LHqtxDxVMpLT6kAwYmsM7NUY4oPLSRq7ng3tJq-E9`.
- RMM API (internal): `http://172.16.3.30:3001` (JWT via rmm-auth.sh).
- VM work files: `C:\Windows\Temp\quality-manifest.txt`, `C:\Windows\Temp\quality-upload.log`.
## Commands & Outputs
- Count via delta: `GET /drives/{drive}/root/delta?$select=...&$top=500` — final live count 3,769.
- Dispatch: `POST {RMM}/api/agents/{id}/command` with
`{command_type:"powershell", command:<script>, timeout_seconds:10800}`.
- Poll: `GET {RMM}/api/commands/{id}` (status completed/failed/cancelled).
- Cancel: `POST {RMM}/api/commands/{id}/cancel`.
- Full run result: `up=1137 skip=2628 err=3 big=283 total=3768 reachedEnd=True`.
- Errors: `409 Conflict` DEV39.Exhibit B.docx; `423 Locked` on both `...Current.docx` files.
- Diff: 0 Datto files missing from SP; 1 extra in SP
(`LOGS/Equipment/3. Temp Excursions/Temperature Excursion Log.xlsx`, created 2026-06-30 by
Mary Ster, edited by Kristin Steen).
## Pending / Incomplete Tasks
- None functional. Migration complete and verified.
- Optional future: parallel-stream uploader (multiple concurrent files + ~60 MiB chunks +
`Expect100Continue=$false`) to beat the ~40 Mbps single-session SharePoint Online ceiling on
large migrations.
- Optional cleanup: remove `C:\Windows\Temp\quality-manifest.txt` / `quality-upload.log` from
ACG-DWP-X-BB (harmless, no secrets).
## Reference Information
- Authoritative doc: `clients/birth-biologic/docs/migration/2026-06-30-quality-sync-COMPLETE.md`
- Command IDs: hung/cancelled `9e0fcfe8-0619-4a39-bd9c-6f5fd75c9b55`; full run
`4c978424-03cf-401c-805a-45162ff52be2`; big-file test
`fb3a6c4b-7158-4b77-9988-4326503753d8`.
- Memories: `gururmm-command-timeout-seconds`, `sharepoint-graph-large-file-upload`.