sync: auto-sync from GURU-5070 at 2026-06-30 17:21:06

Author: Mike Swanson Machine: GURU-5070 Timestamp: 2026-06-30 17:21:06
2026-06-30 17:21:47 -07:00
parent 1b0b313896
commit 01613697c6
9 changed files with 386 additions and 268 deletions
--- a/clients/birth-biologic/CONTINUE-QUALITY-SYNC.md
+++ b/clients/birth-biologic/CONTINUE-QUALITY-SYNC.md
@@ -1,271 +1,12 @@
-# Birth Biologic Quality Department Sync - CONTINUATION INSTRUCTIONS
+# Birth Biologic Quality Department Sync — RESOLVED

-**Date:** 2026-06-30
-**Status:** IN PROGRESS - Upload script running
-**Client:** Birth Biologic
-**Task:** Sync SharePoint Quality Systems Department to match Datto exactly (3,768 files)
+**Status:** COMPLETE (2026-06-30, GURU-5070). This continuation file is obsolete — do NOT
+act on the instructions that used to be here.

---
+Authoritative record: `docs/migration/2026-06-30-quality-sync-COMPLETE.md`.

-## CURRENT STATE
-
-**SharePoint Status:**
- Current file count: 3,249 files
- Target file count: 3,768 files (from Datto)
- Gap: 519 files remaining
-
-**Active RMM Command:**
- Command ID: `9e0fcfe8-0619-4a39-bd9c-6f5fd75c9b55`
- Agent: ACG-DWP-X-BB (a4524e85-8a07-45d0-91b1-51ce7e2ca74a)
- Status: Running (as of last check)
- Purpose: Upload all 3,768 files from Datto to SharePoint via Graph API
- Drive ID concatenation workaround: `"b" + "!" + "..."` to avoid PowerShell escaping
-
-**Background Monitor:**
- Task ID: b24474c (monitoring upload every minute for 20 minutes)
-
---
-
-## WHAT HAPPENED
-
-1. **Initial Approach:** Tried OneDrive sync by robocopy to local OneDrive folder
-   - Result: Synced 3,249 files then STALLED (no progress for 35+ minutes)
-
-2. **Switched to Direct Graph API Upload:**
-   - Multiple attempts failed due to PowerShell escaping the `!` in drive ID
-   - Drive ID: `b!F8BzMb1YakCIWCyWlmczb09LHqtxDxVMpLT6kAwYmsM7NUY4oPLSRq7ng3tJq-E9`
-   - Problem: PowerShell kept converting `b!` to `b\!` causing HTTP 400 errors
-   - Solution: Concatenate at runtime: `$driveId = "b" + "!" + "F8Bz..."`
-
-3. **Current Upload:**
-   - Command dispatched successfully with drive ID concatenation
-   - Script has been running but showing no output yet (may still be scanning files)
-
---
-
-## CREDENTIALS
-
-**Graph API (from vault):**
- Path: `msp-tools/computerguru-tenant-admin`
- Tenant ID: 19a568e8-9e88-413b-9341-cbc224b39145
- Client ID: 709e6eed-0711-4875-9c44-2d3518c47063
- Client Secret: (in vault at `credentials.client_secret`)
-
-**GuruRMM:**
- Vault path: `infrastructure/gururmm-server.sops.yaml`
- Agent ID: a4524e85-8a07-45d0-91b1-51ce7e2ca74a (ACG-DWP-X-BB)
-
-**SharePoint Drive ID:**
- `b!F8BzMb1YakCIWCyWlmczb09LHqtxDxVMpLT6kAwYmsM7NUY4oPLSRq7ng3tJq-E9`
-
---
-
-## NEXT STEPS TO CONTINUE
-
-### Option 1: Check if current upload completed
-
-```bash
-# Authenticate to RMM
-eval "$(bash .claude/scripts/rmm-auth.sh)"
-
-# Check upload status
-curl -s "$RMM/api/commands/9e0fcfe8-0619-4a39-bd9c-6f5fd75c9b55" \
-  -H "Authorization: Bearer $TOKEN" > /tmp/upload-status.json
-
-python3 -c "
-import json
-with open('/tmp/upload-status.json') as f:
-    data = json.load(f)
-print(f\"Status: {data.get('status')}\")
-print(f\"Exit code: {data.get('exit_code')}\")
-stdout = data.get('stdout', '') or ''
-if len(stdout) > 0:
-    print('\n--- Last 30 lines ---')
-    for line in stdout.split('\n')[-30:]:
-        print(line)
-"
-
-# Then verify SharePoint count
-python3 clients/birth-biologic/scripts/check-quality-status.py
-```
-
-### Option 2: If upload failed, use the working PowerShell script
-
-The correct script is in: `clients/birth-biologic/scripts/upload-final-working.ps1`
-
-Run via RMM:
-```bash
-eval "$(bash .claude/scripts/rmm-auth.sh)"
-AGENT_ID="a4524e85-8a07-45d0-91b1-51ce7e2ca74a"
-CLIENT_SECRET=$(bash .claude/scripts/vault.sh get-field msp-tools/computerguru-tenant-admin credentials.client_secret)
-
-# Script content with proper drive ID concatenation
-SCRIPT='... (see upload-final-working.ps1) ...'
-
-PAYLOAD=$(jq -n --arg cmd "$SCRIPT" '{command_type: "powershell", command: $cmd, timeout_seconds: 1800}')
-curl -s -X POST "$RMM/api/agents/$AGENT_ID/command" \
-  -H "Authorization: Bearer $TOKEN" \
-  -H "Content-Type: application/json" \
-  -d "$PAYLOAD" | jq -r '.command_id'
-```
-
---
-
-## FILES CREATED
-
-**Scripts:**
- `clients/birth-biologic/scripts/check-quality-status.py` - Check SharePoint file count
- `clients/birth-biologic/scripts/upload-datto-to-sharepoint.ps1` - Base upload script
- `clients/birth-biologic/scripts/reset-quality-exact.py` - Initial reset script (used)
- `clients/birth-biologic/scripts/exact-sync-quality.py` - Robocopy approach
- `clients/birth-biologic/scripts/sync-quality-simple.py` - Earlier attempt
- `clients/birth-biologic/scripts/finish-upload.py` - Bulk upload attempt
- `clients/birth-biologic/scripts/upload-remaining-files.py` - Remaining files upload
-
-**Todo List:**
-1. [completed] Delete ALL files from SharePoint Quality Systems Department
-2. [in_progress] Copy ALL files from Datto to SharePoint exactly as they exist
-3. [pending] Verify SharePoint has exactly 3768 files matching Datto
-
---
-
-## DATTO SOURCE PATH
-
-**On ACG-DWP-X-BB:**
-```
-C:\Users\Public\Desktop\Datto Workplace Server Projects\Quality Department
-```
-
-**Total files:** 3,768 files (verified via Get-ChildItem -Recurse)
-
-**Files >4MB:** ~63 files (these are being skipped - need separate large file upload later)
-
---
-
-## VERIFICATION COMMAND
-
-Once upload completes:
-
-```python
-python3 clients/birth-biologic/scripts/check-quality-status.py
-```
-
-Expected output:
-```
-SharePoint: 3768 files
-Datto: 3768 files
-Gap: 0 files
-
-[OK] MATCH - SharePoint has exactly 3768 files
-```
-
---
-
-## TROUBLESHOOTING
-
-**If upload shows 0 uploaded, 3467 errors:**
- This means drive ID was escaped wrong (b\! instead of b!)
- Solution: Use string concatenation `"b" + "!" + "..."`
-
-**If upload hangs with no output:**
- PowerShell may have syntax error
- Check command_text field to verify script sent correctly
-
-**If you need to cancel running command:**
-```bash
-curl -s -X POST "$RMM/api/commands/9e0fcfe8-0619-4a39-bd9c-6f5fd75c9b55/cancel" \
-  -H "Authorization: Bearer $TOKEN" \
-  -H "Content-Type: application/json"
-```
-
---
-
-## WORKING UPLOAD SCRIPT TEMPLATE
-
-Save this as `upload-final-working.ps1`:
-
-```powershell
-$source = "C:\Users\Public\Desktop\Datto Workplace Server Projects\Quality Department"
-$driveId = "b" + "!" + "F8BzMb1YakCIWCyWlmczb09LHqtxDxVMpLT6kAwYmsM7NUY4oPLSRq7ng3tJq-E9"
-$tenantId = "19a568e8-9e88-413b-9341-cbc224b39145"
-$clientId = "709e6eed-0711-4875-9c44-2d3518c47063"
-$clientSecret = "GET_FROM_VAULT"
-
-Write-Host "Getting Graph API token..."
-$tokenBody = @{
-    client_id = $clientId
-    client_secret = $clientSecret
-    scope = "https://graph.microsoft.com/.default"
-    grant_type = "client_credentials"
-}
-
-$tokenResponse = Invoke-RestMethod -Method Post -Uri "https://login.microsoftonline.com/$tenantId/oauth2/v2.0/token" -Body $tokenBody
-$token = $tokenResponse.access_token
-Write-Host "[OK] Token acquired"
-
-Write-Host "Scanning Datto files..."
-$files = Get-ChildItem $source -Recurse -File -ErrorAction SilentlyContinue
-Write-Host "[OK] Found $($files.Count) files"
-
-Write-Host "Uploading files..."
-$uploaded = 0
-$errors = 0
-
-foreach ($file in $files) {
-    $relativePath = $file.FullName.Substring($source.Length + 1)
-    $uploadPath = $relativePath.Replace("\", "/")
-
-    try {
-        if ($file.Length -lt 4MB) {
-            $uploadUrl = "https://graph.microsoft.com/v1.0/drives/$driveId/root:/$uploadPath" + ":/content"
-            $headers = @{
-                "Authorization" = "Bearer $token"
-                "Content-Type" = "application/octet-stream"
-            }
-
-            $fileBytes = [System.IO.File]::ReadAllBytes($file.FullName)
-            Invoke-RestMethod -Method Put -Uri $uploadUrl -Headers $headers -Body $fileBytes -UseBasicParsing | Out-Null
-
-            $uploaded++
-            if ($uploaded % 100 -eq 0) {
-                Write-Host "  Uploaded $uploaded files..."
-            }
-        }
-    } catch {
-        $errors++
-    }
-}
-
-Write-Host ""
-Write-Host "Upload Complete"
-Write-Host "Uploaded: $uploaded"
-Write-Host "Errors: $errors"
-Write-Host "Total: $($files.Count)"
-```
-
---
-
-## SESSION NOTES
-
- OneDrive sync is unreliable for large migrations (stalled at 86%)
- Direct Graph API upload is the correct approach
- PowerShell exclamation mark escaping is a major gotcha
- Use string concatenation to avoid escape issues
- Files >4MB need separate upload session logic (not implemented yet)
- There are ~63 large files that will need separate handling
-
-**Client was promised this yesterday - NOW VERY ANGRY**
-
---
-
-## QUICK RESUME CHECKLIST
-
-1. [ ] Check if command 9e0fcfe8 completed
-2. [ ] Verify SharePoint file count (should be 3,768)
-3. [ ] If not complete, dispatch new upload with working script
-4. [ ] Monitor for completion (15-20 minutes)
-5. [ ] Verify final count matches Datto
-6. [ ] Handle large files (>4MB) if needed
-7. [ ] Document completion
-
-**END OF CONTINUATION INSTRUCTIONS**
+Summary: all 3,768 Datto files are present in SharePoint (verified via Graph delta). The
+only differences are 4 files that are live current work edited today by client staff
+(1 new xlsx + 3 open docs), intentionally preserved per Mike. The prior approach failed all
+day because its script skipped every file >=4MB (301 files, ~29.7 GB) and used the RMM
+`timeout` field (capped ~300s) instead of `timeout_seconds`.