wiki: compile birth-biologic (full) — Datto->SharePoint migration complete, Quality sync done, MX cut, #32187 rename scheduled

This commit is contained in:
2026-07-01 09:08:57 -07:00
parent 6f676672a8
commit 02845878d6
2 changed files with 70 additions and 46 deletions

View File

@@ -2,13 +2,20 @@
type: client
name: birth-biologic
display_name: BirthBiologic
last_compiled: 2026-06-26
last_compiled: 2026-07-01
compiled_by: GURU-5070/claude-main
sources:
- clients/birth-biologic/session-logs/2026-04-21-session.md
- clients/birth-biologic/session-logs/2026-06-02-session.md
- clients/birth-biologic/session-logs/2026-06/2026-06-26-mike-birthbio-mail-migration-and-datto-vm.md
- clients/birth-biologic/session-logs/2026-06/2026-06-29-mike-birthbio-google-audit-corruption-restore-mailgroups.md
- clients/birth-biologic/session-logs/2026-06/2026-06-30-mike-birthbio-quality-consolidation-corruption.md
- clients/birth-biologic/session-logs/2026-06/2026-06-30-mike-birthbio-quality-sync-complete.md
- clients/birth-biologic/docs/migration/2026-06-30-quality-sync-COMPLETE.md
- clients/birth-biologic/docs/migration/datto-to-sharepoint-map.md
- clients/birth-biologic/docs/migration/google-to-m365-scope.md
- clients/birth-biologic/docs/migration/2026-06-26-corruption-recovery-plan.md
- clients/birth-biologic/docs/migration/2026-06-29-quality-dept-archival-plan.md
backlinks:
- projects/gururmm
aliases: [birthbiologic]
@@ -18,18 +25,17 @@ aliases: [birthbiologic]
## Profile
- **Company type:** Biological/healthcare services (cord blood / donor services implied by site structure: Donor Services, Quality Department, Birth Biologic Activity Reports); Stilwell, KS
- **Contract type:** Prepaid hour block
- **Company type:** Biological/healthcare services (cord blood / donor services); 19546 Metcalf Avenue, Stilwell KS
- **Contract type:** Prepaid hour block (~$132.03/month recurring + separate project/labor invoices)
- **Key contacts:**
- Annise — primary client contact for migration work; no last name or email documented
- Kristin Steen — ksteen@birthbiologic.com (known Syncro contact; workstation KSTEENBB2025)
- Kristin Steen — ksteen@birthbiologic.com, 316-833-9803 (known Syncro contact; workstation KSTEENBB2025)
- sysadmin@birthbiologic.com — M365/Google shared admin account (ACG-managed); M365 Business Premium license assigned 2026-04-21; SharePoint admin role confirmed
- **Billing rate:** (verify — check Syncro invoices)
- **Hours remaining (prepaid):** 10.0 hrs as of 2026-06-26
- **Billing rate:** (verify — recent labor invoices ~$150/hr remote; confirm in Syncro)
- **Hours remaining (prepaid):** 3.0 hrs as of 2026-07-01 (was 10.0 on 2026-06-26; dropped due to 5.0h migration billing + 2.0h sessions on 2026-06-29)
- **Syncro customer ID:** 17983014
- **Managed assets (Syncro):** 13
- **Open tickets:** 0 as of 2026-06-26
- **Historical ticket:** #109277420 — Datto Workplace to SharePoint Migration; assigned Mike Swanson; contact Annise; closed/historical
- **Active ticket:** #32187 (Scheduled) — SharePoint Migration rename, 2026-07-01 7-8 PM MST
## Infrastructure
@@ -38,19 +44,20 @@ aliases: [birthbiologic]
| Host | IP | Role | OS | Notes |
|---|---|---|---|---|
| BB-SERVER | (verify) | On-premise Windows server | Windows Server 2016 | GuruRMM agent `6c02baa7-0f1c-4990-b466-c9ab9eaefd3b` installed 2026-04-21; Datto Workplace Server installed; custom Datto→SP migration script artifacts at `C:\GuruMigration`; state file shows 160 Supply Mgmt + 49 ITSvcs uploaded April 2026 |
| ACG-DWP-X-BB | 172.16.3.45 | ACG-owned Datto/SPMT migration VM (Jupiter libvirt) | Windows Server 2019 build 17763 (libvirt domain label "Windows Server 2016") | Static IP /22, GW 172.16.0.1, DNS 172.16.0.1+1.1.1.1; virtio NIC 52:54:00:d4:8e:59 on br0 (vnet14); Datto Workplace Server (svc `datto_workplace_server.default`) + SPMT (under Administrator profile); source tree `C:\Users\Public\Desktop\Datto Workplace Server Projects`; GuruRMM agent `a4524e85-8a07-45d0-91b1-51ce7e2ca74a` enrolled 2026-06-26 |
| ACG-DWP-X-BB | 172.16.3.45 | ACG-owned Datto/SPMT migration VM (Jupiter libvirt) | Windows Server 2019 build 17763 (libvirt domain label "Windows Server 2016") | Static IP /22, GW 172.16.0.1, DNS 172.16.0.1+1.1.1.1; virtio NIC 52:54:00:d4:8e:59 on br0 (vnet14); Datto Workplace Server (svc `datto_workplace_server.default`) **stopped + disabled 2026-06-27** (source frozen post-migration); SPMT under Administrator profile; source tree `C:\Users\Public\Desktop\Datto Workplace Server Projects`; GuruRMM agent `a4524e85-8a07-45d0-91b1-51ce7e2ca74a` enrolled 2026-06-26 |
### Email & Identity
- **M365 tenant:** birthbiologic.com / tenant ID `19a568e8-9e88-413b-9341-cbc224b39145`
- **Target delivery domain (migration):** birthbiologic.onmicrosoft.com
- **Accepted domains:** birthbiologic.com (default), birthbiologic.onmicrosoft.com
- **MX (as of 2026-06-29):** **M365** (`birthbiologic-com.mail.protection.outlook.com`) — **cutover done 2026-06-27 (Sat)**; live mail now on M365 (was Google Workspace through 06-26). Always verify MX live; do not trust the 06-26 migration-scope docs.
- **Mail groups / shared mailboxes (created/configured 2026-06-29):**
- `medicalrecords@`**distribution group**, 14 members (12 core staff + `medicaldirector@` + `mmerritt@`), `RequireSenderAuthenticationEnabled=$false` (external processors can email it). Functions as all-staff but is a distinct named group for time-sensitive processor outreach.
- **MX (as of 2026-06-29, confirmed live):** **M365** (`birthbiologic-com.mail.protection.outlook.com`) — **cutover done 2026-06-27 (Sat)**; live mail now on M365. Do not trust pre-2026-06-27 assumptions.
- **SPF / DKIM / autodiscover / DMARC:** (verify — should have been updated at MX cutover 2026-06-27; no session log confirms)
- **Mail groups / shared mailboxes (configured 2026-06-29):**
- `medicalrecords@`**distribution group**, 14 members (12 core staff + `medicaldirector@` + `mmerritt@`), `RequireSenderAuthenticationEnabled=$false` (external processors can email it).
- `info@`**shared mailbox**; Full Access + Send As: Brandy Burgess, Julie Beck.
- `quality@`**shared mailbox**; Full Access + Send As: Brandy Burgess, Julie Beck, Mary Ster, Alicia Meneely, Kristin Steen, Vicki Fountain.
- Other existing shared mailboxes: `accounting@`, `operations@` (user mailbox).
- Other shared mailboxes: `accounting@`, `operations@` (user mailbox).
- **DNS host:** SiteGround (`ns1/ns2.us92.siteground.us`); Registrar: Name.com; `www` → GCP 35.215.115.203 (not in scope)
- **M365 licensing (all consumed as of 2026-06-26):**
- Business Premium (skuId `cbdc14ab-d96c-4c30-b9f4-6ada7cdc1d46`): 14/14
@@ -62,11 +69,11 @@ aliases: [birthbiologic]
- Security Investigator: consented (SP `bf684a4b-…`)
- Tenant Admin: consented (app client_id `709e6eed-0711-4875-9c44-2d3518c47063`; SP object `7a199b11-97fb-4e65-917d-f8d29a53ba49`; consent redirect URI must be `https://azcomputerguru.com`, NOT `https://rmm.azcomputerguru.com`)
- Exchange Operator: consented 2026-06-26 (SP `bab4699b-32a3-4434-9cad-7a4a08cc4d9e`; Exchange Administrator role)
- User Manager: consented 2026-06-26 (SP `3347ebcc-…`)
- User Manager: consented 2026-06-26 (SP `3347ebcc-…`; has Group.ReadWrite.All — use this app for M365 group deletes, not Tenant Admin)
- Defender Add-on: consented 2026-06-26 (SP `161b8f61-…`)
- **Note:** sysadmin@birthbiologic.com did not have a SharePoint/M365 license prior to 2026-04-21. For SharePoint app-only access, use Tenant Admin app with `Sites.ReadWrite.All` (no user license required for app-only).
### Google Workspace (source tenant — migration in progress)
### Google Workspace (source tenant — mail migration completed for live users)
- **Super-admin:** sysadmin@birthbiologic.com; password vaulted at `clients/birth-biologic/google-workspace.sops.yaml` (`credentials.password`)
- **Domain-wide delegation:** acg-msp-access SA (`acg-msp-access@acg-msp-access.iam.gserviceaccount.com`); OAuth2 client ID `102231607889615995452`; GCP project `acg-msp-access` (number 806899474449)
@@ -74,37 +81,38 @@ aliases: [birthbiologic]
`https://mail.google.com/,https://www.googleapis.com/auth/calendar,https://www.google.com/m8/feeds/,https://www.googleapis.com/auth/gmail.settings.sharing,https://www.googleapis.com/auth/contacts`
- **GCP APIs enabled on acg-msp-access:** Gmail, Calendar (calendar-json), People
- **Google roster (DWD pull, 2026-06-26):** 20 accounts — 15 active, 5 suspended
- **DWD status (as of 2026-06-29):** `m8/feeds` scope was missing at that point (was present on 06-26 when migration ran, then dropped); must be re-added before running any final Gmail migration delta or Batch 2.
### Gmail Migration Status (as of 2026-06-26)
### Gmail Migration Status
- **Method:** Native MS "Migration from Google Workspace" via Exchange Operator REST InvokeCommand
- **Endpoint:** `BB-Gmail` (type: Gmail; impersonation admin: sysadmin@birthbiologic.com)
- **Batch 1 (BB-Batch1):** 14 live mailboxes, mail + calendar + contacts, TargetDeliveryDomain `birthbiologic.onmicrosoft.com`, AutoStart, NotificationEmails sysadmin@; **Status: Syncing** (created 2026-06-26)
- **Batch 2:** Not started — 5 former employees; pending un-suspend in Google + free Workspace seats
- **Batch 1 (BB-Batch1):** 14 live mailboxes, mail + calendar + contacts, TargetDeliveryDomain `birthbiologic.onmicrosoft.com`; **Status: Synced** (created + auto-started 2026-06-26; confirmed Synced 14/14, 0 failures, 7 skipped items as of 2026-06-29); DataConsistencyScore=Investigate (from 7 skipped items); **batch not yet finalized/completed**
- **Batch 2:** Not started — 5 former employees (`aboutte`, `araso`, `khoffman`, `pnelson`, `sabron`); pending un-suspend in Google + free Workspace seats
### File Storage
- **Pre-migration source:** Datto Workplace (server on ACG-DWP-X-BB; original custom-script artifacts on BB-SERVER at `C:\GuruMigration`)
- **Pre-migration source:** Datto Workplace (server on ACG-DWP-X-BB; original custom-script artifacts on BB-SERVER at `C:\GuruMigration`); Datto service stopped + disabled 2026-06-27
- **Post-migration target:** Microsoft SharePoint (M365)
- **Migration tools:** Custom PowerShell script (`clients/birth-biologic/scripts/migrate-datto-to-sharepoint.ps1`) + SPMT (on ACG-DWP-X-BB under Administrator profile)
- **Migration tools:** Custom PowerShell scripts (see `clients/birth-biologic/scripts/`) + SPMT (on ACG-DWP-X-BB under Administrator profile)
### SharePoint Site Map
| Datto Folder | SharePoint Site | Size / Files | Status |
|---|---|---|---|
| Admin | birthbiologic.sharepoint.com/sites/Admin | 5.8 GB / 6,279 files | SPMT last ran 2026-04-29; completion UNCONFIRMED |
| Birth Biologic Activity Reports | birthbiologic.sharepoint.com/sites/Admin (subfolder) | 1 file | SPMT; SPMT preserves source folder name as subfolder; UNCONFIRMED |
| Donor Services | birthbiologic.sharepoint.com/sites/DonorServices | 109 GB / 56,826 files | SPMT last ran 2026-04-29; completion UNCONFIRMED |
| Quality Department | **canonical: birthbiologic.sharepoint.com/sites/QualitySystemsDepartment** (interim/orphan site: /sites/QualityDepartment) | 28 GB / 3,714 files | Quality content split across TWO sites (corrected 2026-06-29). Canonical working site = Quality Systems Department; the OneDrive-sync target on ACG-DWP-X-BB is the interim QualityDepartment site. ~1,006 files (~2.1 GB) orphaned in QualityDepartment, missing from Quality Systems Department (LOGS 485/1.6GB, Bone Bank Onsite Audit 2025 427/440MB) — migration/reconciliation in progress. 4 Quality Systems Department-* spoke sites exist but empty. Orphan list: clients/birth-biologic/docs/migration/quality-orphaned-files.txt |
| Admin | birthbiologic.sharepoint.com/sites/Admin | 5.8 GB / ~6,300 files | Reconciled to 0 missing 2026-06-27 (delta-recon-v2 + delta-upload-v3) — COMPLETE |
| Birth Biologic Activity Reports | birthbiologic.sharepoint.com/sites/Admin (subfolder) | small / 1 file | SPMT; preserves source folder name as subfolder; reconciled 0 missing 2026-06-27 — COMPLETE |
| Donor Services | birthbiologic.sharepoint.com/sites/DonorServices | 109 GB / ~56,800 files | Reconciled to 0 missing 2026-06-27 — COMPLETE |
| Quality Department (Datto) | **canonical: birthbiologic.sharepoint.com/sites/QualitySystemsDepartment** | ~29.7 GB / 3,768 Datto files | COMPLETE 2026-06-30: all 3,768 Datto files present (0 missing); 1 staff-created file also in SP (3,769 total); 4 live-work files preserved. Old /sites/QualityDepartment duplicate site soft-deleted 2026-06-29 (group restorable ~30 days, site ~93 days from that date). |
| Supply Management | birthbiologic.sharepoint.com/sites/SupplyManagement | 33 MB / 160 files | 160/160 migrated via custom PS script 2026-04-21 — COMPLETE |
| ITSvcs | EXCLUDED | 52 files | ACG-owned folder; never client data |
Site IDs hardcoded in `$SITE_MAP` hashtable in the migration script.
Site IDs hardcoded in `$SITE_MAP` hashtable in the migration script. QSD site ID: `3173c017-58bd-406a-8858-2c969667336f` (drive `b!F8BzMb1YakCIWCyWlmczb09LHqtxDxVMpLT6kAwYmsM7NUY4oPLSRq7ng3tJq-E9`). Graph app for all SharePoint work: vault `msp-tools/computerguru-tenant-admin` (tenant `19a568e8-9e88-413b-9341-cbc224b39145`).
### Network
- **ACG Jupiter (Datto VM host):** LAN 172.16.0.0/22, GW pfSense 172.16.0.1; Jupiter at 172.16.3.20 (Unraid, virsh); guest-exec helper `/root/gx.sh`
- **ACG-DWP-X-BB:** 172.16.3.45/22 static (was APIPA after ~2 months parked; pfSense DHCP not leasing that MAC; fixed 2026-06-26)
- **ACG-DWP-X-BB:** 172.16.3.45/22 static (was APIPA after ~2 months parked; pfSense DHCP not leasing that MAC; fixed 2026-06-26); pfSense DHCP reservation for MAC `52:54:00:d4:8e:59` not yet confirmed
- **ISP / WAN (BirthBio site):** (verify)
- **Firewall (BirthBio site):** (verify)
- **VPN:** (verify)
@@ -129,7 +137,7 @@ Site IDs hardcoded in `$SITE_MAP` hashtable in the migration script.
| KSTEENBB2025 | KSTEENBB2025 | Windows 11 | `ee3c6aea-e9cc-4d2f-9e79-a38dd0eb129e` | — | Kristin Steen's workstation |
| EVO-X1 | EVO-X1 | Windows 11 | `9595f002-5cfe-4db6-b7aa-1df4a20e9f9b` | — | Vicki Fountain's workstation; SmartBadge fleet reference machine |
| BB-Office2 | BB-Office2 | Windows 11 | `48763401-4859-49f9-b64a-7a50d0148b23` | — | Shared/office workstation |
| ACG-DWP-X-BB | ACG-DWP-X-BB | Windows Server 2019 | `a4524e85-8a07-45d0-91b1-51ce7e2ca74a` | 172.16.3.45 | ACG-owned; Jupiter libvirt VM; Datto Workplace Server + SPMT migration host; enrolled 2026-06-26 under BirthBiologic/Main Office |
| ACG-DWP-X-BB | ACG-DWP-X-BB | Windows Server 2019 | `a4524e85-8a07-45d0-91b1-51ce7e2ca74a` | 172.16.3.45 | ACG-owned; Jupiter libvirt VM; Datto source + SPMT migration host; enrolled 2026-06-26; Datto service stopped 2026-06-27 |
## Access
@@ -138,15 +146,16 @@ Site IDs hardcoded in `$SITE_MAP` hashtable in the migration script.
- **Google Workspace admin:** sysadmin@birthbiologic.com (same account; password vaulted)
- **Vault paths:**
- `clients/birthbiologic/gururmm-site-main.sops.yaml` — GuruRMM site enrollment key
- `msp-tools/computerguru-tenant-admin.sops.yaml``credentials.credential` — Tenant Admin app secret
- `msp-tools/computerguru-tenant-admin.sops.yaml``credentials.client_secret` — Tenant Admin app secret (NOTE: field is `client_secret`, NOT `credential`; `credential` returns 4-char null)
- `msp-tools/computerguru-exchange-operator.sops.yaml``credentials.client_secret` — Exchange Operator app secret
- `msp-tools/computerguru-user-manager.sops.yaml``credentials.client_secret` — User Manager app secret (use for M365 group deletes)
- `msp-tools/acg-msp-access-google-workspace.sops.yaml``credentials.credential` — Google SA JSON key (full)
- `clients/birth-biologic/google-workspace.sops.yaml``credentials.password` — Google Workspace super-admin password
- `clients/birth-biologic/m365-medicaldirector.sops.yaml` — Dr. Chris Gillis M365 initial password (forceChangePasswordNextSignIn=true)
- `clients/birth-biologic/m365-mmerritt.sops.yaml` — Michael Merritt M365 initial password (forceChangePasswordNextSignIn=true)
- **Tenant Admin app:** client_id `709e6eed-0711-4875-9c44-2d3518c47063`; consent redirect URI must be `https://azcomputerguru.com` (NOT `https://rmm.azcomputerguru.com`)
- **Exchange Operator SP:** `bab4699b-32a3-4434-9cad-7a4a08cc4d9e`; Exchange Administrator role; drive via REST InvokeCommand (see Patterns)
- **Migration script:** `clients/birth-biologic/scripts/migrate-datto-to-sharepoint.ps1`
- **Migration scripts:** `clients/birth-biologic/scripts/` (migrate-datto-to-sharepoint.ps1, enumerate-datto.ps1, upload-quality-final.ps1, bb-recover.py)
- **Migration runbook:** `projects/msp-tools/runbooks/google-workspace-to-m365-migration.md` (updated 2026-06-26 — exact 5-scope string, all-or-nothing gotcha, Contacts API retired/People API, GCP-owner requirement)
## Patterns & Known Issues
@@ -154,11 +163,16 @@ Site IDs hardcoded in `$SITE_MAP` hashtable in the migration script.
- **Datto Workplace fleet standard = "Datto Workplace" v10.53.4 (installs to `C:\Program Files\Datto\Workplace2\`).** EVO-X1 and BB-Office2 run this version only. **Never** run the older "Datto Workplace **Desktop**" v8.50.13 (folder `…\Workplace Desktop\`) alongside it — having both installed breaks the Excel SmartBadge add-in (see below). Note the confusing naming: despite "Desktop" sounding newer, v8 Desktop is the *older* product; plain "Datto Workplace" v10 is current.
- **SmartBadge Excel add-in failure from dual Datto Workplace installs:** When both Workplace2 (v10) and Workplace Desktop (v8) are present, the `_CC` COM class `{3C639243-95A2-400D-B4B4-4384DA7F61D3}` gets a 64-bit InprocServer32 pointing at the wrong DLL (or only a 32-bit WOW64 entry), so 64-bit Excel can't load the shim and silently drops the SmartBadge ribbon tab. Excel then auto-disables the add-in (per-user `LoadBehavior=2`). **Fix = align to fleet:** remove Workplace Desktop v8 (Revo for a full leftover sweep), install Workplace v10.53.4, ensure only the `_CC` add-in (HKLM+WOW64, `LoadBehavior=3`) with the `_CC` CLSID → `…\Workplace2\SmartBadge\DattoSmartBadgeShim_x64/x86.dll`, and reset the user's `LoadBehavior` to 3 + clear Excel Resiliency. Reference machine: EVO-X1. Scripts: `.claude/scripts/ksteen-smartbadge-verify.ps1`, `.claude/scripts/ksteen-smartbadge-fix.ps1`.
- **Windows Server 2016 TLS:** BB-SERVER defaults to TLS 1.0. PowerShell scripts must include `[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12` at the top or Graph API calls will fail.
- **GuruRMM command timeout on long-running processes:** The RMM command channel times out on operations running longer than ~300 seconds. An 8 MB PDF upload at ~77 KB/s exceeded this limit during the migration. Workaround: base64-encode file on server, capture stdout, decode and upload locally.
- **GuruRMM command dispatch: use `timeout_seconds`, not `timeout`:** The RMM agent **ignores** the `timeout` field and caps commands at ~300 seconds. For any long-running upload/migration command, use `timeout_seconds` (e.g. 10800 for 3h) — sending both fields is safe. Commands dispatched with only `timeout` go zombie ("running", no output, never complete). Root cause confirmed 2026-06-30 during Quality sync. Memory: `gururmm-command-timeout-seconds`.
- **SharePoint Graph uploads: chunked upload sessions required for files >=4MB:** Simple PUT to `/content` works only for files <4MB. Files >=4MB **must** use Graph upload sessions (`POST .../createUploadSession`, then PUT chunks with `Content-Range`; 10 MB chunks work reliably). Failing to use upload sessions silently skips large files — the Quality sync Mac session failed all day because 301 large files (~29.7 GB) were skipped this way. Memory: `sharepoint-graph-large-file-upload`.
- **SharePoint 409 Conflict on retry:** If a chunked upload session is interrupted, a partial item remains in SharePoint. Subsequent upload sessions against the same path return 409 Conflict. Fix: DELETE the item before creating a new upload session.
- **Long Windows paths (>260 chars) require `\\?\` prefix:** The Datto source tree contains paths exceeding MAX_PATH. Use `\\?\` prefix for `[IO.File]` reads in PowerShell. Note: `Rename-Item` and `File.Move` in PS5.1 do NOT support `\\?\` — use `robocopy` or SPMT for long-path rename/move operations.
- **SharePoint single-session upload throttles ~40 Mbps:** For large migrations, parallel-stream uploaders (multiple concurrent file uploads, larger chunk sizes) would significantly improve throughput.
- **Tenant Admin app cannot delete M365 groups (403):** The Tenant Admin app has GroupMember write only, not Group.ReadWrite.All. `DELETE /groups/{id}` returns 403 via Tenant Admin app. Use the **User Manager app** for group deletes (returns HTTP 204). Also: the Tenant Admin app cannot manage SP site lock/spoke-site grants (`Unsupported app only token` on SP REST) — use PnP PowerShell as SharePoint Admin.
- **Byte-array stringification bug — RETIRED path:** The 2026-06-26 custom-script upload path passed file bytes as `"$bytes"`, which stringified the .NET byte array to space-separated decimal text instead of raw binary. Corrupt files are inflated ~3-4x; headers are decimal (e.g. `80 75 3 4...` for PK, `37 80 68 70...` for %PDF). 84 files were corrupted and restored from Datto source. This code path is permanently retired. **Never stringify a byte array in PowerShell** — use `[IO.File]::WriteAllBytes` for binary output.
- **SPMT requires sysadmin to be SharePoint admin:** SPMT destination access requires the running account to have SharePoint admin rights. Confirm before scheduling future SPMT runs.
- **Syncro comment rendering:** Use `<br>` for line breaks in Syncro comments. `<ul>/<li>` collapses into a single line in the Syncro renderer.
- **Syncro duplicate comments on #109277420:** Two duplicate comments were noted in the session log. GUI deletion only (no API delete for comments). Verify status next time in ticket view.
- **Syncro duplicate comments on #109277420/#32187:** Two duplicate comments were noted in the 2026-04-21 session log. GUI deletion only (no API delete for comments). Verify status next time in ticket view.
- **ITSvcs folder exclusion:** The `ITSvcs` folder on the Datto share is ACG-owned, not client data. Always exclude from any migration or client-facing file audit.
- **GuruRMM command body requirements:** `command_type` field is required (use `"powershell"` for PS scripts). Missing field returns 422. JWT must include `sub`, `role`, `orgs`, `exp`, `iat` claims — any missing claim returns 401.
- **GuruRMM `.stdout` null handling in watch scripts:** `jq -r '.stdout'` emits the literal 4-char string `"null"` when the API returns JSON `null` for stdout. Always use `.stdout // empty` (or `.stdout // ""`) so that a null field becomes an empty string, not the word "null". Affects any script that greps command output for a sentinel line.
@@ -167,29 +181,39 @@ Site IDs hardcoded in `$SITE_MAP` hashtable in the migration script.
- **Enabling GCP APIs in acg-msp-access requires ACG project owner identity:** Running `gcloud services enable` as a client super-admin (`sysadmin@birthbiologic.com`) fails — that account has no rights to ACG's `acg-msp-access` GCP project. Must be authenticated as the ACG GCP project owner.
- **Exchange driven via REST InvokeCommand — EXO PS module not available:** Exchange Operator app token (`scope=https://outlook.office365.com/.default`), endpoint `POST https://outlook.office365.com/adminapi/beta/{tenant}/InvokeCommand`, body `{"CmdletInput":{"CmdletName":"…","Parameters":{…}}}`. EXO PowerShell module not installed; the app has no vaulted cert, so `Connect-ExchangeOnline` app-only auth is not available. Byte-array parameters (`ServiceAccountKeyFileData`, `CSVData`) must be passed as base64 strings.
- **`vault.sh get-field` requires dotted field path for nested secrets:** `credentials.client_secret` and `credentials.credential` work; bare leaf names (`client_secret`) return a literal 4-char `null`. Always specify the full dotted path.
- **Tenant Admin vault field is `credentials.client_secret`, not `credentials.credential`:** The pre-06-29 wiki and 04-21 session documented `credentials.credential` for the Tenant Admin app secret — this is WRONG. The correct field is `credentials.client_secret`. Using the wrong field returns 4-char null silently. Corrected 2026-06-29.
- **Tenant's real Business Premium skuId is `cbdc14ab-d96c-4c30-b9f4-6ada7cdc1d46`:** The scope doc had a stale GUID (`cbdc14ab-d96c-4132-b7f4-1f3a3a819bb4`). License assign 400'd until corrected. Pull skuId live from Graph `/subscribedSkus` before any license assignment.
- **Datto→SharePoint additive push caused "reappearing files":** The April 2026 SPMT/script run was additive (never a live sync). Files deleted from Datto after April remained in SharePoint, creating phantom files that appeared to "reappear." Resolved 2026-06-27 by treating Datto as source of truth and mirroring SP to it (deleted 1,564 stale SP files to recycle bin). SharePoint and Datto are now synchronized.
- **Quality content two-site confusion:** A `Quality Department` site (/sites/QualityDepartment) was the original April 2026 migration landing target; `Quality Systems Department` (/sites/QualitySystemsDepartment) was created 2026-06-02 as the canonical site. The old site was soft-deleted 2026-06-29 after content parity was verified and the one divergent file was preserved in QSD. Do not expect /sites/QualityDepartment to exist after ~2026-09-29 (recycle bin expiry).
## Active Work
- **Google → M365 mail migration (IN PROGRESS):** BB-Batch1 auto-started 2026-06-26, Status: Syncing, 14 live mailboxes (mail + calendar + contacts). Pending:
- Monitor BB-Batch1: Provisioning → Syncing → Synced
- When Synced: flip MX in SiteGround DNS → M365; update SPF (`include:spf.protection.outlook.com`); enable/publish DKIM (2 CNAMEs); autodiscover CNAME → `autodiscover.outlook.com`; review DMARC; run final delta; complete batch
- Batch 2 — 5 former employees → shared mailboxes: un-suspend each in Google (free Workspace seats by suspending migrated live users first), run Gmail migration batch (`aboutte`, `araso`, `khoffman`, `pnelson`, `sabron` — already EXO-licensed, sign-in disabled), convert to shared mailboxes (<=50 GB = free), reclaim 5 EXO licenses
- Confirm Valerie VanEaton's status (active or departed since mid-May; if departed → former/shared track)
- Confirm Michael Merritt's long-term licensing tier
- Confirm `operations@` fate post-cutover (retain BP or convert to shared)
- **Datto → SharePoint migration reconciliation (BLOCKED — awaiting ACG-DWP-X-BB Datto re-sync):**
- Supply Management complete (160/160 files, 2026-04-21)
- 4 large SPMT folders (Admin 5.8 GB, Donor Services 109 GB, Quality 28 GB, Activity Reports) last SPMT run 2026-04-29; completion UNCONFIRMED — reconciliation pending Datto re-sync on ACG-DWP-X-BB
- After re-sync: compare source vs each SharePoint site, determine what April SPMT run left incomplete, schedule completion run(s)
- Notify Annise to test SharePoint access once confirmed complete; run delta sync (`-DeltaOnly`) post-confirmation
- **pfSense:** add DHCP reservation for 172.16.3.45 (MAC `52:54:00:d4:8e:59`) or confirm it is outside the DHCP pool
| Ticket | Syncro ID | Status | Summary | Next Action |
|---|---|---|---|---|
| #32187 | 109277420 | Scheduled | SharePoint Migration - Datto Workplace to SharePoint Online | Off-hours rename: Quality Systems Department Team + SharePoint site → "Quality Department"; update Staff Portal link (the URL `/sites/QualitySystemsDepartment` does NOT auto-change). Scheduled 2026-07-01 7-8 PM MST. Coord todo `c051e97d`. Do NOT use CIPP — toggle "Do Not Invite" on appointment `5628749055` in Syncro GUI if customer calendar invite is unwanted. |
**Pending items (not yet ticketed or deferred):**
- **QMS corruption recovery (DEFERRED, coord todo `28e3e7ab`):** ~81 corrupt files remain in Quality Systems Department (decimal-text byte corruption from 2026-06-26). Run `clients/birth-biologic/scripts/bb-recover.py birthbiologic.sharepoint.com:/sites/QualitySystemsDepartment` dry-run, then `--apply` (set env `BBSEC` = Tenant Admin `client_secret` from vault). Re-scan live first; do NOT trust the saved 47-list from an earlier pass. Also widen scan tenant-wide (Admin/Donor Services/Supply were in the same 06-26 corrupt batch).
- **89 deferred long-path files:** Cloud-only OneDrive files at >=260-char paths modified 2026-06-26 with no Datto source mapping (Quality 59, Admin 30). Not yet assessed. Handle via robocopy or SPMT (long-path native).
- **Gmail migration — Batch 1 finalize:** BB-Batch1 is Synced but not yet completed/finalized. Review 7 skipped items; investigate DataConsistencyScore=Investigate. Before running final delta, re-add `m8/feeds` scope to DWD in Google Admin (was missing as of 2026-06-29).
- **Gmail migration — Batch 2:** 5 former employees (`aboutte`, `araso`, `khoffman`, `pnelson`, `sabron`). Un-suspend each in Google (free Workspace seats by suspending migrated live users first); run Gmail migration batch (they are already EXO-licensed, sign-in disabled); convert to shared mailboxes (<=50 GB = free); reclaim 5 EXO licenses.
- **Valerie VanEaton status:** Confirm active or departed since mid-May 2026. If departed, move to former/shared-mailbox track.
- **Michael Merritt long-term licensing tier:** Confirm whether Exchange-only (current) is appropriate long-term.
- **`operations@` fate post-cutover:** Retain Business Premium or convert to shared mailbox.
- **pfSense DHCP reservation:** Add reservation for 172.16.3.45 (MAC `52:54:00:d4:8e:59`) or confirm it is outside the DHCP pool (prevents APIPA recurrence on ACG-DWP-X-BB).
- **SP-only user files** (Shift Coms / DEMO and similar content created directly in SharePoint) — decide whether to fold into Datto archive.
## History Highlights
| Date | Event |
|---|---|
| 2026-06-26 | Mike (GURU-5070): Google→M365 mail migration initiated; BB-Batch1 live (14 mailboxes, Status: Syncing). Identified Datto/SPMT migration VM as Jupiter libvirt domain ACG-DWP-X-BB (actual WS2019 build 17763); had APIPA after ~2 months parked (pfSense not leasing MAC); fixed with static IP 172.16.3.45/22; GuruRMM agent enrolled (`a4524e85-…`); Datto Workplace Server reconnected + re-syncing. Confirmed April SPMT run (4 large folders) completion unconfirmed. Fully onboarded BirthBio M365 to ACG suite (Exchange Operator + User Manager + Defender Add-on consented via `onboard365.sh provision`). Provisioned Exchange-only mailboxes for Dr. Chris Gillis (`medicaldirector@`) and Michael Merritt (`mmerritt@`); license redistribution: Mei Mei + Valerie +BP, Savanna BP→EXO, 4 disabled formers +EXO. Created Gmail migration endpoint BB-Gmail; created + auto-started BB-Batch1 (14 mailboxes, TargetDeliveryDomain birthbiologic.onmicrosoft.com). Vaulted Google super-admin creds + new M365 user passwords. |
| 2026-07-01 | Mike (GURU-5070): Ticket #32187 posted customer-visible completion note (Quality sync done, all 3,768 files) and Annise reply re rename request. Ticket status → Scheduled. Off-hours rename (Quality Systems Department → Quality Department + Staff Portal link) scheduled 2026-07-01 7-8 PM MST. Coord todo `c051e97d`. Remote appointment `5628749055` created. |
| 2026-06-30 | Mike (GURU-5070): Quality Systems Department final sync COMPLETED. All 3,768 Datto files present in SharePoint (0 missing); 301 large files (>=4MB, ~29.7 GB total, largest a 3.94 GB .mov) uploaded via Graph chunked upload sessions; ~700 size-mismatched files silently repaired by idempotent uploader. 4 live-work files intentionally preserved (staff had them open). Root causes identified: prior Mac script skipped all >=4MB files; RMM agent ignores `timeout` field, requires `timeout_seconds`. Memories `gururmm-command-timeout-seconds` and `sharepoint-graph-large-file-upload` saved. |
| 2026-06-29 (session 2) | Mike (GURU-5070): Quality content consolidated into QSD. Datto-hash-based dedup: removed 811 byte-identical duplicates (kept Datto-aligned copies), removed 195 stale SP-only files, backfilled 31 files missing from QSD. Archived old QualityDepartment site: forked Surgenex xlsx preserved in QSD, then M365 group soft-deleted via User Manager app (Tenant Admin app 403'd — has GroupMember only, not Group.ReadWrite.All). 81 corrupt files found in QSD (more than 06-29 session 1's 84 due to orphan propagation); bb-recover.py graduated to repo (`clients/birth-biologic/scripts/bb-recover.py`), recovery deferred (coord todo 28e3e7ab). QSD verified: 0 Datto files missing. |
| 2026-06-29 (session 1) | Mike (GURU-5070): Confirmed MX live on M365 (cut 2026-06-27 — stale wiki assumption corrected). BB-Batch1 confirmed Synced (14/14, 0 failures, 7 skipped). Diagnosed 2026-06-26 byte-array stringification bug (84 corrupt files: 59 pdf, 20 docx, 5 xlsx across 4 libraries); restored all 84 from Datto source (83 direct + 1 decoded from decimal-text). Created `medicalrecords@` distribution group (14 members, external senders allowed). Granted Full Access + Send As on `info@` and `quality@` shared mailboxes. Tickets #32187 + #32451 updated; 2.0h billed; prepaid block 10.0→3.0. |
| 2026-06-27 | Mike (GURU-5070, continuation of 06-26 session): MX cut to M365 (SiteGround DNS). Datto→SP delta completed — all sites (Admin, Birth Biologic Activity Reports, Donor Services, Quality, Supply) reconciled to 0 missing. Quality Department SP site restored from deleted-site recycle bin (was soft-deleted when operations@ deleted its M365 Group); Quality content relocated to QSD via server-side copy. Mirror-execute ran: 1,564 stale SP files moved to recycle bin, 160 refreshed, 11 user-touched files protected. Datto Workplace Server service stopped + disabled on ACG-DWP-X-BB (source frozen). Ticket #32187 billed 5.0h Labor - Remote ($150/hr). |
| 2026-06-26 | Mike (GURU-5070): Google→M365 mail migration initiated; BB-Batch1 live (14 mailboxes, Status: Syncing). Identified Datto/SPMT migration VM as Jupiter libvirt domain ACG-DWP-X-BB (actual WS2019 build 17763); had APIPA after ~2 months parked; fixed with static IP 172.16.3.45/22; GuruRMM agent enrolled (`a4524e85-…`); Datto Workplace Server reconnected + re-syncing. Fully onboarded BirthBio M365 to ACG suite (Exchange Operator + User Manager + Defender Add-on consented). Provisioned Exchange-only mailboxes for Dr. Chris Gillis (`medicaldirector@`) and Michael Merritt (`mmerritt@`); license redistribution: Mei Mei + Valerie +BP, Savanna BP→EXO, 4 disabled formers +EXO. Created Gmail migration endpoint BB-Gmail; created + auto-started BB-Batch1. Vaulted Google super-admin creds + new M365 user passwords. |
| 2026-06-02 | Mike (BEAST/discord-bot): SMARTBADGE-WATCH fired a false-positive DRIFT alert. Root cause: `jq -r '.stdout'` emitting literal `"null"` when RMM API returned JSON null stdout. Live re-verify via RMM confirmed KSTEENBB2025 clean (`RESULT: PASS`). Fixed `check-ksteen-smartbadge.sh` (commit `551aaf2`): `.stdout // empty` coercion, INFRA-ERROR vs DRIFT distinction, stderr/exit_code in diagnostics, poll window 80s→120s. |
| 2026-05-29 | Mike: Corrected the SmartBadge fix — Kristin's machine had been left on the *older* Workplace Desktop v8 (diverged from fleet). Revo-removed v8, installed Workplace v10.53.4 (Workplace2), aligned SmartBadge `_CC` add-in/CLSID to EVO-X1, cleared her stuck per-user `LoadBehavior=2`. Verified working. Public tech notes + 1hr warranty on Syncro #32339. Stood up a 7-day daily verification (scheduled task on GURU-5070 + coord todo `4a5b09b3`, expires 2026-06-05). |
| 2026-05-28 | Mike: Initial Kristin Steen SmartBadge remediation (Syncro #32339) — diagnosed dual Workplace2/Workplace Desktop install; **uninstalled the wrong one (Workplace2 v10)**, leaving v8 Desktop (corrected 2026-05-29). |

View File

@@ -24,7 +24,7 @@ Run `/wiki-lint` to check for stale entries and broken backlinks.
| [Jimmy Company](clients/jimmy.md) | Break-fix, $150/hr; single aging workstation BLASTER2 (Win10 22H2 EOL, i5-3470/3.8GB — replace); backups the recurring theme (QuickBooks data); onboarded to GuruRMM 2026-06-19 (RDP NLA + Kaseya removal + cleanup); MSP360 local backup drive full, 90-day retention set, space reclaim pending in console (cloud B2 healthy) | 2026-06-19 |
| [Valley Wide Plastering](clients/valleywide.md) | Prepaid block, 15.5 hrs remaining; plastering/stucco contractor; HP DL360 Gen10 + XenServer; VWP-FILES (G:) on Hyper-V — SMB1 enabled for the legacy XP Orders VM (V-XP); VB6 app modernization project; RDWeb brute-force incident; 11 Yealink phones pending | 2026-06-23 |
| [ACG Internal Infrastructure](clients/internal-infrastructure.md) | ACG's own hosting infra — Neptune Exchange (cert expires 2026-05-31, DkimSigner disabled), IX server, Cloudflare tunnel workaround, ACG M365 tenant gaps | 2026-05-24 |
| [BirthBiologic](clients/birth-biologic.md) | Bio/healthcare (cord blood/donor services), Stilwell KS; Syncro 17983014, prepaid 10.0 hrs; **Google Workspace→M365 mail migration LIVE** (Batch 1 syncing — 14 mailboxes, mail+cal+contacts; MX still on Google, cutover pending); tenant FULLY onboarded (Exchange Operator/User Manager/Defender added 2026-06-26); 14 Business Premium + 7 Exchange-Online-P1 (all consumed); **Datto→SharePoint** migration VM ACG-DWP-X-BB (Jupiter, 172.16.3.45) recovered + RMM-enrolled + re-syncing — 4 SPMT folders (Admin/Donor Services 109GB/Quality/Activity) UNCONFIRMED pending sync | 2026-06-26 |
| [BirthBiologic](clients/birth-biologic.md) | Bio/healthcare (cord blood/donor services), Stilwell KS; Syncro 17983014, prepaid 3.0 hrs; **Datto→SharePoint migration COMPLETE** — all sites reconciled 0 missing; Quality Systems Department final sync done 2026-06-30 (3,768 files, 4 live-work files preserved); **MX cut to M365 2026-06-27**, mail Batch 1 Synced (14/14), Batch 2 (5 formers) pending; tenant fully onboarded (Exchange Operator/User Manager/Defender); migration VM ACG-DWP-X-BB (Jupiter, 172.16.3.45, Datto svc frozen); active: #32187 Scheduled — off-hours site rename → "Quality Department" (Wed 2026-07-01 7-8 PM MST) | 2026-07-01 |
| [CryoWeave](clients/cryoweave.md) | Custom cryogenic cable assemblies; cPanel on IX; website redesign + SEO project in progress; Syncro ID not documented | 2026-05-24 |
| [Darrell Delphen](clients/darrell-delphen.md) | Break-fix residential (Yantis, TX); single Windows workstation DDDOffice072023 (GuruRMM); 2026-06-18 Outlook email links failing = ISP-managed Extreme EXOS gateway "NetIQ" SNI-filtering of Intermedia's url.emailprotection.link rewriter (WARP interim bypass, ISP disabled the feature for permanent fix); Syncro #35996725 | 2026-06-18 |
| [Glaz-Tech Industries](clients/glaztech.md) | ~200 users, 9 locations; prepaid ~22.25 hrs; web server WWW (192.168.8.72 / 65.113.52.88) — IIS 10/VB.NET e-commerce; CRITICAL security posture: website connects to GTI-INV-SQL as sysadmin (login `tom`, named SQL login, C0 top finding) + plaintext PANs+CVV (stored by GTIware PSA, not website) + plaintext passwords + SQLi via `quo()` + XSS; apex 404 fixed + payment TLS fixed 2026-06-03; intrusion/brute-force log review 2026-06-04 (no attacker found; H5 detection blind spot confirmed — HTTP 200 on both success/failure + no failed-login logging); #32378 Waiting on Customer (assessment + reports + Appendix A delivered); M365 no MFA; SCL bypass rules for vendor DMARC + MailProtector digests | 2026-06-04 |