diff --git a/clients/birth-biologic/session-logs/2026-06/2026-06-26-mike-birthbio-mail-migration-and-datto-vm.md b/clients/birth-biologic/session-logs/2026-06/2026-06-26-mike-birthbio-mail-migration-and-datto-vm.md new file mode 100644 index 00000000..9ad72152 --- /dev/null +++ b/clients/birth-biologic/session-logs/2026-06/2026-06-26-mike-birthbio-mail-migration-and-datto-vm.md @@ -0,0 +1,193 @@ +## User +- **User:** Mike Swanson (mike) +- **Machine:** GURU-5070 +- **Role:** admin + +## Session Summary + +Worked two BirthBiologic migrations in parallel: the Datto Workplace → SharePoint migration and a +new Google Workspace → M365 mail migration. The Datto/SharePoint thread started as a "verify current +state" task but Mike clarified the migration host is a Jupiter VM, not BB-SERVER. Located it as the +libvirt domain "Windows Server 2016" (actual Windows hostname **ACG-DWP-X-BB**, actually Server 2019 +build 17763) — an ACG-owned migration box running Datto Workplace Server + SPMT, **not enrolled in +RMM** and sitting on an APIPA address (no LAN). Diagnosed: host bridging was fine (vnet14 enslaved to +br0, carrier up); the guest simply wasn't getting a DHCP lease from pfSense after ~2 months parked. +Fixed with a static IP (172.16.3.45/22), installed the GuruRMM agent (enrolled under BirthBiologic / +Main Office), and confirmed Datto Workplace Server reconnected and is re-syncing. Established (via the +qemu guest agent and SPMT job storage) that the April 2026 migration only completed Supply Management +(160 files, custom script) + ITSvcs (excluded); the four large folders (Admin 5.8 GB, Donor Services +109 GB, Quality 28 GB, Activity Reports) were SPMT's job and last ran 2026-04-29 — completion still +unconfirmed. Per Mike, full reconciliation waits until Datto finishes re-syncing. + +The larger thread was standing up the **Google Workspace → M365 mail migration** end-to-end. Confirmed +the Google super-admin (`sysadmin@birthbiologic.com`) lives in 1Password (Clients vault item "Google"); +read it via the SOPS-vaulted 1Password service-account token and mirrored it into SOPS. Onboarded +BirthBio's tenant for **Exchange Operator** (already had Tenant Admin consented, so the suite was +provisioned programmatically — Exchange Operator SP created + Exchange Administrator role). Pulled the +authoritative Google roster via domain-wide delegation (20 accounts: 15 active, 5 suspended), +reconciled against M365, and surfaced two active accounts not on Mike's list (Dr. Chris Gillis +`medicaldirector@`, Michael Merritt `mmerritt@`) plus an address mismatch (Mindi is `mindim@` in +Google, `mmaher@` in M365). + +Provisioned the M365 target side to Mike's licensing rules: active-12 → Business Premium (assigned BP +to Mei Mei + Valerie, freed Savanna's BP by moving her to Exchange-only); created Gillis + Merritt with +Exchange-only and vaulted their passwords; licensed the 4 disabled former employees with Exchange-only +(kept sign-in disabled) as future shared-mailbox targets. License math closed exactly: 14 Business +Premium + 7 Exchange Online Plan 1, all consumed. + +Hit a real blocker creating the Gmail migration endpoint: Google returned `unauthorized_client … not +authorized for any of the scopes requested`. Root cause = the DWD grant had only 3 of Microsoft's +required **5** scopes (missing `m8/feeds` and `gmail.settings.sharing`); Google rejects the migration +token all-or-nothing. Verified the exact 5-scope string against live MS Learn + a Grok live-search +cross-check (Gemini CLI was down on this box), updated our runbook, and Mike re-authorized all 5 in the +BB Google console. After that the endpoint (`BB-Gmail`) created cleanly and **Batch 1 (14 live +mailboxes, mail + calendar + contacts) was created and auto-started — Status: Syncing**. + +## Key Decisions + +- **Datto VM gets a static IP (172.16.3.45), not a pfSense DHCP fix.** The fault was pfSense not + leasing this MAC after a long park; a static on the ACG server range (172.16.3.x) is the reliable, + convention-consistent fix. Follow-up: add a pfSense reservation or confirm it's outside the DHCP pool. +- **Enrolled the Datto VM under BirthBiologic / Main Office** (not AZ Computer Guru) since the box exists + solely for BirthBio's migration and we had that site key; reversible (agents can be moved). +- **Former employees migrate to shared mailboxes via a temp Exchange-only license** (migrate into a + licensed mailbox → convert to shared ≤50 GB = free → reclaim license). Source Google accounts must be + **un-suspended** during migration (Gmail API can't read suspended accounts). +- **Licensing tiers (Mike's rules):** active-users-list → Business Premium; live Google accounts not yet + in M365 (Gillis, Merritt) → Exchange-only ("E1" = Exchange Online Plan 1); formers → Exchange-only + (reclaimable). `operations@` stays BP through migration. Existing BP users left on BP (not downgraded). +- **Batch sequencing:** live users first (Batch 1); formers as Batch 2 after un-suspending them in Google + and freeing Workspace seats by suspending already-migrated live users. +- **Mindi mapped via the CSV `Username` column** (`EmailAddress=mmaher@`, `Username=mindim@`) — the + proper MS mechanism — plus a belt-and-suspenders `mindim@` proxy on her mailbox. +- **Target delivery domain = `birthbiologic.onmicrosoft.com`** for Batch 1 (no routing subdomain exists; + acceptable for a near-term cutover; MS prefers a subdomain for long coexistence). +- **Drove Exchange via REST `InvokeCommand`** (Exchange Operator app token) — the EXO PowerShell module + isn't installed and the app has no vaulted cert, so app-only Connect-ExchangeOnline wasn't available. + +## Problems Encountered + +- **Datto VM on APIPA (no LAN).** Host bridging fine; pfSense wasn't leasing the MAC. Fixed with static + 172.16.3.45/22, GW 172.16.0.1, DNS 172.16.0.1+1.1.1.1. Verified gateway/internet/DNS + RMM check-in. +- **`vault.sh get-field` returned `null` (len 4)** for nested secrets until the field arg used dotted + path: `credentials.client_secret`, `credentials.credential`. Plain leaf names don't resolve. +- **SPB skuId mismatch.** The scope doc's BP GUID (`cbdc14ab-d96c-4132-b7f4-1f3a3a819bb4`) was stale; the + tenant's real SPB skuId is `cbdc14ab-d96c-4c30-b9f4-6ada7cdc1d46`. License assign 400'd until corrected. +- **License seat propagation lag** — Valerie's BP assign 400'd ("no available licenses") immediately after + freeing Savanna's seat; succeeded on retry seconds later. +- **`proxyAddresses` read-only via Graph** — adding Mindi's alias required Exchange `Set-Mailbox` (EXO), + not a Graph PATCH. +- **Gmail migration endpoint failed: `unauthorized_client … not authorized for any of the scopes + requested`.** DWD had 3 of 5 required scopes. Got the verbatim 5-scope string from MS Learn + Grok; + Mike re-authorized; endpoint then created. +- **`onboard365.sh` vault path** — looked at `/c/Users/guru/.claude/identity.json`; fixed by exporting + `VAULT_ROOT_ENV=/d/vault` (logged as friction). +- **GCP API enable initially run as the wrong identity** — Mike first ran `gcloud services enable` as + `sysadmin@birthbiologic.com` (no rights to ACG's project); succeeded once run as the ACG owner of + `acg-msp-access`. +- **Gemini CLI down** (`throwIneligibleOrProjectIdError`, needs interactive re-login) — used Grok for the + live-doc cross-check instead. Logged to errorlog. + +## Configuration Changes + +- **ACG-DWP-X-BB (Jupiter "Windows Server 2016" VM):** static IP 172.16.3.45/22, GW 172.16.0.1, DNS + 172.16.0.1 + 1.1.1.1 (persistent). GuruRMM agent installed (universal installer), enrolled BirthBiologic + / Main Office, agent `a4524e85-8a07-45d0-91b1-51ce7e2ca74a`. +- **BirthBio M365 tenant (19a568e8-…):** onboarded Exchange Operator (+ Defender Add-on) SPs via + `onboard365.sh provision`; roles assigned (Exchange Admin on Exchange Operator + Security Investigator, + CA Admin on Tenant Admin, User Admin + Auth Admin on User Manager). + - License changes: Mei Mei (`msenthavy`) +BP; Valerie (`vvaneaton`) +BP; Savanna (`sabron`) BP→EXO; + created `medicaldirector@` (Gillis) +EXO and `mmerritt@` (Merritt) +EXO; licensed `aboutte`, `araso`, + `khoffman`, `pnelson` with EXO (kept sign-in disabled). + - `Set-Mailbox mmaher@` added secondary `smtp:mindim@birthbiologic.com`. + - Created Gmail migration endpoint `BB-Gmail`; created + auto-started migration batch `BB-Batch1` (14 + users, TargetDeliveryDomain `birthbiologic.onmicrosoft.com`, NotificationEmails sysadmin@). +- **Vault (pushed):** `clients/birth-biologic/google-workspace.sops.yaml`, + `clients/birth-biologic/m365-medicaldirector.sops.yaml`, `clients/birth-biologic/m365-mmerritt.sops.yaml`. +- **Repo:** updated `projects/msp-tools/runbooks/google-workspace-to-m365-migration.md` (exact 5-scope + string + all-or-nothing gotcha + Contacts-API-retired/People-API + GCP-owner notes). +- **errorlog.md:** gemini CLI failure entry (+ onboard365 vault-path friction). + +## Credentials & Secrets + +- **Google Workspace super-admin** `sysadmin@birthbiologic.com` (source tenant) — sourced from 1Password + Clients vault item "Google"; mirrored to SOPS `clients/birth-biologic/google-workspace.sops.yaml` + (`credentials.password`, 19 chars). Used for admin.google.com console (DWD/API) + as the migration + impersonation admin. +- **M365 mailbox — Dr. Chris Gillis** `medicaldirector@birthbiologic.com` — created this session; password + vaulted at `clients/birth-biologic/m365-medicaldirector.sops.yaml` (forceChangePasswordNextSignIn=true). +- **M365 mailbox — Michael Merritt** `mmerritt@birthbiologic.com` — created this session; password vaulted + at `clients/birth-biologic/m365-mmerritt.sops.yaml` (forceChangePasswordNextSignIn=true). +- App secrets used (already vaulted): Tenant Admin `msp-tools/computerguru-tenant-admin` + (`credentials.client_secret`); Exchange Operator `msp-tools/computerguru-exchange-operator` + (`credentials.client_secret`); Google SA `msp-tools/acg-msp-access-google-workspace` + (`credentials.credential`, full JSON); 1Password service token + `infrastructure/1password-service-account.sops.yaml`. + +## Infrastructure & Servers + +- **ACG-DWP-X-BB** — Jupiter libvirt domain "Windows Server 2016" (actually WS2019, build 17763). Windows + hostname ACG-DWP-X-BB. NIC virtio 52:54:00:d4:8e:59 on br0 (vnet14). Static 172.16.3.45/22. Runs Datto + Workplace Server (svc `datto_workplace_server.default`, proc WorkplaceServer) + SPMT (under + Administrator profile). RMM agent `a4524e85-8a07-45d0-91b1-51ce7e2ca74a`. Datto source tree + `C:\Users\Public\Desktop\Datto Workplace Server Projects`. +- **Jupiter** 172.16.3.20 (Unraid, virsh host). LAN 172.16.0.0/22, GW pfSense 172.16.0.1. guest-exec + helper at `/root/gx.sh` on Jupiter. +- **BB-SERVER** — RMM agent `6c02baa7-0f1c-4990-b466-c9ab9eaefd3b`. Also has Datto Workplace Server + the + original custom-script artifacts at `C:\GuruMigration` (bb-migration-state.json shows 160 Supply Mgmt + + 49 ITSvcs uploaded in April). +- **BirthBio M365 tenant** `birthbiologic.com` / `19a568e8-9e88-413b-9341-cbc224b39145`. + - SPs: Tenant Admin `7a199b11-97fb-4e65-917d-f8d29a53ba49`; Exchange Operator + `bab4699b-32a3-4434-9cad-7a4a08cc4d9e`; Security Investigator `bf684a4b-…`; User Manager `3347ebcc-…`; + Defender Add-on `161b8f61-…`. New user objects: Gillis `1bd491e1-3ba6-4214-8c6d-46426f8681da`, Merritt + `117a3367-cd5f-4565-af11-af5ff089224f`. + - SKUs: Business Premium (SPB) `cbdc14ab-d96c-4c30-b9f4-6ada7cdc1d46` (14/14); Exchange Online Plan 1 + (EXCHANGESTANDARD) `4b9405b0-7788-4568-add1-99614e613b69` (7/7). Accepted domains: birthbiologic.com + (default), birthbiologic.onmicrosoft.com. +- **Google project** `acg-msp-access` (number 806899474449). SA `acg-msp-access@acg-msp-access.iam.gserviceaccount.com`, + OAuth2 client ID `102231607889615995452`. APIs enabled: Gmail, Calendar (calendar-json), People. +- **Google roster (DWD pull):** 15 active, 5 suspended. Active staff emails per `clients/birth-biologic/ + docs/migration/google-to-m365-scope.md`; Mindi = `mindim@` (Google) ↔ `mmaher@` (M365). + +## Commands & Outputs + +- **Required Google DWD scopes (exact, 5, comma-separated, no spaces):** + `https://mail.google.com/,https://www.googleapis.com/auth/calendar,https://www.google.com/m8/feeds/,https://www.googleapis.com/auth/gmail.settings.sharing,https://www.googleapis.com/auth/contacts` + (`m8/feeds` is a still-valid alias for the contacts scope, served by People API; legacy Contacts API + retired 2022, not enableable, not needed.) +- EXO via REST: `POST https://outlook.office365.com/adminapi/beta/{tenant}/InvokeCommand` with Exchange + Operator app token (`scope=https://outlook.office365.com/.default`), body + `{"CmdletInput":{"CmdletName":"…","Parameters":{…}}}`. byte[] params (ServiceAccountKeyFileData, CSVData) + passed as **base64 strings**. +- `New-MigrationEndpoint -Gmail -Name BB-Gmail -ServiceAccountKeyFileData -EmailAddress sysadmin@birthbiologic.com` → created. +- `New-MigrationBatch -Name BB-Batch1 -SourceEndpoint BB-Gmail -CSVData -TargetDeliveryDomain birthbiologic.onmicrosoft.com -AutoStart -NotificationEmails sysadmin@` → Status=Syncing, Total=14. +- Get-MigrationUser BB-Batch1 → 14 Provisioning, 0 skipped (normal initial state). +- Datto source counts (ACG-DWP-X-BB): Admin 6,279/5.8GB · Donor Services 56,826/109GB · Quality 3,714/28GB + · Supply Mgmt 160/33MB · Activity Reports 1 · ITSvcs 52 (excluded). + +## Pending / Incomplete Tasks + +- **Batch 1 monitor → MX cutover.** Watch `BB-Batch1` Provisioning→Syncing→Synced. When Synced: flip MX in + SiteGround → M365, update SPF (`include:spf.protection.outlook.com`), enable/publish DKIM (2 CNAMEs), + autodiscover CNAME → autodiscover.outlook.com, run final delta, then **complete** the batch. +- **Batch 2 — 5 former employees → shared.** Un-suspend each in Google (free Workspace seats by suspending + migrated live users), run a Gmail batch (targets already EXO-licensed: aboutte, araso, khoffman, pnelson, + sabron), then convert to shared mailboxes and reclaim the 5 EXO licenses. +- **Datto → SharePoint reconciliation.** After ACG-DWP-X-BB finishes re-syncing with Datto cloud, compare + source vs each SharePoint site to confirm what the April SPMT run left unfinished (Admin / Donor Services + / Quality / Activity Reports). +- **pfSense:** add a DHCP reservation for 172.16.3.45 (MAC 52:54:00:d4:8e:59) or confirm it's outside the pool. +- **Valerie VanEaton** — active (receiving daily; last *sent* 2026-05-13). Julie to confirm whether the + mid-May send drop-off = leave/departure; if departed, move her to the former→shared track. +- **Decisions still open:** confirm Merritt's long-term tier; whether `operations@` becomes shared post-migration. +- **Wiki:** BirthBio article is stale (says migration incomplete / 13 mailboxes) — recompile. + +## Reference Information + +- Migration scope doc: `clients/birth-biologic/docs/migration/google-to-m365-scope.md`. +- Runbook (updated): `projects/msp-tools/runbooks/google-workspace-to-m365-migration.md`. +- MS Learn: `manually-configuring-gsuite-for-migration` (scope string), `automated-migration-neweac`, + `google-workspace-migration-prerequisites`, `perform-g-suite-migration`. +- RMM install one-liner (BirthBio site): `irm https://rmm.azcomputerguru.com/install/BRIGHT-PEAK-5980/windows | iex`. +- Discord DMs to Mike: message_id 1520034139900739627 (initial DWD), 1520055625302675537 (corrected 5-scope). +- Vault enrollment key: `clients/birth-biologic/gururmm-site-main` (site BRIGHT-PEAK-5980, id 3b20ef97-…). diff --git a/errorlog.md b/errorlog.md index b953d8da..12d06209 100644 --- a/errorlog.md +++ b/errorlog.md @@ -17,6 +17,18 @@ Categories (the `[type]` tag): _(none)_ = skill/command execution failure · +2026-06-26 | GURU-5070 | agy/gemini | gemini CLI headless failed: throwIneligibleOrProjectIdError / _doSetupUser (auth-eligibility, needs interactive re-login) [ctx: task=verify-gws-migration-scopes] + +2026-06-26 | GURU-5070 | agy | gemini returned no response (empty after 3 attempts) [ctx: mode=search err= at process.processTicksAndRejections (node:internal/process/task_queues:104:] + +2026-06-26 | GURU-5070 | remediation-tool | onboard-tenant: grant_app_role appRoleAssignment failed [ctx: role=bf394140-e372-4bf9-a898-299cfc7564e5 msg=Resource '161b8f61-5c16-4e1a-9a23-4bb7076b0946' does not exist or one of its que] + +2026-06-26 | GURU-5070 | remediation-tool | onboard-tenant: grant_app_role appRoleAssignment failed [ctx: role=6931bccd-447a-43d1-b442-00a195474933 msg=Resource 'bab4699b-32a3-4434-9cad-7a4a08cc4d9e' does not exist or one of its que] + +2026-06-26 | GURU-5070 | remediation-tool | onboard-tenant: grant_app_role appRoleAssignment failed [ctx: role=df021288-bdef-4463-88db-98f22de89214 msg=Resource 'bab4699b-32a3-4434-9cad-7a4a08cc4d9e' does not exist or one of its que] + +2026-06-26 | GURU-5070 | remediation-tool | onboard-tenant: failed to acquire Tenant Admin token [ctx: tenant=19a568e8-9e88-413b-9341-cbc224b39145 exit=3] + 2026-06-26 | Howard-Home | drive-map | drive-map verify failed on DESKTOP-LPOPV30 [ctx: cmd=e932bc94-0557-4913-a0b1-c97c1aa5da26] 2026-06-26 | Howard-Home | drive-map | drive-map verify failed on DESKTOP-LPOPV30 [ctx: cmd=18fec38b-8fae-4a1b-a3d8-5b90b124dbc2] diff --git a/projects/msp-tools/runbooks/google-workspace-to-m365-migration.md b/projects/msp-tools/runbooks/google-workspace-to-m365-migration.md index 9b02def3..6fad1c61 100644 --- a/projects/msp-tools/runbooks/google-workspace-to-m365-migration.md +++ b/projects/msp-tools/runbooks/google-workspace-to-m365-migration.md @@ -48,8 +48,14 @@ ACG already has a Google service account for Workspace access: ## 4. MS native migration — end to end **Step 1 — Source (Google) prep** -1. In **GCP** (project `acg-msp-access` or a new one): ensure the service account exists and a JSON key is in the vault. Enable APIs: **Gmail, Google Calendar, Google People (Contacts), Admin SDK (Directory)**. -2. In the SOURCE **Google Admin console** → Security → API controls → **Domain-wide delegation** → add the SA **Client ID** with the Microsoft-required OAuth scopes (Gmail/Calendar/Contacts/Directory — copy the exact scope list from the EAC migration wizard so they match). +1. In **GCP** (project `acg-msp-access` or a new one): ensure the service account exists and a JSON key is in the vault. Enable APIs: **Gmail API, Google Calendar API, People API**. (The legacy *Contacts API* was retired by Google in 2022 and **cannot be enabled** — the `m8/feeds` contacts scope is now an alias served by the People API, so People API enablement covers it. Enabling the APIs in `acg-msp-access` requires being signed in as the **ACG owner** of that project — a *client* super-admin has no rights to ACG's GCP project.) +2. In the SOURCE **Google Admin console** → Security → API controls → **Domain-wide delegation** → add the SA's **OAuth2 Client ID** (the SA's numeric "Unique ID", NOT the app client_id) with the **exact 5-scope string below, comma-separated, no spaces**. Google rejects the migration token request **all-or-nothing** — if even one scope is missing the endpoint fails later with `unauthorized_client … not authorized for any of the scopes requested`. Verified current 2026-06 (MS Learn `manually-configuring-gsuite-for-migration` + Grok live cross-check): + + ``` + https://mail.google.com/,https://www.googleapis.com/auth/calendar,https://www.google.com/m8/feeds/,https://www.googleapis.com/auth/gmail.settings.sharing,https://www.googleapis.com/auth/contacts + ``` + + Propagation can take 15 min–24 h (usually minutes). Do NOT rely on a smaller "mail+calendar+contacts" set — `m8/feeds` and `gmail.settings.sharing` are both required by the MS endpoint. 3. Confirm a Google super-admin mailbox exists for the migration to impersonate. **Step 2 — Target (M365) prep** diff --git a/wiki/systems/uos-server.md b/wiki/systems/uos-server.md index 914d75e8..657d7b7e 100644 --- a/wiki/systems/uos-server.md +++ b/wiki/systems/uos-server.md @@ -2,10 +2,11 @@ type: system name: uos-server display_name: UOS Server (UniFi OS Server) -last_compiled: 2026-06-21 -compiled_by: HOWARD-HOME/claude-main +last_compiled: 2026-06-26 +compiled_by: GURU-5070/claude-main sources: - session-logs/2026-06/2026-06-21-howard-unifi-pfsense-control-verbs.md + - 2026-06-26 Rocky 9.1->9.8 host OS patch (mike) backlinks: - systems/jupiter - systems/pfsense @@ -23,7 +24,7 @@ backlinks: ## What / where it is -- **Guest:** Rocky Linux 9.1, hostname-internal "UOS Server". Guest IP **`172.16.3.29`** (ACG internal LAN). +- **Guest:** Rocky Linux **9.8** (kernel `5.14.0-687.17.1.el9_8`; last patched 2026-06-26), hostname-internal "UOS Server". Guest IP **`172.16.3.29`** (ACG internal LAN). - **Hypervisor:** Jupiter (`172.16.3.20`, Unraid) — virsh domain **`Unifi`** (id 1). `virsh list` to confirm running. - **App stack (inside guest):** UniFi Network = `ace.jar` (Java) + classic **MongoDB `ace`** on `127.0.0.1:27117`, plus `unifi-core` (Postgres) for UniFi-OS identity/integration. All of it runs **inside a rootless podman container `uosserver`** (host user `uosserver`, uid 1000) — so the app files and mongo are NOT on the guest rootfs. - **ui.com cloud:** host id `2d6b654d-9b79-4eaa-b2e1-52062a5690ef` in the Site Manager account. @@ -85,6 +86,30 @@ There is **no mongo client on the guest host**; the shell is `/usr/bin/mongo` *i - **`rogue`** — neighbor/over-the-air BSSIDs seen by APs. **Not ACG gear** — a MAC hit here is someone else's WiFi, ignore it for device hunts. - **Pending/unadopted devices:** the controller only persists a discovered device into `device` with `adopted:false`. If `db.device.count({adopted:false})` is `0`, there are **no** pending devices controller-wide — an "unadopted" device that returns nothing here simply has not reached this controller (not on a network it can discover, or managed by a different console). The cloud API and integration API show adopted gear only, so they cannot find it either; locating it then needs L2/DHCP/ARP on the gateway of the site it is physically cabled to. +## Host OS maintenance (Rocky) + +The UniFi app self-updates (the `uosserver-updater.service` rebuilds the rootless podman +container; UOS Server **5.1.19** / Network **10.4.57** as of 2026-06-26). The **host OS is +NOT auto-patched** — it must be updated manually over SSH. The two layers are independent: +patching the host does not touch the container/UniFi data (named podman volumes persist). + +**Procedure** (root via the fleet SSH key): +```bash +ssh -i root@172.16.3.29 'dnf -y update' # ~few min; kernel updates need a reboot +ssh -i root@172.16.3.29 'systemctl reboot' # controller drops briefly; APs keep serving WiFi +# verify after ~3-4 min: +ssh -i root@172.16.3.29 'uosserver status' # container Up (healthy) +ssh -i root@172.16.3.29 'curl -sk -o /dev/null -w "%{http_code}\n" https://127.0.0.1:11443/' # 200 +``` +- **Safety net:** daily UniFi auto-backups live at + `~uosserver/.local/share/containers/storage/volumes/uosserver_var_lib_unifi/_data/backup/autobackup/` + (newest `autobackup__.unf`) — survives the OS update. For the OS layer itself, + a virsh snapshot of the **"Unifi"** VM on [[jupiter]] is the rollback point (optional). +- **Reboot impact:** only central management/stats drop while the VM reboots (~30-60s back to + SSH, ~3-4 min to container healthy); APs/switches keep forwarding traffic the whole time. +- **History:** 2026-06-26 — Rocky **9.1 -> 9.8** (362 pkgs, kernel `162.6.1.el9_1` -> + `687.17.1.el9_8`, full security backlog cleared); clean reboot, controller back healthy. + ## Related tooling — pfSense gateway layer (works together) This UOS controller and the **pfSense gateway tooling** are the two halves of the **`unifi-wifi`