From 46d2ae7a5bab37f95bd0fea38cf46d3fe585e381 Mon Sep 17 00:00:00 2001 From: Howard Enos Date: Fri, 10 Jul 2026 15:12:08 -0700 Subject: [PATCH] sync: auto-sync from HOWARD-HOME at 2026-07-10 15:11:39 Author: Howard Enos Machine: HOWARD-HOME Timestamp: 2026-07-10 15:11:39 --- .../2026-07-10-howard-server-migration.md | 166 ++++++++++++++++++ 1 file changed, 166 insertions(+) create mode 100644 clients/prairie-schooner/session-logs/2026-07/2026-07-10-howard-server-migration.md diff --git a/clients/prairie-schooner/session-logs/2026-07/2026-07-10-howard-server-migration.md b/clients/prairie-schooner/session-logs/2026-07/2026-07-10-howard-server-migration.md new file mode 100644 index 00000000..83844287 --- /dev/null +++ b/clients/prairie-schooner/session-logs/2026-07/2026-07-10-howard-server-migration.md @@ -0,0 +1,166 @@ +## User +- **User:** Howard Enos (howard) +- **Machine:** Howard-Home +- **Role:** tech + +# The Prairie Schooner — Server Migration (TPS-SERVER -> TPS-SVR) + +> Work performed 2026-07-07; logged 2026-07-10. Migration is PAUSED mid-way — see Pending. + +## Session Summary + +Diagnosed the two Prairie Schooner servers to plan replacing the aging **TPS-SERVER** +(Windows Server 2016 Essentials, Lenovo ThinkServer TS140, 3.8 GB RAM, C: 87% full, +backup dead since 2018) with **TPS-SVR** (Windows Server 2019 Standard, HPE ProLiant +MicroServer Gen10 Plus v2, 32 GB RAM, 5.3 TB). TPS-SERVER was the sole DC (all 5 FSMO), +Enterprise Root CA, DNS, file server, and QuickBooks Enterprise 24 database host. TPS-SVR +had roles pre-installed but was never configured (AD DS role present, never promoted; +DHCP/DNS installed but inert). Domain `TPS.local`, single-domain forest, functional level +2016 (schema v87). + +Started with data protection. A Windows Server Backup BMR to `\\TPS-SVR\TPSBackup` failed +at ~12% ("media is write protected" — the known WSB-VHDX-over-SMB fragility), so pivoted +to robocopy pre-seed of the file/QuickBooks data (which doubles as migration step 5). +Pre-seeded `C:\Share` (Quickbooks 144 GB + Deployment 22 GB) TPS-SERVER -> `\\TPS-SVR\Share`. +Rebooted the three workstations (Mel-PC, TPS-JAYMI, TPS-Tina) with a 2-minute warning to +release QuickBooks client locks, then stopped the server QB service briefly to copy a clean, +consistent `schoonerQB2025.QBW` (3 GB, verified byte-for-byte), and restarted QB so users +resumed on the old server. + +Ran a full role audit of TPS-SERVER: no shared printers, no folder redirection/roaming/home +dirs/logon scripts, no real QuickBooks Web Connector integration (only Intuit test samples), +DFS namespace not published. The Q: drive map (`\\tps-server\quickbooks`) and software-deploy +GPOs are all keyed to the name `tps-server`, and QuickBooks references the host by IP in the +`.ND` (`ServerIp=192.168.1.125`) — but the `.ND` regenerates when the hosting server changes. + +Promoted TPS-SVR to a domain controller. `Test-ADDSDomainControllerInstallation` first failed +because the built-in Administrator was disabled, then because its password did not match the +value Howard provided; enabled + reset Administrator (via guru) to `r3tr0gradE99#`, test passed, +ran the real promotion as a detached SYSTEM task. Promotion succeeded; TPS-SVR rebooted and came +up as a healthy 2nd DC + GC + DNS, AD/DNS fully replicated (17 users, 11 computers, 6 GPOs, +zones), replication 0 errors both ways. Set TPS-SVR IPv4 static (192.168.1.135), cleaned its +DNS/forwarders. Re-disabled Administrator. + +Hit a blocking constraint: **no SonicWall access.** The SonicWall (192.168.1.1) is gateway + +DHCP and hands clients DNS = 192.168.1.125 (old DC); we cannot change that, so retiring the old +DC would break client DNS unless the new server assumes .125. Howard decided to replace the +SonicWall with a UniFi gateway (to regain DHCP/DNS control) and PAUSE the migration. Left the +OLD server as primary (all FSMO, QB host, file server, client DNS), new server as an idle +healthy replica. Verified stable, cleaned up one-time tasks, saved resume state to the wiki. + +## Key Decisions + +- **robocopy pre-seed instead of BMR image** — WSB BMR to an SMB share failed ("media write + protected"); robocopy is reliable and doubles as a migration step. Copied data+timestamps + only (`/COPY:DAT`), ACLs deferred to cutover (machine account can't set ownership remotely). +- **Clean QB copy via brief service stop** — rebooted workstations to drop client locks, stopped + `QuickBooksDB34`+`QBCFMonitorService` on the server to prevent re-lock during the ~3 GB copy, + restarted after so users resume on the old server. Final `.QBW` delta re-copied at cutover. +- **Alias approach over rename/re-IP (initially)** — keep TPS-SVR's own name/IP and add + `TPS-SERVER` as a DNS+SPN alias, avoiding a risky live-DC rename and IP collision. +- **REVERSED by SonicWall lockout** — since we can't change the DHCP-handed DNS (.125), the new + server must end up owning .125 so client DNS survives old-DC retirement. Done safely: assign + .125 only AFTER the old DC is demoted+powered off (no collision). Howard's original instinct + (match the old IP) was correct; UniFi swap will restore flexibility. +- **Enable+reset built-in Administrator temporarily** (Howard chose over elevating guru) — the + only account with Enterprise/Schema Admin for the v87->v88 schema adprep. Re-disabled after. +- **Promotion as a detached scheduled task** with `-NoRebootOnCompletion` — a reaper-killed RMM + command mid-dcpromo would leave a half-promoted DC; detached task avoids that and gives a clean + success signal before a controlled reboot. +- **CA = migrate, not decommission** — Enterprise Root CA still auto-enrolls DC certs (fresh + DomainController cert exp 2027). Migrate under the preserved name. MUST test key export first + (key marked non-exportable, on Microsoft Software KSP). + +## Problems Encountered + +- **WSB BMR failed at ~12% ("media is write protected")** — WSB VHDX-over-SMB fragility. + Resolved by pivoting to robocopy for data; system-state/BMR deferred. +- **False disk-full alarm** — a `Get-Volume` reading showed 55 GB free mid-copy (impossible; + source is 405 GB). Stopped the copy as a precaution; verified no symlink loop, disk actually + 4877 GB free — glitched reading. Resumed; copy completed (166.55 GB / 113,971 files, verified). +- **`Test-ADDSDomainControllerInstallation` STATUS: Error, empty `.Messages`** — real error was in + the warning/verbose stream (`*>&1`): first "account is currently disabled" (built-in Admin + disabled), then "user name or password is incorrect" (provided pw wasn't Administrator's). + Enabled + reset Administrator via guru; test then passed. +- **iconv missing on Howard-Home** — `ps-encoded.sh` failed; used `jq -n --arg` and base64 for + script/credential delivery instead. +- **No SonicWall access** — blocks the client-DNS cutover; Howard replacing it with UniFi. + +## Configuration Changes + +- **TPS-SVR promoted to DC** (`Install-ADDSDomainController`, DomainName TPS.local, InstallDns, + GC, Site Default-First-Site-Name, NTDS/SYSVOL default C:\ paths). Forest schema adprep'd v87->v88. +- **TPS-SVR IPv4 set static:** 192.168.1.135/24, gw 192.168.1.1; NIC DNS = 192.168.1.125,127.0.0.1; + DNS server forwarders = 1.1.1.1,8.8.8.8 (removed dead Norton ConnectSafe 199.85.x). +- **TPS-SVR shares created:** `\\TPS-SVR\Share` (C:\Share, data pre-seed target), + `\\TPS-SVR\TPSBackup` (C:\Backups\TPS-SERVER, failed BMR target). Data staged in C:\Share. +- **Domain built-in Administrator:** enabled -> password reset to provided value -> RE-DISABLED + (net state: disabled, as before, but password now known/vaulted). +- **One-time scheduled tasks removed** post-run: TPS-BMR-Backup, TPS-Preseed-Copy (TPS-SERVER), + TPS-DCPromo (TPS-SVR). +- Wiki: `wiki/clients/the-prairie-schooner.md` — added migration-status + resume checklist. +- No changes to TPS-SERVER roles/FSMO/shares — it remains the untouched primary. + +## Credentials & Secrets + +All vaulted (SOPS). Domain `TPS.local`: +- `clients/prairie-schooner/tps-domain-admin` — user `guru` / `r3tr0gradE99!` (Domain Admin; + same on both servers; console+RDP). +- `clients/prairie-schooner/tps-administrator` — built-in `Administrator` / `r3tr0gradE99#` + (Enterprise+Schema+Domain Admin; **currently DISABLED**; pw was reset to this value). +- `clients/prairie-schooner/tps-svr-dsrm` — DSRM (Directory Services Restore Mode) pw + `TpsDSRM-Recover-2026#` for TPS-SVR. +- `clients/prairie-schooner/gururmm-site-main` — pre-existing RMM enrollment key. + +## Infrastructure & Servers + +- Domain: `TPS.local` (NetBIOS TPS), forest/domain functional level 2016, schema now v88. +- **TPS-SERVER** (OLD, primary): 192.168.1.125 (+ .120, + 10.10.10.5 multi-homed cruft), + Server 2016 Essentials, all 5 FSMO, Enterprise Root CA `TPS-TPS-SERVER-CA`, DNS, file server, + QuickBooks host. RMM agent id `4f8dff0c-b033-4f66-9101-9350e32e0a81`. +- **TPS-SVR** (NEW, 2nd DC): 192.168.1.135/24 static, Server 2019 Standard, GC+DNS, no FSMO. + RMM agent id `799675cf-5c94-465a-b653-63e08d016526`. +- Workstations: Mel-PC (192.168.1.149, `eb2f2aeb-f56e-4fc6-8e5a-73257b95e5f0`, user erin), + TPS-JAYMI (`1fbf4e45-1b48-4c82-af85-847fc1344e12`, user jaymi), TPS-Tina + (`c8d76770-dc9e-45b7-94c7-fdcab3e105a2`, user tina). All DHCP, DNS=192.168.1.125. +- Gateway/DHCP: **SonicWall 192.168.1.1** (hands clients DNS=.125) — being replaced with UniFi. +- QuickBooks: Enterprise Server 24 (34.0.401x), company file `C:\Share\Quickbooks\schoonerQB2025.QBW` + (~3 GB), engine `QB_TPS-SERVER_34`, `.ND` `ServerIp=192.168.1.125` ServerPort 64189. +- GuruRMM API `http://172.16.3.30:3001` (vault `infrastructure/gururmm-server.sops.yaml`). + +## Commands & Outputs + +- Pre-seed: `robocopy C:\Share \\TPS-SVR\Share /E /COPY:DAT /DCOPY:DAT /B /R:1 /W:2 /Z /XJ /MT:16` + (detached SYSTEM task). Result: 166.55 GB / 113,971 files. +- Clean QB copy: stop `QuickBooksDB34`,`QBCFMonitorService` -> `robocopy ... schoonerQB2025.QBW + schoonerQB2025.QBW.TLG /COPY:DAT` -> restart services. SRC=DST 3044.8 MB, MATCH True. +- DC test error surfaced only via `Test-ADDSDomainControllerInstallation ... *>&1` (warning stream). +- Replication verified: `repadmin /replsummary` -> 0 fails/5 both directions. +- Static IP: `netsh interface ipv4 set address "name=Embedded LOM 1 Port 1" source=static + address=192.168.1.135 mask=255.255.255.0 gateway=192.168.1.1` (agent "restarted during + execution" = expected NIC blip; verified DHCP=Disabled, PrefixOrigin=Manual). + +## Pending / Incomplete Tasks + +Resume AFTER Howard swaps SonicWall -> UniFi gateway (regains DHCP/DNS control): +1. Transfer 5 FSMO roles to TPS-SVR (briefly re-enable Administrator, then re-disable). +2. Migrate Enterprise Root CA `TPS-TPS-SERVER-CA` — TEST `Backup-CARoleService` key export FIRST + (key non-exportable / Microsoft Software KSP); it still auto-enrolls DC certs. +3. Configure QuickBooks Database Server Manager on TPS-SVR (host `C:\Share\Quickbooks`, service + Automatic, DELETE stale `.ND` so it regenerates with new host IP, open QB firewall ports); + recreate shares with proper ACLs (pre-seed was data-only); re-enable Shadow Copies. +4. With UniFi: point DHCP DNS at the new DC (removes forced .125 takeover); decide final DC IP + (keep .135 + name alias, or assume .125 + name); reserve the DC IP in UniFi DHCP. +5. Cutover (brief, users out of QB): stop QB on old, final `.QBW` delta copy, verify a workstation + opens QB from the new server. +6. Demote + power off TPS-SERVER; add `TPS-SERVER` DNS+SPN alias on new box; clean multi-homing + (.120 / 10.10.10.5 DNS records); remove unused DHCP/Print roles on TPS-SVR; prune legacy GPOs + (IP SEC, NeT Communication, LT Deployment). +7. **Stand up a working backup on the new server** — the original reason this started (nothing has + been backed up since 2018). + +## Reference Information + +- Resume checklist: `wiki/clients/the-prairie-schooner.md` (migration section). +- Syncro cid 3664974. Contact: Tina Coalter, Tucson AZ. +- GPS audit: `wiki/projects/gps-rmm-audit.md`.