sync: auto-sync from HOWARD-HOME at 2026-07-14 20:16:29

Author: Howard Enos
Machine: HOWARD-HOME
Timestamp: 2026-07-14 20:16:29

clients/prairie-schooner/session-logs/2026-07/2026-07-14-howard-network-cutover-prep.md

@@ -0,0 +1,176 @@
+## User
+- **User:** Howard Enos (howard)
+- **Machine:** Howard-Home
+- **Role:** tech
+
+# The Prairie Schooner — network-only cutover prep (QB shutdown, data staging, UDM Pro config)
+
+> Evening session on Howard's desktop, resumed from the laptop prep log
+> (`2026-07-14-howard-sonicwall-usg-to-udmpro-swap-prep.md`). Ends with everything
+> staged for a NETWORK-ONLY cable swap tomorrow morning.
+
+## Session Summary
+
+Resumed the Prairie Schooner UDM Pro swap on HOWARD-HOME. First completed the discovery
+left pending by the laptop session: pulled the ScreenConnect inventory (CP1 filter
+`CustomProperty1 LIKE '*Schooner*'`), matched machines to the RMM ping-sweep MACs, and
+identified the four "unknown Ubiquiti" LAN devices via TLS/HTTP probes — they are 3x
+Yealink desk phones (.141/.142/.146; 80:5E:C0 is Yealink OUI, not Ubiquiti) and 1x Dahua
+NVR (.145). With every device identified, wrote `udm-dhcp-plan.md` (LAN 192.168.1.1/24,
+pool .100–.199, 7 MAC reservations, statics .120/.125/.135 excluded).
+
+On Howard's "end quickbooks on all machines but the servers": closed QB fleet-wide via RMM.
+MEL-PC died clean; TINA and JAYMI had EDR-protected/kernel-zombie QBW.exe processes that
+survived taskkill, WMI Terminate (rc=2), owner-session kills, and even logoff. Verified
+logoff DID release the server-side SMB locks, then rebooted both machines (staff gone for
+the day) and stopped QBCFMonitorService on all three PCs. Zero open handles remained on
+\\tps-server\Quickbooks.
+
+Howard then ran the staging robocopy (elevated, on TPS-SVR pulling from the old server)
+— Quickbooks 144.2 GB and Deployment 22.4 GB into C:\Share\*, both FAILED=0. Discovered
+during prep that Q: is a GPP drive map in Default Domain Policy (Drives.xml →
+\\tps-server\quickbooks), the one artifact that would have silently broken a server move.
+
+**PIVOT (Howard's call):** decouple the migrations. Tomorrow = NETWORK SWAP ONLY —
+SonicWall/USG out, UDM Pro in; Q: and QB stay on the OLD server so a QB failure can never
+be blamed on the network. The QB/file-server move to TPS-SVR happens later as its own
+step. Rewrote `cutover-runbook.md` and `verify-cutover.sh` around this; GPP untouched;
+staged data + TPS-SVR shares held in reserve.
+
+Finished with a screen-by-screen remote walkthrough of the UniFi Network config
+(Howard NOT onsite — unifi.ui.com). Caught and corrected: DHCP range set to .125–.135
+(the servers' statics), .125 typed into the Auto Default Gateway override (would have
+handed every client the old DC as gateway), Auto DNS left checked, and DNS entered
+.135-only. Final verified config: LAN 192.168.1.1/24, DHCP .100–.199, gateway auto (.1),
+manual DHCP DNS **.125, .135** (same primary as the SonicWall hands today — zero
+observable change at swap), domain tps.local, Ping Conflict Detection on, firmware up to
+date. Fixed-IP reservations get pinned in the morning once clients appear.
+
+## Key Decisions
+
+- **Tomorrow is network-swap-only.** QB stays on \\tps-server; the server move is a
+  separate later visit with its own test. Rationale: isolate variables — if QB works
+  after the cable move, the network is proven.
+- **DHCP DNS handed = .125, .135 (old DC first)** for the swap — identical to today.
+  Flip to .135-first only after the network is proven; that 10-second control is the
+  point of owning DHCP.
+- **Reboot as the QBW-zombie fix.** After confirming logoff released server-side SMB
+  locks, reboots (users gone) were the only reliable way to clear kernel-zombie QBW.exe.
+- **Keep every device on its current IP** via superset pool + MAC reservations; statics
+  .120/.125/.135 kept out of leases (Ping Conflict Detection as backstop).
+- **Staged data/shares on TPS-SVR left in place** — harmless, and makes the later server
+  move a delta-robocopy-only refresh.
+- **Rollback = plug the SonicWall back in.** Nothing else changes tomorrow.
+
+## Problems Encountered
+
+- **QBW.exe unkillable on TINA/JAYMI**: SYSTEM taskkill, WMI Terminate (rc=2 access
+  denied), owner-context kill, CloseMainWindow all failed; process survived logoff as a
+  kernel zombie with a hung SMB handle. Resolved by reboot after verifying server locks
+  were released by the logoff.
+- **RMM quote-stripping**: `-Filter "Name = \"QBW.exe\""` was mangled by the dispatch
+  layer → false negatives + parse errors. Fixed with quote-free
+  `-Filter ("ProcessId = " + $g.Id)`. Logged `--friction`
+  (ref=feedback_windows_quote_stripping).
+- **OUI misidentification carried from prior session**: 80:5E:C0 / E4:24:6C assumed
+  Ubiquiti — TLS cert subjects proved Yealink + Dahua. Logged `--correction`.
+- **Robocopy "no permissions" as guru**: UAC filtered token — ran from an elevated
+  prompt; /COPY:DATSO fallback prepared for SACL error 1314 but not needed.
+- **UniFi misconfigs caught via screenshots**: DHCP range .125–.135; .125 in the Auto
+  Default Gateway override field; Auto DNS checked; DNS .135-only. All corrected;
+  final screenshot verified correct.
+- **ScreenConnect query empty on first run**: `$CLAUDETOOLS_ROOT` unset in fresh shell;
+  reran with explicit paths.
+
+## Configuration Changes
+
+- CREATED `clients/prairie-schooner/udm-dhcp-plan.md` — LAN/DHCP/reservation plan (edited
+  twice; final DNS order .125, .135 for the swap).
+- CREATED then REWROTE `clients/prairie-schooner/cutover-runbook.md` — network-swap-only
+  runbook; LATER section holds the deferred QB/file-server move + AD retirement.
+- CREATED then EDITED `clients/prairie-schooner/verify-cutover.sh` — RMM-dispatched
+  post-swap verification (TPS-Tina net/UNC/Q: + TPS-SERVER services). `bash -n` clean.
+- Endpoints: QB closed fleet-wide; TINA + JAYMI rebooted; QBCFMonitorService stopped on
+  TINA/JAYMI/MEL-PC (will auto-start again with QB use — not disabled).
+- TPS-SVR: C:\Share\Quickbooks (144.2 GB) + C:\Share\Deployment (22.4 GB) staged,
+  robocopy FAILED=0; Quickbooks + Deployment SMB shares exist (staging, unused).
+- UDM Pro (remote, unifi.ui.com): LAN 192.168.1.1/24, domain tps.local, DHCP .100–.199,
+  manual DHCP DNS .125 + .135, Auto Default Gateway, Ping Conflict Detection on. LAN
+  port still isolated — nothing live changed at the client site tonight.
+- `errorlog.md`: one `--correction` (OUI misID), one `--friction` (quote-stripping).
+- GPP Q: drive map **untouched** (still \\tps-server\quickbooks).
+
+## Credentials & Secrets
+
+- No new credentials this session. Howard will obtain user logon passwords for the
+  morning QB test himself.
+- Prior-session state stands: `clients/prairie-schooner/sonicwall-admin` password
+  REJECTED-2026-07-14-INVALID (SonicWall being removed; not needed);
+  `infrastructure/unifi-site-manager-api` key still 401 (needs rotation);
+  domain creds unchanged (`clients/prairie-schooner/tps-domain-admin` guru, etc.).
+
+## Infrastructure & Servers
+
+- Domain TPS.local, flat LAN 192.168.1.0/24, gw .1 (SonicWall, being removed;
+  MAC c0-ea-e4-33-cd-c8). SonicWall egress 184.176.147.183 → becomes Cox-dynamic
+  (184.191.143.52 at adoption) after swap; no inbound services.
+- TPS-SERVER (old DC/QB host/file server, 2016): .125 + .120 alias, MAC
+  6C:0B:84:A5:D7:57, RMM `4f8dff0c-b033-4f66-9101-9350e32e0a81`.
+- TPS-SVR (new DC, 2019): .135, MAC 5C:ED:8C:EB:26:A6, RMM
+  `799675cf-5c94-465a-b653-63e08d016526`.
+- Workstations: TPS-JAYMI .130 C0:25:A5:D5:2D:17 (RMM `1fbf4e45-1b48-4c82-af85-847fc1344e12`),
+  TPS-TINA .139 C0:25:A5:D5:2D:8B (RMM `c8d76770-dc9e-45b7-94c7-fdcab3e105a2`),
+  MEL-PC .149 00:BE:43:D3:72:BD user erin (RMM `eb2f2aeb-f56e-4fc6-8e5a-73257b95e5f0`).
+  (Resolve UUIDs live before dispatch.)
+- Yealink phones .141/.142/.146; Dahua NVR .145 — all outbound-only (hosted SIP /
+  Dahua P2P); public-IP change is a non-issue.
+- USG previously .136, role unconfirmed — physically trace + remove at flip.
+- Company file: Q:\schoonerQB2025.QBW (2.9 GB) on \\tps-server\Quickbooks.
+
+## Commands & Outputs
+
+- RMM auth: `eval "$(bash .claude/scripts/rmm-auth.sh 2>/dev/null)"` (never piped).
+- QBW zombie diagnostics: Stop-Process silent-fail; `taskkill /F` and WMI
+  `Terminate()` rc=2 as SYSTEM; process persisted post-logoff (kernel zombie).
+- Server-side lock check: `Get-SmbOpenFile | Where Path -like *quickbooks*` → empty
+  after logoffs (the gate that made reboots safe).
+- Robocopy (elevated on TPS-SVR): `/E /COPYALL /DCOPY:DAT /R:1 /W:1 /MT:16 /NP /TEE
+  /LOG:` — QB 144.201 GB total / 10.720 GB delta copied, FAILED 0; Deployment
+  22.386 GB, FAILED 0.
+- GPP discovery: `\\tps.local\SYSVOL\...\{31B2F340-016D-11D2-945F-00C04FB984F9}\USER\
+  Preferences\Drives\Drives.xml` → Q: = \\tps-server\quickbooks, action=U, persistent=0.
+- Quote-safe WMI kill pattern: `-Filter ("ProcessId = " + $g.Id)` (no embedded quotes).
+- Morning verification: `bash clients/prairie-schooner/verify-cutover.sh`.
+
+## Pending / Incomplete Tasks
+
+Tomorrow morning (Howard onsite):
+1. On the Pro BEFORE cabling: confirm Security/CyberSecure toggles (Ad Block, DNS
+   Shield/content filtering, IDS/IPS) are OFF — never visually confirmed tonight.
+2. Pull SonicWall (.1) + USG; patch office switch into the Pro's LAN port.
+3. Workstation `ipconfig /renew` → same IP, gw .1, DNS .125/.135, internet up.
+4. User logon (Howard has passwords): Q: maps to \\tps-server\Quickbooks, open
+   schoonerQB2025.QBW — QB works exactly as yesterday.
+5. Phones dial tone/registration (3x Yealink); NVR local UI + remote view.
+6. `bash clients/prairie-schooner/verify-cutover.sh` for the scripted pass/fail.
+7. Pin the 7 Fixed-IP reservations in Client Devices as clients appear
+   (per udm-dhcp-plan.md).
+Rollback: plug the SonicWall back in — nothing else was changed.
+
+LATER (separate visit): QB/file-server move — QB closed → delta robocopy → QB DBSM
+(match QB year) scan C:\Share\Quickbooks → GPMC Q: → \\tps-svr\Quickbooks → relogon
+multi-user test → flip Pro DNS order to .135-first. Then AD retirement: FSMO, CA
+migration (test Backup-CARoleService export first), demote TPS-SERVER + power off,
+real backup. Old server stays ON until then.
+
+## Reference Information
+
+- Runbook: `clients/prairie-schooner/cutover-runbook.md` (authoritative for tomorrow).
+- Plan: `clients/prairie-schooner/udm-dhcp-plan.md`. Verify:
+  `clients/prairie-schooner/verify-cutover.sh`.
+- Prior logs: `2026-07-14-howard-sonicwall-usg-to-udmpro-swap-prep.md` (laptop, same
+  day), `2026-07-10-howard-server-migration.md`.
+- Wiki: `wiki/clients/the-prairie-schooner.md`. Syncro cid 3664974. Contact Tina
+  Coalter, Tucson.
+- RMM `http://172.16.3.30:3001`; UniFi Site Manager unifi.ui.com (owner
+  mike@azcomputerguru.com; Pro is a standalone console).