diff --git a/.claude/memory/MEMORY.md b/.claude/memory/MEMORY.md index 4b9dce8a..478f1aad 100644 --- a/.claude/memory/MEMORY.md +++ b/.claude/memory/MEMORY.md @@ -34,6 +34,7 @@ - [Syncro prepay: full-GET only](feedback_syncro_prepay_full_get_only.md) — read prepay_hours ONLY from GET /customers/{id}; the customer search/list endpoint returns null/stale prepay. Never assert "no block" in a billing preview from search data. - [Syncro priority/type format](feedback_syncro_priority_type_format.md) — every ticket create needs a number-prefixed priority ("2 Normal", not bare "Normal" which renders blank) AND a valid problem_type. Winter flagged #32193/#32194. Use the syncro skill's create flow. - [RMM drive-map Explorer refresh](reference_rmm_drive_map_explorer_refresh.md) — drive mapped via RMM user_session works but the user's running Explorer won't show it until SHChangeNotify(DRIVEADD); also UNC \\ gets eaten in heredoc+jq, build it from [char]92. +- [Verify live before acting](feedback_verify_live_before_acting.md) — pull LIVE data (OMSA/iDRAC/live API) before acting on a hardware/infra flag; wiki/logs go stale. Cascades CS-SERVER "degraded RAID" was 9-day-stale (mirror self-recovered, SSDs bought needlessly). Windows can't see RAID member health. - [AAD Connect msDS-KeyCredentialLink writeback](reference_aadconnect_keycredlink_writeback.md) — "completed-export-errors" + 8344 INSUFF_ACCESS_RIGHTS on a protected admin account = WHfB key writeback blocked by AdminSDHolder. Diagnose with csexport /f:x; fix with dsacls WP;msDS-KeyCredentialLink on AdminSDHolder + SDProp. - [UniFi Site Manager cloud API](reference_unifi_site_manager_api.md) — `api.ui.com` + `X-API-KEY` (vault `services/unifi-site-manager`) = remote access to the WHOLE ACG UniFi fleet (~36 consoles) outside UOS. Tier1 `/v1/hosts|sites|devices|isp-metrics` = inventory+health+WAN. Tier2 CONNECTOR `/v1/connector/consoles/{id}/proxy/network/api/s/default/stat/{device,sta}` = **full UOS parity** (per-radio cu_total airtime + per-client RSSI) for ANY console, remote. Backend `unifi-wifi/scripts/gw-sitemanager.sh` (`fleet|devices|sites|isp|net`). Standalone UDM WAN SSH usually firewalled; per-console SSH pw at `clients//udm-ssh`. - [reference_sqlx_migrations_immutable](reference_sqlx_migrations_immutable.md) -- NEVER edit an already-applied sqlx migration file — even a comment. sqlx::migrate! checksums each file at compile time and validates against _sqlx_migrations at startup; a changed checksum crash-loops the server with "migration N was previously applied but has been modified". Code review MUST flag any edit to an applied migration. diff --git a/.claude/memory/feedback_verify_live_before_acting.md b/.claude/memory/feedback_verify_live_before_acting.md new file mode 100644 index 00000000..5500a1a3 --- /dev/null +++ b/.claude/memory/feedback_verify_live_before_acting.md @@ -0,0 +1,30 @@ +--- +name: feedback_verify_live_before_acting +description: Always pull LIVE current data before acting on (or alarming about) a hardware/infra finding — wiki/session-logs are point-in-time snapshots that go stale +metadata: + type: feedback +--- + +Before acting on, or raising alarm about, any hardware/infrastructure state — **pull live current +data first and lead with THAT, not the wiki or a recalled fact.** The wiki, session logs, and memory +are point-in-time snapshots; a "broken/degraded/failing" flag may have changed by the time you read it. +This matters most for **hard-to-reverse or money-spending actions** (drive swaps, hardware pulls, +parts purchases, "it's down" escalations). + +**Why:** 2026-06-24 — the Cascades CS-SERVER wiki carried a `[CRITICAL] RAID degraded / failing +drive` flag from 2026-06-15. Acting on it, **SSDs were purchased** and Howard went onsite ready to +hot-swap the "failing" drive. A **live Dell OMSA `omreport` query (via the RMM agent)** then showed +the OS mirror had **self-recovered** (the flaky drive dropped out and re-synced after a power cycle): +all 5 disks Online/Ok, all LEDs green, and the "5th unused drive" was actually the **global hot +spare**. Acting on the 9-day-stale flag nearly pulled a healthy drive and wasted a drive purchase. +Howard's directive: "always go with live current data to make sure our findings are real." + +**How to apply:** +- For Dell servers: `omreport storage controller|vdisk|pdisk controller=0` + `omreport system esmlog` + via the RMM agent (OMSA reads the controller directly — authoritative). iDRAC/Redfish is the + out-of-band equivalent (no iDRAC skill yet; creds not vaulted as of 2026-06-24). +- Windows `Get-PhysicalDisk`/`Get-Disk` shows only the VIRTUAL disks as "Healthy" even when a member + is degraded — it CANNOT see the array; never conclude RAID health from the OS view alone. +- For any infra claim sourced from the wiki/a recalled fact: re-verify the specific file/flag/host is + still true before recommending action. State the data's timestamp and source. +- See [[reference_rmm_drive_map_explorer_refresh]] for the OMSA-via-RMM pattern context. diff --git a/clients/cascades-tucson/docs/REMAINING-WORK-PLAN.md b/clients/cascades-tucson/docs/REMAINING-WORK-PLAN.md index b0b29bb0..c45a4c51 100644 --- a/clients/cascades-tucson/docs/REMAINING-WORK-PLAN.md +++ b/clients/cascades-tucson/docs/REMAINING-WORK-PLAN.md @@ -141,8 +141,8 @@ test scope to real caregivers, one device at a time. (Detail: wiki "Entra Access ## Workstream 5 — Server / infrastructure -- **Verify cloud backup** (MSP360 -> ACG-backup) first full completed + set retention. [GATE for RAID work] -- **CS-SERVER degraded OS RAID-1** -> replace with 2x 480 GB enterprise SATA SSD (gate on backup verified). Real fix = DC migration off the 16-yr-old R610. +- **Cloud backup (MSP360 -> ACG-backup): VERIFIED running 2026-06-24** (last run Success, 0 failed, 575 GB baseline in cloud, incrementals working). Still confirm it is image/bare-metal/system-state (looks file-level) + set retention. [GATE for any drive work] +- **CS-SERVER RAID -- CORRECTED 2026-06-24: HEALTHY, not degraded** (live OMSA: both mirrors Ok, all 5 disks Online, all LEDs green; the 6/15 degraded self-recovered). **NO emergency drive swap.** 1:0:4 = global hot spare (do not remove). **Planned** reliability upgrade: replace the 2 consumer 320 GB drives (esp. flaky WD 0:0:3) with the 2x enterprise SSD **already purchased**, on a scheduled window w/ confirmed image/system-state backup. **[WARN] PSU redundancy lost** -- one PSU not delivering, check onsite. Service Tag 9MQFTK1. Real fix = DC migration off the 16-yr-old R610. - Clean up old-MSP agent sprawl (Datto RMM/CentraStage + Datto EDR/Infocyte) thrashing the spindle. - Synology -> backup-only (Team Folder migration of the real shares; close the workgroup/Kerberos quirk). - Rotate the Synology signin-portal credential (was committed plaintext historically). diff --git a/errorlog.md b/errorlog.md index f9e8d6ee..7d02eb86 100644 --- a/errorlog.md +++ b/errorlog.md @@ -17,6 +17,8 @@ Categories (the `[type]` tag): _(none)_ = skill/command execution failure · +2026-06-24 | Howard-Home | rmm/cascades-cs-server | [correction] led with a 9-day-stale wiki '[CRITICAL] degraded RAID / failing drive' flag and recommended drive replacement (SSDs were purchased, tech went onsite to hot-swap); a LIVE Dell OMSA omreport query then showed the OS mirror had self-recovered and is healthy (all 5 disks Online, all LEDs green), and the '5th unused drive' was actually the global hot spare. Always pull live OMSA/iDRAC before acting on a stale hardware flag; Windows Get-PhysicalDisk cannot see RAID member health. [ctx: ref=feedback_verify_live_before_acting host=CS-SERVER tag=9MQFTK1] + 2026-06-24 | Howard-Home | process/client-deliverables | [correction] did not gate outbound client/vendor deliverables through the impeccable skill; rule: run impeccable on anything sent externally 2026-06-24 | Howard-Home | syncro/ticket-create | [correction] created #32193/#32194 with priority 'Normal' instead of Syncro's canonical number-prefixed '2 Normal'; the value did not match the priority dropdown so it displayed blank (Winter flagged it). Always set priority as 'N Name' (e.g. '2 Normal','4 Urgent') AND a valid problem_type (Onsite/Remote/etc.) on every ticket create via the syncro skill. [ctx: ref=syncro-skill priority-format] diff --git a/wiki/clients/cascades-tucson.md b/wiki/clients/cascades-tucson.md index 65de4e4f..b275cdbf 100644 --- a/wiki/clients/cascades-tucson.md +++ b/wiki/clients/cascades-tucson.md @@ -181,9 +181,24 @@ Because per-user **Intune** never provisioned tenant-wide (`INTUNE_A = PendingIn | cascadesDS (Synology NAS) | 192.168.0.120 | NAS / legacy file storage | DSM 7.2.1-69057 | Port 5000 HTTP. Workgroup name is "CASCADES" -- same as AD short name, causing Kerberos auth failures from domain-joined machines. Slated to become backup-only. **Synology Drive Server 3.5.0-26088** (active, port 6690 SSL). Current Drive sync: CS-SERVER Drive Client (v7.5.0.16085, runs as sysadmin) syncs Sync-user My Drive (`/volume1/homes/Sync/Drive/`) -> `D:\Shares\Main` (one-way download). Real shared folders (Server 1.9 G, Management 5.5 G, Public ~50 G, SalesDept ~23 G, etc.) are NOT in scope -- Team Folder migration pending. | | pfSense Firewall | 192.168.0.1 | Perimeter firewall, inter-VLAN routing, DHCP/DNS | pfSense Plus 25.07-RELEASE | Netgate device. cert CN=pfSense-685f277aa6886. Dual-WAN. All DHCP (CS-SERVER DHCP role has no scopes). 199 DHCP subnets (per-unit /28 VLANs, assisted-living L2 isolation). SSH shell access works (no interactive menu). Admin vault: `clients/cascades-tucson/pfsense-firewall`. OpenVPN user Howard: vault `clients/cascades-tucson/pfsense-openvpn-howard`. **Config vaulted 2026-06-17:** `clients/cascades-tucson/pfsense-config-backup-2026-06-17.sops.yaml`. pfSense is ZFS (power-loss resilient). Logs are PLAIN TEXT (not clog). | -**[CRITICAL] CS-SERVER hardware -- RAID degraded (2026-06-15):** Dell R610, basic SAS 6/iR controller (3 Gbps, no cache). The **OS RAID-1 mirror (Virtual Disk2 = C:, holds OS / AD / SQL / page file) is DEGRADED** -- Physical Disk 0:0:3 (320 GB WD SATA laptop drive, `WDC WD3200BEVT`) is Critical/Removed, leaving C: on a single surviving 320 GB Hitachi `HTS545032B9A300` 5400 RPM spindle with ZERO redundancy. A 1.2 TB SAS disk (1:0:4) sits "Ready" but is the wrong size/type to rebuild the 320 GB mirror. D: is a separate healthy RAID-1 (2x 1.2 TB SAS). Degraded mirror on a slow laptop spindle is root cause of "CS-SERVER slow" reports. Recommended replacement: 2x 480 GB enterprise 2.5" SATA SSD (e.g. Solidigm D3-S4520 or Samsung PM893; SATA negotiates to 3 Gbps; no Dell drive lockout). Gating: **verify cloud backup first full + image-based + retention before any drive work.** DC migration is the real fix. +**[CORRECTED 2026-06-24 -- LIVE OMSA] CS-SERVER RAID is HEALTHY, not degraded.** Dell PowerEdge R610 (Service Tag **9MQFTK1**), basic **SAS 6/iR Integrated** controller (3 Gbps, no cache), Status Ok. A live `omreport` query (Dell OMSA on CS-SERVER via RMM) shows **both virtual disks Ok/Ready and all 5 physical disks Online/Ok, Failure Predicted: No, all LEDs green.** The 2026-06-15 "degraded" state (PD 0:0:3 Critical/Removed) **self-recovered** -- the flaky consumer drive dropped out and re-synced after a power cycle (the ESM hardware log shows repeated drive remove/install events across the 6/17 + 6/23 outages). **Do NOT pull a drive -- there is nothing failed to swap.** -**[INFO] Backup -- gap closed (2026-06-15):** Mike installed ACG cloud backup (MSP360/CloudBerry -> ACG-backup server) on CS-SERVER, addressing the longstanding SS164.308(a)(7) "no backup" HIPAA gap. Verify the first full completes and set retention. + **Live physical-disk map (OMSA, 2026-06-24):** + | ID | Size/Type | Make / Serial | Role | + |---|---|---|---| + | 0:0:0 | 1.2 TB SAS | Seagate ST1200MM0088 / Z400WHK8 | VD0 (D:) mirror member, Online | + | 0:0:1 | 1.2 TB SAS | Seagate ST1200MM0088 / S400RL2N | VD0 (D:) mirror member, Online | + | 0:0:2 | 320 GB SATA | Hitachi HTS545032B9A300 / …1DR | VD2 (C:) mirror member, Online | + | 0:0:3 | 320 GB SATA | WDC WD3200BEVT / WD-WXEX08URD116 | VD2 (C:) mirror member, Online (the 6/15 flaky drive) | + | 1:0:4 | 1.2 TB SAS | Seagate ST1200MM0088 / Z400WHML | **GLOBAL HOT SPARE** (protects the 1.2 TB D: mirror; do NOT remove) | + + - **VD0 = D: (1,117 GB RAID-1)** Ok; **VD2 = C: (297.5 GB RAID-1)** Ok. Windows sees only these 2 virtual disks. + - **1:0:4 is the GLOBAL HOT SPARE** (not "unused") -- matched to the D: mirror, gives it auto-rebuild protection. Pulling it strips D:'s safety net. It cannot rebuild the 320 GB C: mirror (size mismatch), so the C: mirror has no spare. + - **[WARN] PSU redundancy lost** (ESM log "Power supply redundancy is lost") -- one of the dual PSUs isn't delivering; check cords/feeds/LEDs onsite. + - **Planned (NOT emergency) reliability upgrade:** replace the two consumer 320 GB drives (0:0:2 Hitachi + 0:0:3 WD, esp. the flaky WD) with **2x enterprise SATA SSD (already purchased)** on a scheduled window with a confirmed image/system-state backup. DC migration off the 16-yr-old R610 remains the real long-term fix. + - **LESSON:** the prior "[CRITICAL] degraded -- replace drive" flag was a 9-day-stale snapshot; acting on it (SSDs purchased) before a live check was premature. Always pull live OMSA/iDRAC before drive action. + +**[INFO] Backup -- gap closed (2026-06-15); verified running 2026-06-24.** Mike installed ACG cloud backup (MSP360/CloudBerry -> ACG-backup server) on CS-SERVER, addressing the longstanding SS164.308(a)(7) "no backup" HIPAA gap. **Live check 2026-06-24:** last run (6/24 00:10) = "Plan status: Success", 0 failed; 575.7 GB / 248k-file dataset already in the cloud (only 465 MB changed -> full baseline exists, incrementals working). **Still to confirm: this looks FILE-LEVEL, not image/bare-metal/system-state -- for a DC that is a DR gap; confirm with Mike whether a separate image/system-state backup exists before treating it as full disaster coverage.** Set/confirm retention. **[WARNING] CS-SERVER endpoint-agent sprawl:** CS-SERVER is NOT in the ACG Bitdefender/GravityZone tenant (Cascades company id `66b0448e1e0441d02508bad8`; 3 endpoints there, CS-SERVER absent). The previous MSP's **Datto RMM/CentraStage + Datto EDR/Infocyte** are still installed alongside Syncro + GuruRMM + ScreenConnect + KPAX -- overlapping agents thrashing the degraded spindle. Clean up the Datto stack. @@ -514,8 +529,8 @@ Syncro live pull 2026-06-24: **6 open tickets** -- #32194 (spare machine for new - LAPTOP-8P7HDSEI: upgrade Win 10 -> Win 11 before PHI use - Edge UNC download bug (Chromium 149): decide fix path for Ashley Jensen + Lois Lane and fleet; no fix applied as of 2026-06-08 - ALIS app session timeout: lower from 20 to 15 min (Howard, ALIS admin) -- PENDING -- **[CRITICAL] CS-SERVER degraded RAID-1 (2026-06-15):** OS mirror (C:) running on single 320 GB Hitachi 5400 RPM laptop spindle. Recommended replacement: 2x 480 GB enterprise 2.5" SATA SSD (e.g. Solidigm D3-S4520 or Samsung PM893). Gated on backup verification. -- **[INFO] CS-SERVER cloud backup (MSP360/CloudBerry, installed 2026-06-15):** verify first full completes + confirm image-based / bare-metal + system-state + retention before any drive work. +- **[CORRECTED 2026-06-24] CS-SERVER RAID is HEALTHY (live OMSA), not degraded.** The 6/15 degraded state self-recovered after a power cycle; both mirrors Ok, all 5 disks Online, all LEDs green, 1:0:4 = global hot spare. **No emergency drive swap.** Planned reliability upgrade: replace the 2 consumer 320 GB drives (esp. flaky WD 0:0:3) with the 2x enterprise SSD already purchased, on a scheduled window with a confirmed image/system-state backup. **[WARN] PSU redundancy lost** (one PSU not delivering -- check onsite). Service Tag 9MQFTK1. See Infrastructure for the full live disk map. +- **[INFO] CS-SERVER cloud backup (MSP360/CloudBerry):** **verified running 2026-06-24** -- last run Success, 0 failed, 575.7 GB baseline in cloud (incrementals working). Still confirm it's image-based/bare-metal/system-state (looks file-level) + retention. - **[CLEANUP] CS-SERVER agent sprawl:** remove the previous MSP's leftover Datto RMM (CentraStage) + Datto EDR (Infocyte) stack. --- @@ -565,6 +580,7 @@ Syncro live pull 2026-06-24: **6 open tickets** -- #32194 (spare machine for new | 2026-06-19 | **Voice VLAN migration COMPLETE (29/29 Poly) + band-selection diagnosis + Vertical 5 GHz handoff.** Howard walked the building, re-keyed all remaining Poly handsets to voice PPSK. Per-phone re-look: most phones on clean 5 GHz (Lauren .202: 2.4/50% -> 5GHz/12%), but several stuck on 2.4 despite -50 to -60 dBm signal -- controller band-steering not holding Poly OUI on 5 GHz. Phone-side fix: **5 GHz-only lock request sent to Richard Turner (Vertical)**, awaiting response = the last voice item. Kitchen server phone bad (pulled by John); Bistro phone relocated to Kitchen; Bistro now has no phone (replacement pending). Billed ticket #32444 (7h: 4 onsite + 3 remote), block 55.75->48.75. | | 2026-06-23 | **Planned power outage (05:30-09:00 MST) -- clean shutdown executed + verified.** Building electrical work; to avoid the 6/17 dirty-shutdown damage (and given CS-SERVER's degraded OS mirror), all three core devices were armed 6/22 ~19:06 to self-shut-down on local schedules (CS-SERVER task 05:28, Synology 05:28, pfSense 05:30) -- firing independent of any remote session/tunnel, UPS carrying them through the cut. Verified clean at 05:31: CS-SERVER offline via RMM cloud (last_seen 05:29:49 MST); pfSense/Synology unreachable as expected (pfSense = VPN endpoint). Pre-flight confirmed cloud backup last full SUCCESS (0 errors), iDRAC AC-recovery + Synology auto-restart backstops ON. Bring-up (~09:00, John onsite) pending. Runbook: `docs/runbooks/2026-06-23-planned-power-outage.md`. | | 2026-06-24 | **Syncro ticket review + #32193 Executive share + device-readiness audit + consolidated plan.** Reviewed/closed a batch of tickets; built restricted share `\\cs-server\Executive` for Ashley.Jensen + Meredith.Kuhn (NTFS+share scoped, E: mapped both machines RW-verified, billed 0.5h block, invoice #1650785728, block 48.75->48.25). Diagnosed two real RMM gotchas (UNC `\\` eaten in dispatch -> build from [char]92; mapped drive not shown until SHChangeNotify DRIVEADD). Fixed malformed priority on #32193/#32194 (Winter flag -> memory). Live AD+RMM domain-join diff: 12 staff PCs joined, ~17 to migrate; **5 on Windows Home blocked until Home->Pro** (Howard handling). Built `docs/REMAINING-WORK-PLAN.md` (7 workstreams). Decision: caregivers stay TEST-scoped until all devices domain-ready. | +| 2026-06-24 | **CS-SERVER RAID live-verified -- the "degraded/failing" flag was STALE; mirror is healthy.** Howard onsite ready to hot-swap a failing drive; live Dell OMSA (`omreport` via RMM) showed both virtual disks Ok, all 5 physical disks Online/Ok, Failure Predicted No, all LEDs green. The 6/15 "degraded" (PD 0:0:3 WD) self-recovered after a power cycle (ESM log shows repeated drive remove/install across the outages). The "5th unused drive" (1:0:4) is the **GLOBAL HOT SPARE** for the D: mirror -- NOT removable. Also surfaced: **PSU redundancy lost** (one PSU not delivering). Backup verified running (last run Success, 0 failed, 575 GB baseline; confirm BMR/system-state). **Outcome:** no drive pulled; the 2x enterprise SSD already purchased become a *planned* upgrade, not an emergency. Lesson logged: always pull live OMSA/iDRAC before acting on a stale hardware flag. Service Tag 9MQFTK1. | --- @@ -576,6 +592,7 @@ Syncro live pull 2026-06-24: **6 open tickets** -- #32194 (spare machine for new - Profile: hours + active-tickets lines updated; Active Work now points at the new `docs/REMAINING-WORK-PLAN.md` and carries the 2026-06-24 device-readiness audit (Home-edition blockers, ready-to-join set, caregiver-test-scoped decision). - Migration phase-status table: added 2026-06-24 domain-join reality (Home-blocked set, ready set, HEALTH-SERVICES/Lois joined). - History Highlights: added 2026-06-24 entry. Sources: added the 2026-06-24 session log + REMAINING-WORK-PLAN.md. +- **[CORRECTION 2026-06-24, live OMSA] CS-SERVER RAID is HEALTHY, not degraded.** Replaced the stale `[CRITICAL] RAID degraded (2026-06-15)` Infrastructure block + Active-Work blocking line with the live disk map: both mirrors Ok, all 5 disks Online/green, 1:0:4 = global hot spare; the 6/15 degraded self-recovered after a power cycle. Flagged PSU redundancy lost (Service Tag 9MQFTK1). Backup verified running. The 2x SSD already purchased are now a *planned* (not emergency) reliability upgrade. Lesson saved to memory `feedback_verify_live_before_acting`. **2026-06-23 recompile (HOWARD-HOME/claude-main) changes vs. prior (2026-06-20, GURU-5070):** - Surgical/additive full recompile -- the prior compile was current; the only new knowledge was the 2026-06-23 planned power outage. All other sections preserved verbatim. diff --git a/wiki/index.md b/wiki/index.md index 003d4ba5..0b735d79 100644 --- a/wiki/index.md +++ b/wiki/index.md @@ -18,7 +18,7 @@ Run `/wiki-lint` to check for stale entries and broken backlinks. | Article | Summary | Last Compiled | |---|---|---| -| [Cascades of Tucson](clients/cascades-tucson.md) | Prepaid block $175/hr, **48.25 hrs remaining** (live 2026-06-24); senior living; active domain migration + HIPAA caregiver-lockdown project (GPOs deployed; Entra Hybrid Join + CA allow-list + ALIS SSO model proven); single DC (CS-SERVER) on aging R610, OS RAID-1 degraded 2026-06-15 (data-loss risk; cloud backup started); **Planned power outage 2026-06-23** clean self-shutdown executed + verified (bring-up ~09:00, John onsite); **Voice VLAN 30 migration COMPLETE 2026-06-19** (~38 devices: 29 Poly + 8 AudioCodes + desktop; awaiting Vertical to set Poly 5GHz-only); **UniFi RF optimized 2026-06-19** (77 U7-Pro APs/~587 clients: 2.4GHz power->Medium on 47 radios + 5GHz clean-DFS 40MHz channel plan -> 5GHz retry halved; 6GHz blocked by WPA3 on PPSK SSID); Syncro 6 open tickets, device-readiness audit done (5 PCs on Win Home need Home->Pro before join); remaining-work plan: docs/REMAINING-WORK-PLAN.md | 2026-06-24 | +| [Cascades of Tucson](clients/cascades-tucson.md) | Prepaid block $175/hr, **48.25 hrs remaining** (live 2026-06-24); senior living; active domain migration + HIPAA caregiver-lockdown project (GPOs deployed; Entra Hybrid Join + CA allow-list + ALIS SSO model proven); single DC (CS-SERVER) on aging R610 -- RAID **live-verified HEALTHY 2026-06-24** (the 6/15 "degraded" self-recovered; both mirrors Ok, 1:0:4 = global hot spare; consumer 320GB drives + lost-PSU-redundancy are planned follow-ups, NOT an emergency); cloud backup verified running; **Planned power outage 2026-06-23** clean self-shutdown executed + verified (bring-up ~09:00, John onsite); **Voice VLAN 30 migration COMPLETE 2026-06-19** (~38 devices: 29 Poly + 8 AudioCodes + desktop; awaiting Vertical to set Poly 5GHz-only); **UniFi RF optimized 2026-06-19** (77 U7-Pro APs/~587 clients: 2.4GHz power->Medium on 47 radios + 5GHz clean-DFS 40MHz channel plan -> 5GHz retry halved; 6GHz blocked by WPA3 on PPSK SSID); Syncro 6 open tickets, device-readiness audit done (5 PCs on Win Home need Home->Pro before join); remaining-work plan: docs/REMAINING-WORK-PLAN.md | 2026-06-24 | | [Dataforth Corporation](clients/dataforth.md) | Prepaid block ~$2,099/mo, **31.5 hrs remaining** (live 2026-06-23); signal-conditioning manufacturer; 64 DOS test stations; 2025 ransomware recovery + incomplete file restore (migration-gap audit); 2026-03 phishing + MFA rollout; test-datasheet pipeline (DSCA cert publish via Hoffman API + testdatadb UI on AD2); mail stack INKY->Mailprotector CloudFilter->EXO; FreePBX 17 outage fixed 2026-06-08/09 (qualify_frequency=0; no RTP-forward); shares-ACL project (all open to staff; Phase 2 target-state strawman drafted 2026-06-22); Syncro asset reconciliation 2026-06-02; GuruRMM fleet ~45; Bitdefender phase-off | 2026-06-23 | | [Instrumental Music Center](clients/instrumental-music-center.md) | Prepaid block $175/hr, 12.5 hrs remaining; music retail/repair; AIMsi POS on SQL Server 2019; phantom DC causing slow logons; GuruRMM enrolled (IMC1) | 2026-05-24 | | [Jimmy Company](clients/jimmy.md) | Break-fix, $150/hr; single aging workstation BLASTER2 (Win10 22H2 EOL, i5-3470/3.8GB — replace); backups the recurring theme (QuickBooks data); onboarded to GuruRMM 2026-06-19 (RDP NLA + Kaseya removal + cleanup); MSP360 local backup drive full, 90-day retention set, space reclaim pending in console (cloud B2 healthy) | 2026-06-19 |