wiki: compile cascades-tucson (full) — WiFi RF + network/pfSense + SSH backend, 55.75h
This commit is contained in:
@@ -60,6 +60,7 @@ sources:
|
||||
- .claude/memory/feedback_cascades_user_security_group.md
|
||||
- .claude/memory/project-cascades-migration-plan.md
|
||||
- .claude/memory/feedback_cascades_folder_redirect.md
|
||||
- .claude/memory/howard-home-lan-shadow.md
|
||||
backlinks:
|
||||
- projects/gururmm
|
||||
- wiki/systems/uos-server
|
||||
@@ -75,37 +76,37 @@ Senior living / assisted living facility in Tucson, AZ. Single 6-floor building
|
||||
|
||||
**In one line:** a HIPAA-driven, identity-based access-control system that splits staff into two security postures and enforces them with **Microsoft Entra Conditional Access** on top of **hybrid identity** (Entra Connect), with **ALIS (clinical EHR) wired for SSO**. Tickets: #109412123 (Entra setup), #110680053 (domain migration).
|
||||
|
||||
### Foundation — hybrid identity
|
||||
### Foundation -- hybrid identity
|
||||
- On-prem AD `cascades.local` synced to Entra/M365 via **Entra Connect** (PHS + Seamless SSO). UPN suffix `cascadestucson.com`, so a user's **Windows login = email = M365/ALIS identity** (one credential everywhere).
|
||||
|
||||
### Two user buckets (the core design)
|
||||
1. **Restricted — caregivers + medtechs** (group `SG-Caregivers`, `8b8d9222`): sign in **only on the Cascades network** and **only on approved devices** (shared Galaxy phones + a set of caregiver laptops/desktops). **No MFA** (no personal devices) — protected by **location + device** controls + 8h sign-in frequency instead. Effect: caregiver credentials are **useless off-site or off an approved device** — the anti-hacker / bad-employee-from-home control.
|
||||
2. **Privileged — admins / directors / managers / nurses** (NOT in `SG-Caregivers`): email + ALIS **from anywhere**, **seamless onsite / 2FA offsite** (Authenticator/PIN). Untouched by the caregiver lockdown.
|
||||
1. **Restricted -- caregivers + medtechs** (group `SG-Caregivers`, `8b8d9222`): sign in **only on the Cascades network** and **only on approved devices** (shared Galaxy phones + a set of caregiver laptops/desktops). **No MFA** (no personal devices) -- protected by **location + device** controls + 8h sign-in frequency instead. Effect: caregiver credentials are **useless off-site or off an approved device** -- the anti-hacker / bad-employee-from-home control.
|
||||
2. **Privileged -- admins / directors / managers / nurses** (NOT in `SG-Caregivers`): email + ALIS **from anywhere**, **seamless onsite / 2FA offsite** (Authenticator/PIN). Untouched by the caregiver lockdown.
|
||||
|
||||
### Conditional Access enforcement (caregivers)
|
||||
- `CSC - Block caregivers off Cascades network` (`e35614e1`)
|
||||
- `CSC - Block caregivers on non-compliant device` (`ede985e2`) — being replaced by a **device allow-list** (`CSC - Caregivers: allow-listed devices only`, `1b7fd025`): phones (`displayName -startsWith "CSC-"`) + tagged caregiver machines (`extensionAttribute1 -eq "CSCCaregiverDevice"`, or explicit deviceId). Note: extensionAttribute changes lag >70 min into CA's filter cache — **deviceId matching is the lag-free lever** for the small device set.
|
||||
- `CSC - Block caregivers on non-compliant device` (`ede985e2`) -- being replaced by a **device allow-list** (`CSC - Caregivers: allow-listed devices only`, `1b7fd025`): phones (`displayName -startsWith "CSC-"`) + tagged caregiver machines (`extensionAttribute1 -eq "CSCCaregiverDevice"`, or explicit deviceId). Note: extensionAttribute changes lag >70 min into CA's filter cache -- **deviceId matching is the lag-free lever** for the small device set.
|
||||
- `CSC - Caregiver sign-in frequency 8h` (`7d491c7a`)
|
||||
- Rollout is **per-user via group membership** (test group `SG-Caregivers-DeviceTest` `db5849ec` carries the full rule set for one-at-a-time validation; promote to `SG-Caregivers` + disable compliance-block when validated).
|
||||
|
||||
### Devices
|
||||
- **Phones:** Samsung A15s in Intune **Shared Device Mode** (Android Enterprise, device-token enrolled) — live.
|
||||
- **Phones:** Samsung A15s in Intune **Shared Device Mode** (Android Enterprise, device-token enrolled) -- live.
|
||||
- **Laptops/desktops:** caregiver shared machines (Laptop2, LAPTOP-DRQ5L558, LAPTOP-E0STJJE8, ASSISTNURSE-PC, NURSESTATION-PC) joined to Entra so CA recognizes them and they go on the allow-list (group `Cascades - Caregiver Devices` `02c6f698` for policy targeting).
|
||||
|
||||
### ALIS SSO
|
||||
- Entra app registration -> OIDC SSO into ALIS; **tenant-wide admin consent granted** (2026-06-03). Per-user join key = **ALIS staff Email must equal the Entra UPN**. Caregivers SSO silently on phones (ALIS-native 2FA off); office users SSO with offsite MFA.
|
||||
|
||||
### Caregiver desktop/laptop management — Hybrid Entra Join + GPO (the chosen path)
|
||||
Because per-user **Intune** never provisioned tenant-wide (`INTUNE_A = PendingInput`; no Windows device ever Intune-enrolled — MS case open), Windows caregiver devices are managed via **Hybrid Entra Join + on-prem Group Policy** instead. This needs no Intune. The CA access model is unchanged (hybrid join just gives the device an Entra object so the allow-list/deviceId still applies).
|
||||
- **Hybrid join proven on NURSESTATION-PC** (2026-06-05): SCP written (`ConfigureSCP.ps1`), `OU=Caregiver Devices,OU=Staff PCs,OU=Workstations` added to Entra Connect sync scope → device synced to Entra as `trustType: ServerAd`, `dsregcmd` shows AzureAdJoined+DomainJoined YES, pilot.test gets `AzureAdPrt: YES`. On hybrid-joined machines `Ngc PreReqResult: WillNotProvision` (PolicyEnabled NO) → **Windows Hello does not auto-provision** (no Hello popup) — exactly what shared caregiver devices need, so no separate Hello-disable step.
|
||||
### Caregiver desktop/laptop management -- Hybrid Entra Join + GPO (the chosen path)
|
||||
Because per-user **Intune** never provisioned tenant-wide (`INTUNE_A = PendingInput`; no Windows device ever Intune-enrolled -- MS case open), Windows caregiver devices are managed via **Hybrid Entra Join + on-prem Group Policy** instead. This needs no Intune. The CA access model is unchanged (hybrid join just gives the device an Entra object so the allow-list/deviceId still applies).
|
||||
- **Hybrid join proven on NURSESTATION-PC** (2026-06-05): SCP written (`ConfigureSCP.ps1`), `OU=Caregiver Devices,OU=Staff PCs,OU=Workstations` added to Entra Connect sync scope -> device synced to Entra as `trustType: ServerAd`, `dsregcmd` shows AzureAdJoined+DomainJoined YES, pilot.test gets `AzureAdPrt: YES`. On hybrid-joined machines `Ngc PreReqResult: WillNotProvision` (PolicyEnabled NO) -> **Windows Hello does not auto-provision** (no Hello popup) -- exactly what shared caregiver devices need, so no separate Hello-disable step.
|
||||
- **Device control is one-at-a-time:** caregiver machine computer objects are moved into `OU=Caregiver Devices` (only that OU is in sync scope) and into a location group `SG-PC-MainTower` or `SG-PC-MemoryCare`. Add a device = move it into the OU + correct location group.
|
||||
- **App + printer delivery GPO `CSC - Caregiver Workstation`** (`{3B5CD9A6-A278-4676-A9FD-9396D21A8261}`, User-config GPP) — **BUILT + VALIDATED on NURSESTATION as pilot.test (2026-06-05).** Linked at `OU=Caregivers,OU=Departments`; security filter = `SG-Caregivers-Test` (Apply, pilot.test only) + Authenticated Users (Read, for MS16-072). Go-live = swap filter to `SG-Caregivers`. Contents: 3 desktop shortcuts — ALIS, LinkRx, **Helpany** (`https://app.safe-living.com/login` — named "Helpany," the brand caregivers know) — + 6 `\\CS-SERVER` shared printers (NursesPrinter, HealthServices, MCMedTech, MCReception, MCDirector, CopyRoom) with **default printer by device location** (Nurses for `SG-PC-MainTower`, MC MedTech for `SG-PC-MemoryCare`, computer-context ILT) + HKCU `LegacyDefaultPrinterMode=1` so the default sticks. Build scripts: `clients/cascades-tucson/scripts/build-caregiver-gpo.ps1` + `link-caregiver-gpo.ps1`. NOTE: the domain-wide `CSC - Printer Deployment` GPO is intentionally disabled (empty CSE / version 0) and is **not** to be used — reference only.
|
||||
- **Device lockdown GPO `CSC - Caregiver Device Lockdown`** (`{E6174988-2721-4D96-ADF5-F5BB44E92769}`, computer-only, linked to `OU=Caregiver Devices`) — **DEPLOYED 2026-06-05.** Auto-logoff is a HIPAA requirement (§164.312(a)(2)(iii)) for shared PHI devices. Settings (Howard): screen **lock at 3 min**, **auto sign-out at 15 min** total idle, **90-second warning** before sign-out, **never sleep** (display off 10 min). Delivered via a computer **startup script** (`caregiver-lockdown.ps1`, in SYSVOL) that sets `InactivityTimeoutSecs=180`, powercfg, and registers a logon-triggered scheduled task running an idle monitor (`GetLastInputInfo` → `msg.exe` warning at 13.5 min → `shutdown /l` at 15 min) in each caregiver's session. Deploy script: `deploy-device-lockdown-gpo.ps1`. **Startup scripts run at boot — NURSESTATION must reboot** to activate lock@3min / 90s warning / sign-out@15min / never-sleep (not yet verified). **Companion:** ALIS app session timeout 20→15 min (Howard, ALIS admin) **PENDING.** Lock/logoff are **device-level** (affect any user on the device in `OU=Caregiver Devices`).
|
||||
- **App + printer delivery GPO `CSC - Caregiver Workstation`** (`{3B5CD9A6-A278-4676-A9FD-9396D21A8261}`, User-config GPP) -- **BUILT + VALIDATED on NURSESTATION as pilot.test (2026-06-05).** Linked at `OU=Caregivers,OU=Departments`; security filter = `SG-Caregivers-Test` (Apply, pilot.test only) + Authenticated Users (Read, for MS16-072). Go-live = swap filter to `SG-Caregivers`. Contents: 3 desktop shortcuts -- ALIS, LinkRx, **Helpany** (`https://app.safe-living.com/login` -- named "Helpany," the brand caregivers know) -- + 6 `\\CS-SERVER` shared printers (NursesPrinter, HealthServices, MCMedTech, MCReception, MCDirector, CopyRoom) with **default printer by device location** (Nurses for `SG-PC-MainTower`, MC MedTech for `SG-PC-MemoryCare`, computer-context ILT) + HKCU `LegacyDefaultPrinterMode=1` so the default sticks. Build scripts: `clients/cascades-tucson/scripts/build-caregiver-gpo.ps1` + `link-caregiver-gpo.ps1`. NOTE: the domain-wide `CSC - Printer Deployment` GPO is intentionally disabled (empty CSE / version 0) and is **not** to be used -- reference only.
|
||||
- **Device lockdown GPO `CSC - Caregiver Device Lockdown`** (`{E6174988-2721-4D96-ADF5-F5BB44E92769}`, computer-only, linked to `OU=Caregiver Devices`) -- **DEPLOYED 2026-06-05.** Auto-logoff is a HIPAA requirement (SS164.312(a)(2)(iii)) for shared PHI devices. Settings: screen **lock at 3 min**, **auto sign-out at 15 min** total idle, **90-second warning** before sign-out, **never sleep** (display off 10 min). Delivered via a computer **startup script** (`caregiver-lockdown.ps1`, in SYSVOL) that sets `InactivityTimeoutSecs=180`, powercfg, and registers a logon-triggered scheduled task running an idle monitor in each caregiver's session. Deploy script: `deploy-device-lockdown-gpo.ps1`. **Startup scripts run at boot -- NURSESTATION must reboot** to activate (not yet verified). **Companion:** ALIS app session timeout 20->15 min (Howard, ALIS admin) **PENDING.** Lock/logoff are **device-level** (affect any user on the device in `OU=Caregiver Devices`).
|
||||
|
||||
### Status (as of 2026-06-05)
|
||||
- **Proven working end-to-end on a hybrid-joined desktop (NURSESTATION + pilot.test):** caregiver lockdown (CA off-network block + device allow-list) **and** silent ALIS SSO. The allow-list policy `1b7fd025` carries NURSESTATION's current deviceId `d3bf931f-f128-4261-8398-b46c34a4b342` (the old Entra-joined id `e16c4af5` is stale/deleted) and the device is tagged `extensionAttribute1=CSCCaregiverDevice`.
|
||||
- **GPOs DEPLOYED:** `CSC - Caregiver Workstation` (shortcuts + printers + LegacyDefaultPrinterMode, `{3B5CD9A6-A278-4676-A9FD-9396D21A8261}`) **built and validated on pilot.test.** `CSC - Caregiver Device Lockdown` (`{E6174988-2721-4D96-ADF5-F5BB44E92769}`) **deployed to `OU=Caregiver Devices` 2026-06-05** — takes effect on next NURSESTATION reboot (verify lock@3min, 90s warning, sign-out@15min). **Monday go-live:** swap GPO filter `SG-Caregivers-Test` → `SG-Caregivers`; CA allow-list test group → `SG-Caregivers`; move real caregiver machines into `OU=Caregiver Devices` + correct `SG-PC-*` location group one at a time; ALIS email-match the 38 caregivers + medtechs. **Still pending:** lower ALIS app timeout 20→15 min (Howard, ALIS admin); reboot NURSESTATION to verify lockdown.
|
||||
- **Independent open item:** Microsoft case for `INTUNE_A PendingInput` — does NOT block caregiver access (hybrid+GPO path replaces the Intune dependency).
|
||||
- **Proven working end-to-end on a hybrid-joined desktop (NURSESTATION + pilot.test):** caregiver lockdown (CA off-network block + device allow-list) **and** silent ALIS SSO. The allow-list policy `1b7fd025` carries NURSESTATION's current deviceId `d3bf931f-f128-4261-8398-b46c34a4b342` and the device is tagged `extensionAttribute1=CSCCaregiverDevice`.
|
||||
- **GPOs DEPLOYED:** `CSC - Caregiver Workstation` built and validated on pilot.test. `CSC - Caregiver Device Lockdown` deployed to `OU=Caregiver Devices` 2026-06-05 -- takes effect on next NURSESTATION reboot (verify lock@3min, 90s warning, sign-out@15min). **Monday go-live:** swap GPO filter `SG-Caregivers-Test` -> `SG-Caregivers`; CA allow-list test group -> `SG-Caregivers`; move real caregiver machines into `OU=Caregiver Devices` + correct `SG-PC-*` location group one at a time; ALIS email-match the 38 caregivers + medtechs. **Still pending:** lower ALIS app timeout 20->15 min; reboot NURSESTATION to verify lockdown.
|
||||
- **Independent open item:** Microsoft case for `INTUNE_A PendingInput` -- does NOT block caregiver access (hybrid+GPO path replaces the Intune dependency).
|
||||
|
||||
---
|
||||
|
||||
@@ -113,31 +114,27 @@ Because per-user **Intune** never provisioned tenant-wide (`INTUNE_A = PendingIn
|
||||
|
||||
- **Contract type:** Prepaid hour block
|
||||
- **Key contacts:**
|
||||
- Meredith Kuhn — Assistant Manager (ASSISTMAN-PC); internal billing contact. **NEVER set her as ticket contact in Syncro** — she is the wrong default that keeps being selected.
|
||||
- John Trozzi — Maintenance staff, Mac at 201cascades@gmail.com (shared facility account)
|
||||
- Lauren Hasselman — Accounting
|
||||
- Zachary Nelson — Accounting Assistant
|
||||
- Lois Lane — CareTakers department head (DESKTOP-KQSL232); resistant to domain migration; John Trozzi is liaison
|
||||
- Crystal Rodriguez — staff
|
||||
- Sharon Edwards — Life Enrichment Assistant (DESKTOP-DLTAGOI)
|
||||
- Ashley Jensen — Accountant (DESKTOP-U2DHAP0)
|
||||
- Shelby Trozzi — MemCare Director (MDIRECTOR-PC)
|
||||
- Chris Knight — Accounting / Business Office (same access tier as Lauren Hasselman); chris.knight@cascadestucson.com (alias: c.knight@cascadestucson.com); bill.com and BOK Financial recipient (issue investigated 2026-06-04). **Workstation setup 2026-06-08:** machine **DESKTOP-N5G1ROO** (Win 11 Pro for Workstations) domain-joined + GuruRMM-enrolled (agent `205025ee-2676-4498-8a27-e88562a6f69a`, site CascadesTucson), Office (O365) installed. AD account `chris.knight` (OU=Administrative) finished to match Lauren: home folder created, added to `SG-FolderRedirect`, `mail` set, AD password `Cascades2026!` (change-at-logon cleared). Mailbox remains cloud-only/unsynced (same split state as Lauren — see Entra sync note).
|
||||
- JD Martin — Syncro-confirmed contact (jd.martin@cascadestucson.com); role not yet documented.
|
||||
- Meredith Kuhn -- Assistant Manager (ASSISTMAN-PC); internal billing contact. **NEVER set her as ticket contact in Syncro** -- she is the wrong default that keeps being selected.
|
||||
- John Trozzi -- Maintenance staff, Mac at 201cascades@gmail.com (shared facility account)
|
||||
- Lauren Hasselman -- Accounting
|
||||
- Zachary Nelson -- Accounting Assistant
|
||||
- Lois Lane -- CareTakers department head (DESKTOP-KQSL232); resistant to domain migration; John Trozzi is liaison
|
||||
- Crystal Rodriguez -- staff
|
||||
- Sharon Edwards -- Life Enrichment Assistant (DESKTOP-DLTAGOI)
|
||||
- Ashley Jensen -- Accountant (DESKTOP-U2DHAP0)
|
||||
- Shelby Trozzi -- MemCare Director (MDIRECTOR-PC)
|
||||
- Chris Knight -- Accounting / Business Office (same access tier as Lauren Hasselman); chris.knight@cascadestucson.com (alias: c.knight@cascadestucson.com). **Workstation setup 2026-06-08:** machine **DESKTOP-N5G1ROO** (Win 11 Pro for Workstations) domain-joined + GuruRMM-enrolled (agent `205025ee-2676-4498-8a27-e88562a6f69a`), Office installed. AD account `chris.knight` (OU=Administrative) finished to match Lauren. Mailbox remains cloud-only/unsynced (same split state as Lauren).
|
||||
- JD Martin -- Syncro-confirmed contact (jd.martin@cascadestucson.com); role not yet documented.
|
||||
- **Syncro contact emails (authoritative):** ashley.jensen@, jd.martin@, crystal.rodriguez@, John.trozzi@, meredith.kuhn@, accounting@/accountingassistant@cascadestucson.com.
|
||||
- **Billing rate:** $175/hr all labor (prepaid block customer)
|
||||
- **Hours remaining:** **55.75 hrs (live Syncro pull 2026-06-15).** Most recent draws: 1.0h onsite for ASSISTNURSE-PC Win11 reinstall on #32303 (implied by balance chain 57.75→56.75; no dedicated session log captured); 0.5h remote 2026-06-10 Meredith locked Word doc (ticket #32403, invoice $0.00 prepaid, 56.75→56.25); 0.5h remote 2026-06-12 shared mailboxes Grievances+Surveys (ticket #32417, invoice $0.00 prepaid, 56.25→55.75). Always live-check via `GET /customers/20149445` before billing — balance is unreliable across sessions.
|
||||
- **Hours remaining:** **55.75 hrs (live Syncro pull 2026-06-16).** Most recent draws: 0.5h remote 2026-06-10 Meredith locked Word doc (ticket #32403, 56.75->56.25); 0.5h remote 2026-06-12 shared mailboxes Grievances+Surveys (ticket #32417, 56.25->55.75). Always live-check via `GET /customers/20149445` before billing.
|
||||
- **Syncro customer ID:** 20149445
|
||||
- **Managed devices (Syncro):** 29 (live pull 2026-06-15)
|
||||
- **Active tickets:** Syncro live pull 2026-06-15 shows **0 open tickets**. #32370 (eFax/scanner onsite) was confirmed [New]/open on 2026-06-13 — verify/likely closed; Syncro shows 0 open as of 2026-06-15. #32414 [New] was an automated "payment on the way" notification stub, not work.
|
||||
- #110680053 / #32303 — Entra / domain migration project ("Domain setup-entra sync"). Status: **Invoiced** as of 2026-06-05. Latest billing: 7.0h onsite 2026-06-05, invoice #67782 ($0.00 prepaid). Monday caregiver cutover will generate further work on this ticket. Plan: `C:\Users\Howard\.claude\plans\wise-discovering-panda.md`
|
||||
- #109412123 — Entra setup project (may be invoiced as of 2026-05-18; verify status)
|
||||
- #109035475 — John Trozzi desktop WiFi upgrade (billed)
|
||||
- #32370 — **verify/likely closed (Syncro live 2026-06-15 shows 0 open; was confirmed [New]/open 2026-06-13)** — eFax setup on Karen's and Christin's machines + portable scanner setup on both. No appointment scheduled as of 2026-06-02.
|
||||
- #32381 — Tamra scanner onsite (0.5h onsite, billed 2026-06-04, prepaid block)
|
||||
- #32382 — Megan file access onsite (1.5h onsite, billed 2026-06-04, prepaid block)
|
||||
- #32383 — **Resolved (confirmed live 2026-06-13)** — Chris Knight bill.com / BOK email delivery (1.5h remote, billed 2026-06-04, prepaid block; Syncro id 112201209). Fix was sender-side (bill.com support call + SendGrid suppression clear; BOK portal correction); ticket since closed.
|
||||
- #32403 — Meredith locked Word doc / stale owner files (0.5h remote, billed 2026-06-10, prepaid block; Invoiced)
|
||||
- #32417 — Shared mailboxes Grievances+Surveys (0.5h remote, billed 2026-06-12, prepaid block; Invoiced)
|
||||
- **Managed devices (Syncro):** 29 (live pull 2026-06-16)
|
||||
- **Active tickets:** Syncro live pull 2026-06-16 shows **0 open tickets.** See session logs for recent work. #32370 (eFax/scanner onsite) was confirmed [New]/open on 2026-06-13 -- verify/likely closed.
|
||||
- #110680053 / #32303 -- Entra / domain migration project. Status: **Invoiced** as of 2026-06-05. Plan: `C:\Users\Howard\.claude\plans\wise-discovering-panda.md`
|
||||
- #109412123 -- Entra setup project (verify status)
|
||||
- #32403 -- Meredith locked Word doc (0.5h remote, billed 2026-06-10, Invoiced)
|
||||
- #32417 -- Shared mailboxes Grievances+Surveys (0.5h remote, billed 2026-06-12, Invoiced)
|
||||
|
||||
---
|
||||
|
||||
@@ -147,65 +144,62 @@ Because per-user **Intune** never provisioned tenant-wide (`INTUNE_A = PendingIn
|
||||
|
||||
| Host | IP | Role | OS | Notes |
|
||||
|---|---|---|---|---|
|
||||
| CS-SERVER | 192.168.2.254 | DC, DNS, DHCP (no scopes), File Server, Hyper-V host, Print Server | Windows Server 2019 Standard | Dell PowerEdge R610 (~2009 hardware, 16+ years old). **Single DC — CRITICAL risk. No backup.** GuruRMM agent ID: `c39f1de7-d5b6-45ae-b132-e06977ab1713` (re-enrolled; the older `6766e973-...` is stale — **always resolve the agent live by hostname**, never hardcode the UUID). **OS RAID-1 mirror DEGRADED (2026-06-15) — see hardware warning below.** |
|
||||
| CS-SERVER iDRAC | 192.168.2.65 | Out-of-band management | — | Dell OOB interface |
|
||||
| CS-QB (Hyper-V VM on CS-SERVER) | 192.168.2.228 | (label "VoIP server" — STALE) | — | **2026-06-16 recon: SMB/445 only, no SIP response — NOT a live SIP PBX.** Phones appear cloud-registered (Vertical). Label predates the wireless-phone transition; revisit/retire. |
|
||||
| cascadesDS (Synology NAS) | 192.168.0.120 | NAS / legacy file storage | DSM | Port 5000 HTTP. Workgroup name is "CASCADES" — same as AD short name, causing Kerberos auth failures from domain-joined machines. Slated to become backup-only. |
|
||||
| pfSense Firewall | 192.168.0.1 | Perimeter firewall, inter-VLAN routing | pfSense 24.0 | Dual-WAN. All DHCP served here (CS-SERVER DHCP role has no scopes). MAC: 00:f1:f5:34:b3:4a |
|
||||
| CS-SERVER | 192.168.2.254 | DC, DNS, DHCP (no scopes), File Server, Hyper-V host, Print Server | Windows Server 2019 Standard | Dell PowerEdge R610 (~2009 hardware, 16+ years old). **Single DC -- CRITICAL risk. No backup until 2026-06-15.** GuruRMM agent ID: `c39f1de7-d5b6-45ae-b132-e06977ab1713` (re-enrolled; always resolve the agent live by hostname, never hardcode the UUID). **OS RAID-1 mirror DEGRADED (2026-06-15) -- see hardware warning below.** |
|
||||
| CS-SERVER iDRAC | 192.168.2.65 | Out-of-band management | -- | Dell OOB interface |
|
||||
| CS-QB (Hyper-V VM on CS-SERVER) | 192.168.2.228 | (label "VoIP server" -- STALE) | -- | **2026-06-16 recon: SMB/445 only, no SIP response -- NOT a live SIP PBX.** Phones appear cloud-registered (Vertical). Label predates the wireless-phone transition; revisit/retire. |
|
||||
| cascadesDS (Synology NAS) | 192.168.0.120 | NAS / legacy file storage | DSM | Port 5000 HTTP. Workgroup name is "CASCADES" -- same as AD short name, causing Kerberos auth failures from domain-joined machines. Slated to become backup-only. |
|
||||
| pfSense Firewall | 192.168.0.1 | Perimeter firewall, inter-VLAN routing, DHCP/DNS | pfSense Plus 25.07-RELEASE | Netgate device. cert CN=pfSense-685f277aa6886. Dual-WAN. All DHCP (CS-SERVER DHCP role has no scopes). 199 DHCP subnets (per-unit /28 VLANs, assisted-living L2 isolation). SSH shell access works (no interactive menu). Admin vault: `clients/cascades-tucson/pfsense-firewall`. OpenVPN user Howard: vault `clients/cascades-tucson/pfsense-openvpn-howard`. |
|
||||
|
||||
**[CRITICAL] CS-SERVER hardware — RAID degraded (2026-06-15):** Dell R610, basic SAS 6/iR controller (3 Gbps, no cache). The **OS RAID-1 mirror (Virtual Disk2 = C:, holds OS / AD / SQL / page file) is DEGRADED** — Physical Disk 0:0:3 (320 GB WD SATA laptop drive) is Critical/Removed, leaving C: on a single surviving 320 GB Hitachi 5400 RPM spindle with ZERO redundancy. A 1.2 TB SAS disk (1:0:4) sits "Ready" but is the wrong size/type to rebuild the 320 GB mirror, so no auto-rebuild fired. D: is a separate healthy RAID-1 (2x 1.2 TB SAS). The degraded mirror on a slow laptop spindle is the root cause of the "CS-SERVER slow" reports (random-I/O bound). With the single-DC, EOL (16+ yr) posture this is a data-loss emergency — SSD rebuild-then-swap is a valid band-aid (image C: first; enterprise SATA SSD >= 320 GB; no TRIM through this controller) but the DC migration remains the real fix.
|
||||
**[CRITICAL] CS-SERVER hardware -- RAID degraded (2026-06-15):** Dell R610, basic SAS 6/iR controller (3 Gbps, no cache). The **OS RAID-1 mirror (Virtual Disk2 = C:, holds OS / AD / SQL / page file) is DEGRADED** -- Physical Disk 0:0:3 (320 GB WD SATA laptop drive) is Critical/Removed, leaving C: on a single surviving 320 GB Hitachi 5400 RPM spindle with ZERO redundancy. A 1.2 TB SAS disk (1:0:4) sits "Ready" but is the wrong size/type to rebuild the 320 GB mirror, so no auto-rebuild fired. D: is a separate healthy RAID-1 (2x 1.2 TB SAS). The degraded mirror on a slow laptop spindle is the root cause of "CS-SERVER slow" reports (random-I/O bound). With the single-DC, EOL (16+ yr) posture this is a data-loss emergency -- SSD rebuild-then-swap is a valid band-aid (image C: first; enterprise SATA SSD >= 320 GB; no TRIM through this controller) but the DC migration remains the real fix.
|
||||
|
||||
**[INFO] Backup — gap now being closed (2026-06-15):** Mike installed ACG cloud backup (MSP360/CloudBerry -> ACG-backup server) on CS-SERVER and started a backup, addressing the longstanding §164.308(a)(7) "no backup" HIPAA gap. (Synology Active Backup for Business remains blocked — ext4, not Btrfs.) Verify the first full completes and set retention.
|
||||
**[INFO] Backup -- gap closed (2026-06-15):** Mike installed ACG cloud backup (MSP360/CloudBerry -> ACG-backup server) on CS-SERVER and started a backup, addressing the longstanding SS164.308(a)(7) "no backup" HIPAA gap. (Synology Active Backup for Business remains blocked -- ext4, not Btrfs.) Verify the first full completes and set retention.
|
||||
|
||||
**[WARNING] CS-SERVER endpoint-agent sprawl:** CS-SERVER is NOT in the ACG Bitdefender/GravityZone tenant; Defender is replaced by a Syncro-managed "Endpoint Protection Service". The previous MSP's **Datto RMM/CentraStage + Datto EDR/Infocyte** are still installed on top of Syncro + GuruRMM + ScreenConnect + KPAX — overlapping agents thrashing the degraded spindle. Clean up the Datto stack. (Infection sweep 2026-06-15: clean.)
|
||||
**[WARNING] CS-SERVER endpoint-agent sprawl:** CS-SERVER is NOT in the ACG Bitdefender/GravityZone tenant (Cascades company id `66b0448e1e0441d02508bad8`; 3 endpoints there, CS-SERVER absent). Defender is replaced by a Syncro-managed "Endpoint Protection Service". The previous MSP's **Datto RMM/CentraStage + Datto EDR/Infocyte** are still installed on top of Syncro + GuruRMM + ScreenConnect + KPAX -- overlapping agents thrashing the degraded spindle. Clean up the Datto stack. (Infection sweep 2026-06-15: clean.)
|
||||
|
||||
### Email & Identity
|
||||
|
||||
- **M365 tenant:** cascadestucson.com | Tenant ID: `207fa277-e9d8-4eb7-ada1-1064d2221498`
|
||||
- **M365 license:** Business Premium (SPB) — 34 seats enabled, 3 consumed, 31 free. Business Standard (O365_BUSINESS_PREMIUM) — **SUSPENDED**, 31 users still assigned. Relicensing 31 users Business Standard → Business Premium is pending and time-sensitive — those users may have degraded service.
|
||||
- **M365 license:** Business Premium (SPB) -- 34 seats enabled, 3 consumed, 31 free. Business Standard (O365_BUSINESS_PREMIUM) -- **SUSPENDED**, 31 users still assigned. Relicensing 31 users Business Standard -> Business Premium is pending and time-sensitive -- those users may have degraded service.
|
||||
- **On-prem AD domain:** cascades.local | UPN suffix: cascadestucson.com (added 2026-04-13 for Entra Connect SSO readiness)
|
||||
- **MX / mail flow:** Exchange Online (M365). SPF: `v=spf1 a mx ip4:72.194.62.5 include:spf.protection.outlook.com include:spf-0.secureserver.net -all`. DKIM: both M365 selectors published. DMARC: `p=quarantine;pct=100` — upgraded from p=none. Reports to `info@cascadestucson.com` (unmonitored). No third-party email gateway (EOP direct MX).
|
||||
- **MFA:** CA policy "Require MFA for all users" is enabled. Caregiver bypass in progress — caregivers cannot satisfy MFA (no personal device), so three scoped CA policies use BLOCK instead. See Patterns section. Voice-call MFA is **disabled tenant-wide** (SMS + Authenticator are the allowed methods). Exception: security group "MFA - Voice Call Scoped (sysadmin)" (id `304f941e-3594-4705-b8e6-ee676297df11`, single member `sysadmin@`) has Voice method enabled — created 2026-06-05 so Howard has a code-delivery path on the shared GA without a tenant-wide change. `sysadmin@` phone methods after 2026-06-05: mobile/SMS +1 520-289-1912 (Mike); alternateMobile/voice +1 520-585-1310 (Howard, was +1 520-331-5551).
|
||||
- **Entra Connect:** Installed on CS-SERVER 2026-04-25. Exited staging 2026-05-14 — actively syncing (last sync confirmed 2026-05-27). OU=Administrative not yet in sync scope; UPN suffix updates for Administrative OU users pending before that OU can be added.
|
||||
- **Break-glass accounts:** Two planned (`breakglass1-csc@cascadestucson.com`, `breakglass2-csc@cascadestucson.com`). Confirmed not yet created as of 2026-05-27 (live tenant check). FIDO2 YubiKeys ordered — arrival unconfirmed. Vault entries not yet created.
|
||||
- **MX / mail flow:** Exchange Online (M365). SPF: `v=spf1 a mx ip4:72.194.62.5 include:spf.protection.outlook.com include:spf-0.secureserver.net -all`. DKIM: both M365 selectors published. DMARC: `p=quarantine;pct=100` -- upgraded from p=none. Reports to `info@cascadestucson.com` (unmonitored). No third-party email gateway (EOP direct MX).
|
||||
- **MFA:** CA policy "Require MFA for all users" is enabled. Caregiver bypass in progress -- caregivers cannot satisfy MFA (no personal device), so three scoped CA policies use BLOCK instead. See Patterns section. Voice-call MFA is **disabled tenant-wide** (SMS + Authenticator are the allowed methods). Exception: security group "MFA - Voice Call Scoped (sysadmin)" (id `304f941e-3594-4705-b8e6-ee676297df11`, single member `sysadmin@`) has Voice method enabled.
|
||||
- **Entra Connect:** Installed on CS-SERVER 2026-04-25. Exited staging 2026-05-14 -- actively syncing (last sync confirmed 2026-05-27). OU=Administrative not yet in sync scope; UPN suffix updates for Administrative OU users pending before that OU can be added.
|
||||
- **Break-glass accounts:** Two planned (`breakglass1-csc@cascadestucson.com`, `breakglass2-csc@cascadestucson.com`). Confirmed not yet created as of 2026-05-27. FIDO2 YubiKeys ordered -- arrival unconfirmed.
|
||||
- **Admin accounts:**
|
||||
- `admin@cascadestucson.com` — Mike's working admin (cloud-only, Connect-excluded by design)
|
||||
- `sysadmin@cascadestucson.com` — Howard's working admin (cloud-only, Connect-excluded by design). Object id: `471b13dc-3cf8-416b-a132-f5f3bc8d1cc8`. Password rotated by Mike 2026-06-04; vaulted by Howard 2026-06-05 at `clients/cascades-tucson/m365-sysadmin.sops.yaml`.
|
||||
- **ALIS (clinical SaaS):** https://cascadestucson.alisonline.com — Entra SSO live and working; proven end-to-end with pilot.test on Galaxy A15 caregiver phones. Install key: `d796539d-356b-4190-9c17-35f0f1129376`. Vault: `clients/cascades-tucson/alis-sso-app-registration.sops.yaml` (Entra app reg + ALIS Inbound Connections Basic Auth creds + install key). ALIS application ID `d5108493-cba8-4f08-90b6-1bb0bc09eb2a`, client secret expires 2028-05-06 (rotation reminder — expiry breaks ALIS SSO tenant-wide). Per-caregiver: ALIS staff-record Email must match Entra UPN exactly. BAA with Medtelligent not yet verified — confirm with Meredith.
|
||||
- **Admin consent (2026-06-03):** Tenant-wide admin consent (`AllPrincipals` `User.Read`) granted on ALIS Entra service principal (`e1cae4ad-5beb-44ca-82d4-434c9bd835ad`) via Graph API (`oauth2PermissionGrant` id `reTK4etbykSC1ENMm9g1rTplOyzgVClCofKDVRrn-ds`). This resolved `AADSTS65001` sign-in failures that office/clinical staff (megan.hiatt, karen.rossini, memcarereceptionist) were hitting on non-phone devices. Root cause was missing admin consent — NOT Conditional Access, network, or password. Prior state: only two per-user (`Principal`) consent grants existed, so all other users hit 65001. CA policies had `conditionalAccessStatus: success` on all failing sign-ins; both WAN IPs were trusted Named Locations.
|
||||
- **How to enable ALIS SSO for one user (procedure — confirmed 2026-06-03):**
|
||||
1. User needs a valid Entra identity (synced or cloud-only both work).
|
||||
2. Tenant-wide admin consent for the ALIS app must exist — **done globally 2026-06-03**, so this is a one-time prerequisite, NOT per-user.
|
||||
3. In ALIS admin -> Staff -> the user's record, set the **Email field = the user's exact Entra UPN** (e.g. `crystal.rodriguez@cascadestucson.com`). This is the per-user SSO join key.
|
||||
4. User signs in via **"Sign in with Microsoft"** — not the ALIS username/password box.
|
||||
5. Turn off **ALIS-native 2FA** on that user's account (Entra is the second factor; native 2FA conflicts and locked out Karen Rossini on 2026-05-29).
|
||||
- **Diagnostic signature:** a user with **zero ALIS-app sign-in events in the Entra sign-in logs** is still on the old direct-login path (never reached Entra) — the fix is the ALIS Email match, not anything in Entra. Confirmed with Crystal Rodriguez (2026-06-03): identical to Megan Hiatt on identity, sync state, security group, and even held her own per-user consent grant — the ONLY difference was the missing ALIS Email match. Adding her email fixed SSO immediately. Megan worked because her ALIS record was already Email-matched and she used the Microsoft login; Crystal was falling back to direct ALIS login.
|
||||
- **Sweep target:** apply to all office/clinical users (Karen Rossini, MemCare reception, etc.) to standardize everyone onto SSO.
|
||||
- **Caregiver phones:** 22 Samsung Galaxy A15s enrolled in Intune Shared Device Mode (SDM). Enrollment profile: `CSC - Android Shared Phones (Entra SDM)` (`9a0fcc6d-0a88-466e-aa53-44401bb74fca`); 25 devices enrolled per 2026-06-03 Intune pull. Dynamic group: `Cascades - Shared Phones` (`ea96f4b7-3000-45da-ab1f-ddb28f509526`). Used by caregivers for Teams, Outlook, and ALIS. CA policies: block off-network, block non-compliant device (see below re: pending replacement with allow-list), 8h sign-in frequency. Android enrollment token expires 2027-05-08 — token is a join key only; expiry does NOT unenroll existing devices.
|
||||
- **Audit retention:** Approved 2026-04-29. Azure Log Analytics (90d) + Storage Account (6yr) in ACG subscription `e507e953-2ce9-4887-ba96-9b654f7d3267`, RG `rg-audit-cascadestucson`. **Not yet built.** Runbook: `.claude/skills/remediation-tool/references/audit-retention-runbook.md`.
|
||||
- **Inky:** No Inky deployment exists in this tenant. No connector, no transport rule, no OAuth app, no add-in. Confirmed 2026-06-04.
|
||||
- **EXO MSP app auth note (2026-06-04):** When the MSP app cert is not in the Windows cert store on a given machine, use client_credentials flow to obtain an EXO-scoped access token and connect via `Connect-ExchangeOnline -AccessToken`. This bypasses both the cert requirement and interactive MFA. App: ComputerGuru Exchange Operator (`b43e7342-5b4b-492f-890f-bb5a4f7f40e9`). Vault: `msp-tools/computerguru-exchange-operator.sops.yaml`.
|
||||
- **Shared mailboxes (created 2026-06-12):** `grievances@cascadestucson.com` (DisplayName "Grievances") and `Surveys@cascadestucson.com` (DisplayName "Surveys") — both SharedMailbox type, cloud-only, no license consumed (under 50 GB). Delegated to Meredith Kuhn (`meredith.kuhn@`) and Ashley Jensen (`ashley.jensen@`) with FullAccess (auto-mapping enabled) + SendAs on each (Send As chosen over Send on Behalf so outbound mail appears strictly from the shared address). Created via ComputerGuru Exchange Operator MSP app (`b43e7342`), cert-based EXO access token auth, `get-token.sh` tier `exchange-op`. `ExchangeOnlineManagement` module v3.10.0 was installed on Howard-Home (PSGallery, CurrentUser scope) for this session — it was not previously present on that machine. All 8 permission grants verified with `Get-MailboxPermission` / `Get-RecipientPermission` post-creation. Ticket #32417, 0.5h remote, invoice $0.00 prepaid.
|
||||
- `admin@cascadestucson.com` -- Mike's working admin (cloud-only, Connect-excluded by design)
|
||||
- `sysadmin@cascadestucson.com` -- Howard's working admin (cloud-only, Connect-excluded by design). Object id: `471b13dc-3cf8-416b-a132-f5f3bc8d1cc8`. Vaulted at `clients/cascades-tucson/m365-sysadmin.sops.yaml`.
|
||||
- **ALIS (clinical SaaS):** https://cascadestucson.alisonline.com -- Entra SSO live and working. Install key: `d796539d-356b-4190-9c17-35f0f1129376`. Vault: `clients/cascades-tucson/alis-sso-app-registration.sops.yaml`. ALIS application ID `d5108493-cba8-4f08-90b6-1bb0bc09eb2a`, client secret expires 2028-05-06 (rotation reminder -- expiry breaks ALIS SSO tenant-wide). Per-caregiver: ALIS staff-record Email must match Entra UPN exactly. BAA with Medtelligent not yet verified.
|
||||
- **Admin consent (2026-06-03):** Tenant-wide admin consent (`AllPrincipals` `User.Read`) granted on ALIS Entra service principal (`e1cae4ad-5beb-44ca-82d4-434c9bd835ad`). This resolved `AADSTS65001` sign-in failures. CA was NOT the cause.
|
||||
- **How to enable ALIS SSO for one user:** (1) Tenant-wide admin consent already done globally. (2) In ALIS admin -> Staff -> user's record, set **Email = exact Entra UPN**. (3) User signs in via "Sign in with Microsoft." (4) Turn off ALIS-native 2FA (Entra is the second factor; native 2FA conflicts and locked out Karen Rossini).
|
||||
- **Diagnostic signature:** a user with zero ALIS-app sign-in events in Entra sign-in logs is still on the old direct-login path -- fix is the ALIS Email match, not anything in Entra.
|
||||
- **Caregiver phones:** 22 Samsung Galaxy A15s enrolled in Intune Shared Device Mode (SDM). Enrollment profile: `CSC - Android Shared Phones (Entra SDM)` (`9a0fcc6d`); 25 devices enrolled per 2026-06-03 Intune pull. Dynamic group: `Cascades - Shared Phones` (`ea96f4b7`). Android enrollment token expires 2027-05-08 -- expiry does NOT unenroll existing devices.
|
||||
- **Audit retention:** Approved 2026-04-29. Azure Log Analytics (90d) + Storage Account (6yr) in ACG subscription `e507e953-2ce9-4887-ba96-9b654f7d3267`, RG `rg-audit-cascadestucson`. **Not yet built.**
|
||||
- **Inky:** No Inky deployment exists in this tenant. Confirmed 2026-06-04.
|
||||
- **EXO MSP app auth note (2026-06-04):** When the MSP app cert is not in the Windows cert store, use client_credentials flow to obtain an EXO-scoped access token and connect via `Connect-ExchangeOnline -AccessToken`. App: ComputerGuru Exchange Operator (`b43e7342-5b4b-492f-890f-bb5a4f7f40e9`). Vault: `msp-tools/computerguru-exchange-operator.sops.yaml`.
|
||||
- **Shared mailboxes (created 2026-06-12):** `grievances@cascadestucson.com` and `Surveys@cascadestucson.com` -- both SharedMailbox type, cloud-only, no license consumed. Delegated to Meredith Kuhn and Ashley Jensen with FullAccess (auto-mapping) + SendAs on each. All 8 permission grants verified. Ticket #32417.
|
||||
|
||||
### Network
|
||||
|
||||
- **ISP / WAN:** Dual-WAN Cox Fiber (primary, static `184.191.143.62/30`, gateway `184.191.143.61`) + Cox Coax (secondary, DHCP `72.211.21.217`). Both WAN IPs added as Cascades Named Location in Entra (ID: `061c6b06-b980-40de-bff9-6a50a4071f6f`).
|
||||
- **Firewall:** pfSense 24.0 at 192.168.0.1. All DHCP. Inter-VLAN routing. 236 resident room VLANs (per-room /28, `10.[floor].[room].0/28`). Staff/infra VLAN 20 (`10.0.20.0/24`, gateway `10.0.20.1`). Guest VLAN 50 (`10.0.50.0/24`, RFC1918 blocked).
|
||||
- **Switching:** Full UniFi. **77 U7-Pro APs** + ~9 managed switches (1st Floor USW-48 PoE core; floors 2-4 USW-Pro-24-PoE; MemCare USW-Pro-24-PoE; USW Lite 8 PoE; USW-16-PoE VoIP switch). All managed on the shared UOS controller (172.16.3.29; see [[uos-server]]); Cascades site_id `685f39068e65331c46ef6dd2`. Switch hardware replacement on floors 2/3/4 complete.
|
||||
- **ISP / WAN:** Dual-WAN Cox. WAN1 igc0 `184.191.143.62/30` (Cox Fiber, primary, gateway `184.191.143.61`) + WAN2 igc3 `72.211.21.217/27` (Cox Coax, secondary, static); `WAN_Group` gateway group; both active full-duplex, no loss events (verified 2026-06-16). Both WAN IPs added as Cascades Named Location in Entra (ID: `061c6b06-b980-40de-bff9-6a50a4071f6f`).
|
||||
- **Firewall:** pfSense Plus **25.07-RELEASE** (Netgate) at `192.168.0.1`, cert CN=pfSense-685f277aa6886. Admin vault: `clients/cascades-tucson/pfsense-firewall`. SSH shell access works (no interactive menu). OpenVPN user Howard: vault `clients/cascades-tucson/pfsense-openvpn-howard` (split-tunnel; `route 192.168.0.0/22`; use OpenVPN GUI or OpenVPN Connect with DCO disabled for stability -- DCO/TAP instability seen 2026-06-16). pfSense-ssh.sh (unifi-wifi skill) provides scripted audit/dhcp/run access.
|
||||
- **[INFO] pfSense health check (2026-06-16):** gateway ruled out as WiFi factor -- DHCP not exhausted (270/~507 active ~53% on the AP/WiFi pool), unbound DNS up, both WANs full-duplex/stable, firewall states 28-31k/790k, load 0.6. Minor: igc3/WAN2 Intel I225/226 2.5G counter quirk (1707 input-errors+collisions logged, full-duplex active, no loss) -- not a fault, no action needed.
|
||||
- **LAN / VLAN layout:** Primary staff/AP network `192.168.0.0/22` (pfSense .0.1, cascadesDS .0.120, UniFi APs + most WiFi clients on 192.168.2.x/3.x). DHCP pool 192.168.2.2-192.168.3.254 (~507 cap, ~270 active ~53%). Per-unit /28 VLANs: **199 DHCP subnets** total, mostly `10.x.y.0/28` per apartment (assisted-living L2 isolation) + Staff/Internal VLAN 20 (`10.0.20.0/24`, gw `10.0.20.1`) + Guest VLAN 50 (`10.0.50.0/24`, RFC1918 blocked). DHCP backend: ISC (Kea config present, dormant). Unbound DNS.
|
||||
- **Switching:** Full UniFi. **77 U7-Pro APs** + **12 managed switches** (1st Floor USW-48 PoE core; floors 2-4 USW-Pro-24-PoE; MemCare USW-Pro-24-PoE; USW Lite 8 PoE; USW-16-PoE VoIP switch). **[WARN] ~25 switch ports linked at 100 Mbps but gig-capable** (systematic cabling/NIC issue, 1st/2nd/3rd-floor switches; investigate after WiFi Phase A). 3 offline switches: Switch 2nd Floor #2, Switch 4th Floor #2, USW Pro Max 16. PoE budgets healthy. Port p38 (1st Floor USW) 4.0% tx-drop rate. All managed on the shared UOS controller (172.16.3.29, HTTPS 11443; see [[uos-server]]); Cascades site short name `va6iba3v`, site_id `685f39068e65331c46ef6dd2`. **Mesh topology:** 2nd Floor Atrium is wireless-mesh parent for CC Bridge + salon (5 GHz backhaul ch36); 206 U7 Pro carries AP 108. Switch hardware replacement on floors 2/3/4 complete.
|
||||
- **WiFi SSIDs:**
|
||||
- **CSCNet — shared PPSK SSID (corrected 2026-06-16; NOT a simple staff/VLAN-20 SSID).** `private_preshared_keys_enabled`; ~230 per-key->network mappings (most keys -> per-room resident VLANs 101-631; a few -> Default; one phone key -> Internal/VLAN 20). ~1,190 historical clients (residents' IoT/TVs, staff, phones). **Do NOT repoint the SSID to move a subset of clients** — move at the PPSK level (add a dedicated key for the target network). wlanconf `685f39078e65331c46ef7ee5`; cred vault `clients/cascades-tucson/wifi-cscnet.sops.yaml`.
|
||||
- CSC ENT — legacy SSID, main LAN (192.168.0.0/22), being deprecated as migration proceeds
|
||||
- Guest — isolated, VLAN 50
|
||||
- **Wireless RF status (live audit 2026-06-15 — ~574 concurrent clients):**
|
||||
- **2.4 GHz is the primary pain band:** avg TX-retry 11.2%, cu_total 69–94% live, catastrophic neighbor BSSID density (ch6 ~33k BSSIDs, ch1 ~19k, ch11 ~17k). 27 of the 40 worst clients on 2.4 GHz (retry 11–42%), mostly IoT/legacy (Ring cameras, robotic cleaner, smart plugs, EPSON printer, Poly phone, handheld scanners, smartwatch). Root cause: ~75 2.4 GHz radios running at auto (full) power in extreme density.
|
||||
- **5 GHz:** 80 MHz channel width on 76/77 APs (should be 40 MHz at this density). 55 of 77 5 GHz radios currently on DFS channels (52–144). DFS is a **resilience risk, not a throughput killer**: Tucson is near Davis-Monthan AFB + TUS airport radar; radar-detection events force channel-vacate + CAC silence → intermittent area-level client drops. Measured retry rate on DFS (8.4%) ≈ non-DFS clear channels (9.0%) — no throughput penalty observed today.
|
||||
- **6 GHz:** active on 75 radios; only 1 client of 574 connected. Largest untapped, clean, non-DFS capacity — band-steering capable clients to 6 GHz is a top opportunity.
|
||||
- **AP-level satisfaction 95–100 fleet-wide.** Pain is in the client tail, presenting as "bad for SOME users" — those whose devices land or stick on 2.4 GHz.
|
||||
- **Config flags (remediation pending):** 6 APs have 2.4 min-RSSI OFF (615, 608, 505, 517, 622, salon); 4 APs off the 1/6/11 channel plan on auto (128, 108, 108U7 Pro, salon); 2.4 TX power auto on ~75 radios.
|
||||
- **Known hardware:** AP 108 (Floor 1) offline pending a new cable run (expected); stale duplicate controller object ("108" vs "108U7 Pro") to clean up.
|
||||
- **Creds (vault refs only):** `infrastructure/uos-server-ssh-key` (SSH/Mongo access), `infrastructure/uos-server-network-api-rw` (RW controller admin), `clients/cascades-tucson/unifi-ap-ssh` (per-AP device auth via site VPN).
|
||||
- **VoIP (vendor: Vertical — Richard Turner <RTurner@vertical.com>):** Two phone fleets — **8 AudioCodes** (OUI `00:90:8f`, WIRED on USW-16-PoE ports 1-8, Default/main LAN) and **22 Poly** (OUI `48:25:67`, WiFi via CSCNet PPSK -> VLAN 20 Internal). The **Vertical-Remote management desktop** (`192.168.2.180`, MAC `e4:e7:49:52:3a:06`, WIRED USW-16-PoE port 16, Default LAN, **static IP, no ACG login**) is RDP-only (recon 2026-06-16 — not a PBX). No on-prem SIP PBX found -> phones appear to register to a **cloud/hosted PBX** (Vertical). Infra must stay static.
|
||||
- **[PLANNED] Voice VLAN (VLAN 30) consolidation for the phones:** Segmentation left voice gear split (Poly on VLAN 20; AudioCodes + Vertical desktop on the main LAN), and main-LAN -> VLAN 20 is blocked at pfSense — so the desktop can't reach the wireless phones and phone IPs drift. Fix: a dedicated isolated **VLAN 30 VOICE (`10.0.30.0/24`, gw `10.0.30.1`, pfSense igc1.30)** holding ALL phones + the Vertical desktop; internet egress allowed, firewalled off VLAN 20 / main LAN / PHI (HIPAA); Vertical's pfSense OpenVPN scoped to `10.0.30.0/24` via a Client-Specific-Override. Desktop is static + no ACG login -> Vertical sets it to DHCP (or grants temp access) at cutover; reserve `10.0.30.10`. Status: PLANNED — vendor email sent 2026-06-16, awaiting Richard's confirm (cloud-PBX, desktop static, VPN cert CN) + a window. **Full runbook + recon: `clients/cascades-tucson/docs/network/voice-vlan-cutover.md`.**
|
||||
- **CSCNet -- shared PPSK SSID.** `private_preshared_keys_enabled`; ~230 per-key->network mappings (most keys -> per-room resident VLANs 101-631; a few -> Default; one phone key -> Internal/VLAN 20). ~1,190 historical clients (residents' IoT/TVs, staff, phones). **Do NOT repoint the SSID to move a subset of clients** -- move at the PPSK level. wlanconf `685f39078e65331c46ef7ee5`; cred vault `clients/cascades-tucson/wifi-cscnet.sops.yaml`.
|
||||
- CSC ENT -- legacy SSID, main LAN (192.168.0.0/22), being deprecated as migration proceeds
|
||||
- Guest -- isolated, VLAN 50
|
||||
- **Wireless RF status (live audit 2026-06-15/16 -- ~587 concurrent clients):**
|
||||
- **2.4 GHz is the primary pain band:** avg TX-retry ~10%, cu_total 69-94% live, catastrophic neighbor BSSID density (ch6 ~33k BSSIDs, ch1 ~19k, ch11 ~17k). 27 of the 40 worst clients on 2.4 GHz (retry 11-42%), mostly IoT/legacy. Root cause: ~75 2.4 GHz radios running at auto (full) power in extreme density. Experience splits by band -- 5/6 GHz clients are fine; clients stuck on 2.4 GHz suffer.
|
||||
- **5 GHz:** 80 MHz channel width on 76/77 APs (should be 40 MHz at this density). 55/77 radios on DFS channels (52-144). DFS concern is theoretical resilience, not current throughput: `dfs-check.sh` 2026-06-16 confirmed **ZERO real radar events fleet-wide** (55 DFS APs, full `dmesg` sweep). Measured retry DFS (8.4%) ~= non-DFS (9.0%). Still plan to move to non-DFS (UNII-1 36-48 + UNII-3 149-161) for resilience near Davis-Monthan AFB. NOTE: an earlier mid-session claim (2026-06-15 audit) that "DFS was the #1 problem" was an artifact of tooling bugs (raw counter + 15-AP head cap) and was withdrawn -- do not repeat it.
|
||||
- **6 GHz:** active on 75 radios; only 1 client. Largest untapped, clean, non-DFS capacity -- band-steering 6E-capable clients to 6 GHz is the top opportunity.
|
||||
- **AP-level satisfaction 95-100 fleet-wide.** Pain is in the client tail, presenting as "bad for SOME users."
|
||||
- **Production change (2026-06-16):** Floor-4 2.4 GHz power-down pilot applied -- 14/15 radios to 6 dBm from ~23 dBm; avg retry 13.2->9.5% (~28% improvement); clients retained (no coverage loss). AP 445 lagged (config=Low but radio stayed 23dBm); left alone, harmless. AP 128 is disabled (intentionally). Disables for 445/428 held pending further validation. Remaining floors (1-3, 5-6) + full disable plan staged but NOT yet applied -- pending scope go-ahead from Howard.
|
||||
- **Config flags:** 6 APs with 2.4 min-RSSI OFF (615, 608, 505, 517, 622, salon); 4 APs off the 1/6/11 plan (128 disabled, 108 offline, 108U7 Pro auto, salon auto).
|
||||
- **Known hardware:** AP 108 (Floor 1) offline pending a new cable run (expected). Stale duplicate controller object ("108" vs "108U7 Pro") to clean up separately.
|
||||
- **Creds (vault refs only):** `infrastructure/uos-server-ssh-key` (SSH/Mongo), `infrastructure/uos-server-network-api-rw` (RW controller admin), `clients/cascades-tucson/unifi-ap-ssh` (per-AP device auth via site VPN), `clients/cascades-tucson/pfsense-firewall` (pfSense admin for pfsense-ssh.sh).
|
||||
- **VoIP (vendor: Vertical -- Richard Turner <RTurner@vertical.com>):** Two phone fleets -- **8 AudioCodes** (OUI `00:90:8f`, WIRED on USW-16-PoE ports 1-8, Default/main LAN) and **22 Poly** (OUI `48:25:67`, WiFi via CSCNet PPSK -> VLAN 20 Internal). The **Vertical-Remote management desktop** (`192.168.2.180`, MAC `e4:e7:49:52:3a:06`, WIRED USW-16-PoE port 16, Default LAN, **static IP, no ACG login**) is RDP-only (recon 2026-06-16 -- not a PBX). No on-prem SIP PBX found -> phones appear to register to a **cloud/hosted PBX** (Vertical). Infra must stay static.
|
||||
- **[PLANNED] Voice VLAN (VLAN 30) consolidation for the phones:** Segmentation left voice gear split (Poly on VLAN 20; AudioCodes + Vertical desktop on the main LAN), and main-LAN -> VLAN 20 is blocked at pfSense -- so the desktop can't reach the wireless phones and phone IPs drift. Fix: a dedicated isolated **VLAN 30 VOICE (`10.0.30.0/24`, gw `10.0.30.1`, pfSense igc1.30)** holding ALL phones + the Vertical desktop; internet egress allowed, firewalled off VLAN 20 / main LAN / PHI (HIPAA); Vertical's pfSense OpenVPN scoped to `10.0.30.0/24` via a Client-Specific-Override. Desktop is static + no ACG login -> Vertical sets it to DHCP (or grants temp access) at cutover; reserve `10.0.30.10`. Status: PLANNED -- vendor email sent 2026-06-16, awaiting Richard's confirm (cloud-PBX, desktop static, VPN cert CN) + a window. **Full runbook + recon: `clients/cascades-tucson/docs/network/voice-vlan-cutover.md`.**
|
||||
|
||||
### External Vendors & Mail Senders
|
||||
|
||||
@@ -216,23 +210,26 @@ Because per-user **Intune** never provisioned tenant-wide (`INTUNE_A = PendingIn
|
||||
|
||||
## Access
|
||||
|
||||
- **CS-SERVER:** Via ScreenConnect or GuruRMM (live agent ID `c39f1de7-d5b6-45ae-b132-e06977ab1713` as of 2026-06-08; re-enrolls — resolve live by hostname, do not hardcode)
|
||||
- **CS-SERVER:** Via ScreenConnect or GuruRMM (live agent ID `c39f1de7-d5b6-45ae-b132-e06977ab1713` as of 2026-06-08; re-enrolls -- resolve live by hostname, do not hardcode)
|
||||
- **CS-SERVER iDRAC:** 192.168.2.65
|
||||
- **pfSense admin:** https://192.168.0.1 — vault: `clients/cascades-tucson/pfsense-firewall.sops.yaml`
|
||||
- **Synology DSM:** http://192.168.0.120:5000 — vault: `clients/cascades-tucson/` (existing entry)
|
||||
- **M365 admin:** admin@cascadestucson.com — vault: `clients/cascades-tucson/m365-admin.sops.yaml`
|
||||
- **M365 sysadmin:** sysadmin@cascadestucson.com — vault: `clients/cascades-tucson/m365-sysadmin.sops.yaml`
|
||||
- **pfSense admin (HTTPS):** https://192.168.0.1 -- vault: `clients/cascades-tucson/pfsense-firewall.sops.yaml`
|
||||
- **pfSense SSH:** `ssh admin@192.168.0.1` (system OpenSSH; drops to shell directly, no interactive menu) -- vault admin cred: `clients/cascades-tucson/pfsense-firewall.sops.yaml`; pfsense-ssh.sh (unifi-wifi skill) for scripted access.
|
||||
- **pfSense OpenVPN (Howard):** split-tunnel; vault: `clients/cascades-tucson/pfsense-openvpn-howard.sops.yaml` (user `Howard`; route 192.168.0.0/22). Use OpenVPN GUI or OpenVPN Connect with DCO disabled for stability. Note: Howard-Home is now 10.137.42.0/24 (renumbered 2026-06-16) -- Cascades 192.168.0.x now reachable over the VPN.
|
||||
- **Synology DSM:** http://192.168.0.120:5000 -- vault: `clients/cascades-tucson/` (existing entry)
|
||||
- **M365 admin:** admin@cascadestucson.com -- vault: `clients/cascades-tucson/m365-admin.sops.yaml`
|
||||
- **M365 sysadmin:** sysadmin@cascadestucson.com -- vault: `clients/cascades-tucson/m365-sysadmin.sops.yaml`
|
||||
- **WiFi CSCNet:** vault: `clients/cascades-tucson/wifi-cscnet.sops.yaml`
|
||||
- **MDM service account:** vault: `clients/cascades-tucson/mdm-service-account.sops.yaml`
|
||||
- **svc-scan (scan-to-folder service account):** vault: `clients/cascades-tucson/svc-scan.sops.yaml` (`credentials.password`). AD account on CS-SERVER for the Accounting Brother's SMB scans — see Patterns -> File Shares & Scan-to-Folder.
|
||||
- **svc-scan (scan-to-folder service account):** vault: `clients/cascades-tucson/svc-scan.sops.yaml`. AD account on CS-SERVER for the Accounting Brother's SMB scans.
|
||||
- **ALIS SSO app registration:** vault: `clients/cascades-tucson/alis-sso-app-registration.sops.yaml`
|
||||
- **UOS controller SSH (root):** vault: `infrastructure/uos-server-ssh-key` — SSH/Mongo access for `unifi-wifi` skill and `uos-mongo.sh`. Vaulted 2026-06-15 by Mike.
|
||||
- **UOS controller RW admin (Network API):** vault: `infrastructure/uos-server-network-api-rw` — required to apply any radio/config changes. Vaulted 2026-06-15 by Mike.
|
||||
- **UniFi AP device auth (Cascades):** vault: `clients/cascades-tucson/unifi-ap-ssh` — direct AP SSH via site VPN (needed for `watch-ap.sh` live stream). Vaulted 2026-06-15 by Mike.
|
||||
- **GuruRMM — RECEPTIONIST-PC:** agent ID `9c91d324-1073-449c-8cc0-45c5bccfc218` (flaky WebSocket, may lag fleet updates)
|
||||
- **GuruRMM — ASSISTMAN-PC (Meredith Kuhn):** agent ID `cf86fa5e-96a2-494d-9cb1-8be22a518ad0`
|
||||
- **Remediation tool:** Full tiered app suite consented 2026-04-21. All six apps active: Security Investigator, Exchange Operator, User Manager, Tenant Admin, Defender Add-on, Intune Manager. Old app `fabb3421` (ComputerGuru - AI Remediation) still present but superseded.
|
||||
- **ComputerGuru Exchange Operator MSP app:** `b43e7342-5b4b-492f-890f-bb5a4f7f40e9` — vault: `msp-tools/computerguru-exchange-operator.sops.yaml`. Use access token auth when cert not in store (see Email & Identity section).
|
||||
- **UOS controller SSH (root):** vault: `infrastructure/uos-server-ssh-key` -- SSH/Mongo access for `unifi-wifi` skill and `uos-mongo.sh`. Vaulted 2026-06-15 by Mike.
|
||||
- **UOS controller RW admin (Network API):** vault: `infrastructure/uos-server-network-api-rw` -- required to apply any radio/config changes. Vaulted 2026-06-15 by Mike.
|
||||
- **UniFi AP device auth (Cascades):** vault: `clients/cascades-tucson/unifi-ap-ssh` -- direct AP SSH via site VPN (needed for `watch-ap.sh` live stream; L3 reach to 192.168.2.x/3.x via split-tunnel VPN). Vaulted 2026-06-15 by Mike.
|
||||
- **UOS controller (HTTPS):** https://172.16.3.29:11443 (HTTPS 11443, not 8443) -- site `va6iba3v` / site_id `685f39068e65331c46ef6dd2`
|
||||
- **GuruRMM -- RECEPTIONIST-PC:** agent ID `9c91d324-1073-449c-8cc0-45c5bccfc218` (flaky WebSocket, may lag fleet updates)
|
||||
- **GuruRMM -- ASSISTMAN-PC (Meredith Kuhn):** agent ID `cf86fa5e-96a2-494d-9cb1-8be22a518ad0`
|
||||
- **Remediation tool:** Full tiered app suite consented 2026-04-21. All six apps active: Security Investigator, Exchange Operator, User Manager, Tenant Admin, Defender Add-on, Intune Manager.
|
||||
- **ComputerGuru Exchange Operator MSP app:** `b43e7342-5b4b-492f-890f-bb5a4f7f40e9` -- vault: `msp-tools/computerguru-exchange-operator.sops.yaml`.
|
||||
- **Vault root:** `clients/cascades-tucson/` in vault repo
|
||||
|
||||
---
|
||||
@@ -241,18 +238,18 @@ Because per-user **Intune** never provisioned tenant-wide (`INTUNE_A = PendingIn
|
||||
|
||||
### Syncro / Billing
|
||||
|
||||
- **Never set a contact on any Syncro ticket unless explicitly requested.** This is a global rule, not Cascades-specific. At Cascades, Meredith Kuhn is the recurring wrong default that Syncro pre-selects — she is not the correct contact. Leave `contact_id` blank; Syncro routes to the correct distribution emails automatically. Source: `feedback_syncro_blank_contact.md`.
|
||||
- **Billing product for prepaid block draw:** Use a real labor type (Remote, Onsite, etc.) — NOT "Prepaid project labor" (exempt, won't decrement the block).
|
||||
- **Always live-check hours before billing:** `GET /customers/20149445` in Syncro. The 2026-05-01 invoice debit may not have fired correctly — treat all cached hour counts as approximate.
|
||||
- **Never set a contact on any Syncro ticket unless explicitly requested.** At Cascades, Meredith Kuhn is the recurring wrong default that Syncro pre-selects -- she is not the correct contact. Leave `contact_id` blank. Source: `feedback_syncro_blank_contact.md`.
|
||||
- **Billing product for prepaid block draw:** Use a real labor type (Remote, Onsite, etc.) -- NOT "Prepaid project labor" (exempt, won't decrement the block).
|
||||
- **Always live-check hours before billing:** `GET /customers/20149445` in Syncro. Treat all cached hour counts as approximate.
|
||||
|
||||
### Exchange Online / Message Tracing
|
||||
|
||||
- **Get-MessageTrace is hard-deprecated (Sept 2025).** As of 2025-09-01, `Get-MessageTrace` returns `BadRequest` / `ValidationException` via EXO InvokeCommand. Use `Get-MessageTraceV2` instead. Key parameter change: use `ResultSize` (not `PageSize`). The deprecation error may be silently swallowed by downstream jq filters — if a trace returns unexpectedly empty, check the raw response for a deprecation error string before assuming no mail. Source: 2026-06-04 Chris Knight investigation.
|
||||
- **Sender-side suppression (SendGrid ESP):** If a user never receives mail from a specific sender despite a healthy mailbox, and message trace shows zero records (not even bounces), consider a SendGrid suppression list. Resends will also fail silently. Fix requires contacting the sender's support to clear the suppression — there is no M365 action that can resolve this. Confirmed with bill.com / inform.bill.com. Pattern also applies to other high-volume senders using SendGrid.
|
||||
- **Get-MessageTrace is hard-deprecated (Sept 2025).** Use `Get-MessageTraceV2` instead. Key parameter change: use `ResultSize` (not `PageSize`). The deprecation error may be silently swallowed by downstream jq filters -- if a trace returns unexpectedly empty, check the raw response for a deprecation error string before assuming no mail. Source: 2026-06-04 Chris Knight investigation.
|
||||
- **Sender-side suppression (SendGrid ESP):** If a user never receives mail from a specific sender despite a healthy mailbox, and message trace shows zero records (not even bounces), consider a SendGrid suppression list. Fix requires contacting the sender's support to clear the suppression -- there is no M365 action that can resolve this. Confirmed with bill.com / inform.bill.com.
|
||||
|
||||
### Active Directory / User Management
|
||||
|
||||
- **Security group assignment is always explicit.** When creating or adding any Cascades user, always ask which security group(s). OU → group auto-mirror was explicitly declined 2026-05-14. OU placement controls Entra Connect sync scope; group membership controls CA policy — two separate deliberate decisions. Source: `feedback_cascades_user_security_group.md`.
|
||||
- **Security group assignment is always explicit.** When creating or adding any Cascades user, always ask which security group(s). OU -> group auto-mirror was explicitly declined 2026-05-14. Source: `feedback_cascades_user_security_group.md`.
|
||||
|
||||
- **New user mandatory order (folder redirection):**
|
||||
1. Create AD user
|
||||
@@ -265,127 +262,104 @@ Because per-user **Intune** never provisioned tenant-wide (`INTUNE_A = PendingIn
|
||||
|
||||
- **fdeploy1.ini flags:** Changed from `Flags=1211` (included `Grant Exclusive Rights` bit 0x400, causing WRITE_DAC failures on new subfolders) to `Flags=187`. File at `{512B43A4-F049-4CE5-BFAC-860AD13E92BE}\User\Documents & Settings\fdeploy1.ini` on CS-SERVER.
|
||||
|
||||
- **[ROOT CAUSE + FIX 2026-06-08] Native Folder Redirection was DOA on every machine — the config file was MISNAMED.** Every Cascades machine (LE + staff) had needed the manual `fix-shell-redirect.ps1` registry workaround because native FR never worked. Root cause: the redirect targets in GPO `CSC - Folder Redirection` (`{512B43A4-...}`) were saved in a file named **`fdeploy1.ini`**, but the Windows Folder Redirection client-side extension only ever reads **`fdeploy.ini`**. No `fdeploy.ini` existed, so the client knew *which* 5 folders to redirect but received an **empty target path** (FR Operational event 1006 shows `Path = ""`, no 1008 "successfully redirected") and silently did nothing. The file was hand-built by editing `fdeploy1.ini` (the wrong filename). **Fix:** wrote a correct `fdeploy.ini` (5 folders, `Flags=187`, `FullPath=\\CS-SERVER\Homes\%USERNAME%\<Folder>`) into `{512B43A4-...}\User\Documents & Settings\`, bumped the GPO version 917506→983042 (GPT.INI **and** AD `versionNumber` kept in sync), confirmed FR CSE registered. Backup of the original `\User` tree + GPT.INI at `C:\Windows\Temp\frfix-20260608-161144` on CS-SERVER. **Native FR now redirects all 5 folders on first logon — the registry workaround should no longer be needed for new users.** The dead `fdeploy1.ini` was left in place (ignored by Windows) — do NOT edit it; edit redirection only via GPMC or the `fdeploy.ini` artifact in `clients/cascades-tucson/gpo/`.
|
||||
- **LE GPO also broken:** `CSC - Folder Redirection (LE)` (`{889BE7BE-...}`, linked at OU=Life Enrichment) has a **completely empty `\User` tree** — no fdeploy at all. Sharon Edwards / Susan Hicks have likewise only ever worked via the registry workaround. Follow-up: retire the LE GPO and put LE users into `SG-FolderRedirect` (covered by the now-working all-staff GPO inherited at OU=Departments), or apply the same `fdeploy.ini` fix to the LE GPO. **Caveat:** Sharon/Susan are NOT currently in `SG-FolderRedirect` (the all-staff GPO is security-filtered to that group), so add them before relying on inheritance.
|
||||
- **Note:** the all-staff `CSC - Folder Redirection` GPO is linked at **OU=Departments** and security-filtered to **`SG-FolderRedirect`** (members as of 2026-06-08: Megan.Hiatt, Crystal.Rodriguez, Lois.Lane, Ashley.Jensen, lauren.hasselman, Zachary.Nelson, Nurses, chris.knight). Existing members get native redirection at their next sign-in.
|
||||
- **[ROOT CAUSE + FIX 2026-06-08] Native Folder Redirection was DOA on every machine -- the config file was MISNAMED.** Every Cascades machine had needed the manual `fix-shell-redirect.ps1` registry workaround because native FR never worked. Root cause: the redirect targets in GPO `CSC - Folder Redirection` (`{512B43A4-...}`) were saved in a file named **`fdeploy1.ini`**, but the Windows Folder Redirection client-side extension only ever reads **`fdeploy.ini`**. The file was hand-built by editing `fdeploy1.ini` (the wrong filename). **Fix:** wrote a correct `fdeploy.ini` (5 folders, `Flags=187`, `FullPath=\\CS-SERVER\Homes\%USERNAME%\<Folder>`) into `{512B43A4-...}\User\Documents & Settings\`, bumped the GPO version 917506->983042 (GPT.INI **and** AD `versionNumber` kept in sync). **Native FR now redirects all 5 folders on first logon -- the registry workaround should no longer be needed for new users.**
|
||||
- **LE GPO also broken:** `CSC - Folder Redirection (LE)` (`{889BE7BE-...}`, linked at OU=Life Enrichment) has a **completely empty `\User` tree**. Sharon Edwards / Susan Hicks have likewise only ever worked via the registry workaround. Follow-up: retire the LE GPO and put LE users into `SG-FolderRedirect`, or apply the same `fdeploy.ini` fix to the LE GPO. Sharon/Susan are NOT currently in `SG-FolderRedirect` -- add them before relying on inheritance.
|
||||
|
||||
- **Login-screen hide (SpecialAccounts\UserList):** An enabled local admin that does not appear in the Windows sign-in picker is a `SpecialAccounts\UserList` suppression, not a disabled account. Registry path: `HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon\SpecialAccounts\UserList`, value `<username>=0`. Fix: delete the DWORD value (or set it to 1); account reappears after sign-out/reboot. Confirmed on NURSESTATION-PC (RMM agent `f5a89784-834f-47b1-82e2-7e3e9dd337ff`) 2026-06-05 — `localadmin=0` removed; account was already enabled and in Administrators (unchanged).
|
||||
- **Login-screen hide (SpecialAccounts\UserList):** An enabled local admin that does not appear in the Windows sign-in picker is a `SpecialAccounts\UserList` suppression, not a disabled account. Registry path: `HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon\SpecialAccounts\UserList`, value `<username>=0`. Fix: delete the DWORD value; account reappears after sign-out/reboot. Confirmed on NURSESTATION-PC 2026-06-05 -- `localadmin=0` removed; account was already enabled and in Administrators.
|
||||
|
||||
### File Shares & Scan-to-Folder (Accounting)
|
||||
|
||||
- **Accounting department folder + scan dropbox (built 2026-06-09):**
|
||||
- `D:\Shares\Accounting` on CS-SERVER — inheritance broken; **SYSTEM / BUILTIN\Administrators = Full; `lauren.hasselman`, `chris.knight`, `zachary.nelson` = Modify** (no Everyone). Shared as **`\\CS-SERVER\AcctDept`** (Change: those 3 users + `svc-scan`; Full: Admins).
|
||||
- **Share is named `AcctDept`, NOT `Accounting`** — a *printer* share named `Accounting` (Canon MF455DW, `LocalsplOnly`) already exists. Do not collide with it: `New-SmbShare -Name Accounting` / `Grant-SmbShareAccess -Name Accounting` will silently hit the printer share. (Happened 2026-06-09; printer share's Everyone:Read was restored.)
|
||||
- `D:\Shares\Accounting\Scans` — scan dropbox; inherits the 3 users + adds **`CASCADES\svc-scan` = Modify** (least-privilege writer; can't read the rest of Accounting; bypass-traverse lets it reach the subfolder).
|
||||
- `D:\Shares\Accounting` on CS-SERVER -- inheritance broken; SYSTEM / BUILTIN\Administrators = Full; `lauren.hasselman`, `chris.knight`, `zachary.nelson` = Modify (no Everyone). Shared as **`\\CS-SERVER\AcctDept`** (Change: those 3 users + `svc-scan`; Full: Admins).
|
||||
- **Share is named `AcctDept`, NOT `Accounting`** -- a *printer* share named `Accounting` (Canon MF455DW, `LocalsplOnly`) already exists. Do not collide with it.
|
||||
- **`svc-scan`** = dedicated AD service account (CN=Users, PasswordNeverExpires, CannotChangePassword) for the Brother's SMB auth. Vault: `clients/cascades-tucson/svc-scan.sops.yaml`.
|
||||
- **REUSE `svc-scan` for EVERY future scanner→network-folder setup at Cascades** (Howard, 2026-06-09) — do NOT create a per-printer/per-folder scan account. For a new scan destination: grant `CASCADES\svc-scan` Modify on the new scan folder, then enter `cascades\svc-scan` + the vaulted password (NTLMv2) in that scanner's Scan-to-Network profile.
|
||||
- **Brother MFC-L8900CDW "Business Office" printer (10.0.20.220) — Scan-to-Network profile (working 2026-06-09):** Network Folder Path `\\192.168.2.254\AcctDept\Scans`; **Auth Method NTLMv2** (not Auto/Kerberos — printer can't KDC across VLAN); Username `cascades\svc-scan`; PDF Multi-Page. Configured via the printer WBM (`http://10.0.20.220`), panel: Scan -> to Network.
|
||||
- **[NETWORK] CS-SERVER cannot reach the VLAN-20 printers** — main-LAN `192.168.2.x` -> VLAN 20 `10.0.20.x` is blocked at pfSense. Verified: CS-SERVER -> `10.0.20.220`:80/443/445 all fail. So you **cannot configure a 10.0.20.x printer's web UI from CS-SERVER** — use a VLAN-20 PC's browser (e.g. ACCT2-PC `10.0.20.209`) or go onsite. The reverse (printer -> CS-SERVER:445) **is** open, which is all scan-to-folder needs (svc-scan SMB write verified from ACCT2-PC).
|
||||
- **Persistent drive maps to `\\cs-server\AcctDept`** (per-user, via RMM `user_session`): Chris (DESKTOP-N5G1ROO) **Y:**, Zachary (ACCT2-PC) **Y:**, Lauren (DESKTOP-H6QHRR7) **X:** (Y: was already in use on hers).
|
||||
- **REUSE `svc-scan` for EVERY future scanner->network-folder setup at Cascades** (Howard, 2026-06-09) -- do NOT create a per-printer/per-folder scan account. For a new scan destination: grant `CASCADES\svc-scan` Modify on the new scan folder, then enter `cascades\svc-scan` + the vaulted password (NTLMv2) in that scanner's Scan-to-Network profile.
|
||||
- **Brother MFC-L8900CDW "Business Office" printer (10.0.20.220) -- Scan-to-Network profile (working 2026-06-09):** Network Folder Path `\\192.168.2.254\AcctDept\Scans`; **Auth Method NTLMv2** (not Auto/Kerberos -- printer can't KDC across VLAN); Username `cascades\svc-scan`; PDF Multi-Page.
|
||||
- **[NETWORK] CS-SERVER cannot reach the VLAN-20 printers** -- main-LAN `192.168.2.x` -> VLAN 20 `10.0.20.x` is blocked at pfSense. Use a VLAN-20 PC's browser (e.g. ACCT2-PC `10.0.20.209`) or go onsite. The reverse (printer -> CS-SERVER:445) **is** open.
|
||||
- **Persistent drive maps to `\\cs-server\AcctDept`:** Chris (DESKTOP-N5G1ROO) Y:, Zachary (ACCT2-PC) Y:, Lauren (DESKTOP-H6QHRR7) X: (Y: was already in use on hers).
|
||||
|
||||
### Synology NAS (cascadesDS) / Shared File Access
|
||||
|
||||
- **Stale Word owner (lock) files on cascadesDS shares:** Word creates a hidden `~$<truncated filename>` owner file when a document is opened; if the user's session ends without cleanly closing Word (crash, logoff with file open), the `~$` file is orphaned. Any later open of the same document displays "locked for editing by [name]" even with no live session. Confirmed 2026-06-10: five `~$` files dated 2024 on `\\cascadesds\Public\Company Web Docs\Staff Trainings\` caused false lock messages across several training docs. **Diagnosis:** list the folder for `~$` files; check the timestamp — if hours or days old with no matching active session, it is stale. **Fix:** delete the `~$` file(s). If the file is still locked after deleting orphaned owner files, check Synology DSM -> File Services -> Resource Monitor for a live SMB handle and clear it there.
|
||||
- **Accessing cascadesDS from RMM — always use a user session, not CS-SERVER SYSTEM.** The domain-joined CS-SERVER machine account cannot authenticate to the Synology `Public` share because cascadesDS uses workgroup "CASCADES" (same short name as the AD domain), causing Kerberos auth failures. CS-SERVER SYSTEM → `\\cascadesds\*` returns access denied. Workaround: run the command in the `user_session` context of a machine where the target user is actively logged in (e.g. ASSISTMAN-PC agent `cf86fa5e` for Meredith-accessible shares). When constructing UNC paths in PowerShell over the RMM transport, use char-code path construction to avoid backslash loss across bash → jq → agent → PowerShell (`[char]92` for `\`).
|
||||
- **Stale Word owner (lock) files on cascadesDS shares:** Word creates a hidden `~$<truncated filename>` owner file when a document is opened; if the user's session ends without cleanly closing Word, the `~$` file is orphaned. **Fix:** delete the `~$` file(s). Confirmed 2026-06-10: five `~$` files dated 2024 on `\\cascadesds\Public\Company Web Docs\Staff Trainings\` caused false lock messages.
|
||||
- **Accessing cascadesDS from RMM -- always use a user session, not CS-SERVER SYSTEM.** The domain-joined CS-SERVER machine account cannot authenticate to the Synology `Public` share because cascadesDS uses workgroup "CASCADES" (same short name as the AD domain), causing Kerberos auth failures. Workaround: run the command in the `user_session` context of a machine where the target user is actively logged in (e.g. ASSISTMAN-PC agent `cf86fa5e` for Meredith-accessible shares).
|
||||
|
||||
### Browser / Edge
|
||||
|
||||
- **[BUG - FLEET] Edge 149 cannot open Office files via download-list when Downloads is a UNC-redirected folder (Chromium issue 519243472).** A regression introduced in Chromium 149 (feature `LaunchShellExecuteViaExplorer`) prepends `\\?\` to UNC paths without converting to the correct `\\?\UNC\` form, producing a malformed path (`\\?\\\cs-server\...`). **Symptom:** clicking an `.xlsx` or `.docx` in the Edge download panel shows "Windows cannot find '\\?\\\cs-server\homes\<user>\Downloads\<file>'." Text files and PDFs open fine from the same panel (PDF uses Edge's built-in viewer and does not invoke ShellExecute; Office routes through the broken external-launch path). The same Office file double-clicked from File Explorer opens normally. **Trigger:** Downloads folder redirected via GPO Folder Redirection to a UNC path with **no mapped drive letter** (`\\cs-server\homes\<user>\Downloads`) — exactly Cascades' Homes-share redirect configuration. **Affected build:** Edge stable 149.0.4022.52 (Chromium 149 base); last known-good: Chromium 148 (148.0.7778.217). **Cascades exposure as of 2026-06-08:** Ashley Jensen (DESKTOP-U2DHAP0) and Lois Lane (DESKTOP-KQSL232) confirmed on 149.0.4022.52; fleet-wide for any Cascades user whose Downloads is redirected to `\\cs-server\homes` and who is running Edge 149. **Fix options (none applied as of 2026-06-08 session; decision left to Howard):**
|
||||
1. Update Edge forward past the fix (Chromium fix crrev 7900033 "Correctly handle UNC paths in InvokeShellExecute," merged M149/M150, verified Chromium 151.0.7875.0 — preferred when a patched stable ships).
|
||||
2. Interim feature flag: add `--disable-features=LaunchShellExecuteViaExplorer` to the Edge shortcut target (quit Edge fully first; applies only to launches from that shortcut).
|
||||
3. Zero-config workaround: use "Show in folder" in the Edge download panel, then double-click from File Explorer.
|
||||
4. Supported 149→148 rollback (one major back is in-bounds): download 148 stable MSI from https://www.microsoft.com/en-us/edge/business/download; set `HKLM\SOFTWARE\Policies\Microsoft\Edge\RollbackToTargetVersion` (DWORD) = 1 **before** install; pin via `HKLM\SOFTWARE\Policies\Microsoft\EdgeUpdate\TargetVersionPrefix{56EB18F8-B008-4CBD-B6D2-8C97FE7E9062}` = `148.` and `Update{56EB18F8-...}` = 2; unwind the pin once a fixed 149.x/150 ships. Edge stable app GUID: `{56EB18F8-B008-4CBD-B6D2-8C97FE7E9062}`. Note: pinning to 148 forfeits 149 security fixes; prefer option 1 or 3 for HIPAA machines.
|
||||
- **[BUG - FLEET] Edge 149 cannot open Office files via download-list when Downloads is a UNC-redirected folder (Chromium issue 519243472).** A regression introduced in Chromium 149 (feature `LaunchShellExecuteViaExplorer`) prepends `\\?\` to UNC paths without converting to the correct `\\?\UNC\` form, producing a malformed path. **Symptom:** clicking an `.xlsx` or `.docx` in the Edge download panel shows "Windows cannot find '\\?\\\cs-server\...'" Text files and PDFs open fine. The same Office file double-clicked from File Explorer opens normally. **Trigger:** Downloads folder redirected via GPO Folder Redirection to a UNC path with no mapped drive letter -- exactly Cascades' Homes-share redirect configuration. **Affected build:** Edge stable 149.0.4022.52. **Fix options (none applied as of 2026-06-08):** (1) Update Edge past the fix; (2) Interim: `--disable-features=LaunchShellExecuteViaExplorer`; (3) Zero-config: use "Show in folder" then double-click from Explorer; (4) Supported 149->148 rollback. Note: pinning to 148 forfeits security fixes; prefer option 1 or 3 for HIPAA machines.
|
||||
|
||||
### Conditional Access / Caregiver Policies
|
||||
|
||||
- **Phased rollout — never tenant-wide.** CA policies for caregivers now target `SG-Caregivers` (`8b8d9222-5d71-419a-936d-56d895c6c332`) (Entra Connect exited staging 2026-05-14; SG-Caregivers-Pilot superseded). The legacy "Require MFA for all users" policy stays in place. Expansion to other departments uses PATCH on `excludeGroups`, never replace. Source: `project_cascades_ca_phased_rollout.md`.
|
||||
- **Phased rollout -- never tenant-wide.** CA policies for caregivers now target `SG-Caregivers` (`8b8d9222-5d71-419a-936d-56d895c6c332`) (Entra Connect exited staging 2026-05-14; SG-Caregivers-Pilot superseded). The legacy "Require MFA for all users" policy stays in place. Source: `project_cascades_ca_phased_rollout.md`.
|
||||
- **Enforced caregiver CA policy set (unchanged as of 2026-06-03):**
|
||||
- `CSC - Block caregivers off Cascades network` (`e35614e1-e896-4a13-9407-076963af488f`) — BLOCK if location not Cascades
|
||||
- `CSC - Block caregivers on non-compliant device` (`ede985e2-ee7e-4521-88b2-34c847c3db20`) — BLOCK if device non-compliant. **Pending DISABLE** at allow-list cutover (see below).
|
||||
- `CSC - Block caregivers off Cascades network` (`e35614e1-e896-4a13-9407-076963af488f`) -- BLOCK if location not Cascades
|
||||
- `CSC - Block caregivers on non-compliant device` (`ede985e2-ee7e-4521-88b2-34c847c3db20`) -- BLOCK if device non-compliant. **Pending DISABLE** at allow-list cutover.
|
||||
- `CSC - Caregiver sign-in frequency 8h` (`7d491c7a-ad90-4420-9990-40a1e676a76c`)
|
||||
- **Caregiver device allow-list (2026-06-03 — report-only):** The device restriction is being changed from compliance-based to an explicit device allow-list (phones matching `displayName -startsWith "CSC-"` plus 5 tagged laptops/PCs with `extensionAttribute1=CSCCaregiverDevice`). Rationale: tenant has no Windows compliance policy and `secureByDefault=false`, meaning compliance-only would admit any future-enrolled machine. New CA policy created in report-only:
|
||||
- `CSC - Caregivers: allow-listed devices only (REPORT-ONLY)` — id `1b7fd025-1aad-47c8-9274-c32c3e0b163c`; state `enabledForReportingButNotEnforced`
|
||||
- Target group: `SG-Caregivers` (`8b8d9222`). Excludes: `sysadmin@`, `admin@`, `SG-CA-BreakGlass` (`131e51ac-d69b-44b8-9c81-56890537a796`)
|
||||
- Device filter (mode `exclude`): `(device.displayName -startsWith "CSC-") -or (device.extensionAttribute1 -eq "CSCCaregiverDevice")`
|
||||
- **Allowed device list (target — 6 caregiver/medtech devices, tagged `CSCCaregiverDevice`):**
|
||||
|
||||
| Device | OS | GuruRMM agent | Notes |
|
||||
|---|---|---|---|
|
||||
| NURSESTATION-PC | Win 11 (26200) | `8164c6fa-62e7-4aa5-88e4-624f2f656932` | hybrid-join track; tagged |
|
||||
| Laptop2 | Win 11 (26200) | `dc8daf71-a2e6-4181-8cf2-c463c95dcd7d` | already Pro; Entra-joined + tagged |
|
||||
| LAPTOP-DRQ5L558 | Win 11 (26200) | `f9e25b3b-da63-40ff-94a6-8cec3b9a19ce` | Win10 Home→Win11 Pro (our key); joined + tagged |
|
||||
| LAPTOP-E0STJJE8 | Win 11 (26200) | `4ac00700-9a9b-4e7f-a7aa-c51857b77661` | Win10 Home→Win11 Pro (our key); joined + tagged |
|
||||
| LAPTOP-8P7HDSEI | Win 10/11 — verify | `9b74852c-623a-4d4a-bdda-1709ee75ae44` | was Win10 19045; Win11 25H2 upgrade + join/tag pending verification |
|
||||
| ASSISTNURSE-PC | **Win 11 Pro for Workstations 24H2 (clean reinstall 2026-06-08)** | **`62d108d6` (new — re-enrolled after reinstall; old `88891eb8` deleted)** | shared MC medtech device (Christine Nyanzunda + medtechs). **NEW Entra device object** after reinstall → needs re-join + re-tag `CSCCaregiverDevice` before allow-list cutover; old Entra device record to clean. 3 caregiver Public-Desktop shortcuts (ALIS/LinkRx/Helpany) deployed via RMM 2026-06-08 |
|
||||
|
||||
- **Join model (decided 2026-06-03):** The 4 laptops are **Entra-joined (cloud join)**, NOT domain-joined — a domain-only PC has no Entra device object, so the CA device filter cannot allow-list it. The laptops are shared ALIS/Teams/Outlook access points and do not need the on-prem GPO stack. NURSESTATION-PC stays domain-joined and gets **Hybrid Entra Join** (needs on-prem printers + ALDocs share); requires a one-time device-options config in Entra Connect on CS-SERVER, and its stale 2021 Entra record (Workplace, last seen 2021-07-03) should be cleaned. Mixed model is supported.
|
||||
- **Enrollment account:** `devices@cascadestucson.com` (Cloud Device Administrator, `aaca80c6-861b-4294-8068-1033c68d7667`). **Licensed Business Premium + usageLocation=US on 2026-06-04** and ready to join/auto-enroll. The license is needed **only at enrollment time** so auto-MDM-enroll fires; the device stays enrolled and allow-listed afterward regardless of the enroller's license, so the SPB seat can be reclaimed after the batch (30 SPB seats free as of 2026-06-03). One license covers sequential enrollments. Mark each laptop a shared device (remove primary user) to drop per-user license dependency. Confirm MDM user scope = All (Entra -> Devices -> Mobility) before joining — not verifiable via API.
|
||||
- **Printing:** does NOT require domain join — Entra-joined laptops print via direct IP network printers or an Intune-pushed `Add-Printer` config. Printers: FrontDesk Epson ET-5800 `192.168.2.147`, CopyRoom Canon C478iF `192.168.2.230`, MCReception Epson ET-5800.
|
||||
- **Enrollment progress (updated 2026-06-08):** 3 laptops Entra-joined + tagged `CSCCaregiverDevice` — Laptop2, LAPTOP-DRQ5L558, LAPTOP-E0STJJE8 (all Win11 26200). **ASSISTNURSE-PC upgraded 2026-06-08** — clean Win11 reinstall (was Win10 19045; in-place upgrades failed), RMM re-enrolled (`62d108d6`), but the reinstall created a NEW Entra device object so it still needs re-join + re-tag before cutover. Still pending: LAPTOP-8P7HDSEI Win11 25H2 upgrade + join/tag (verify current state). NURSESTATION-PC confirmed permanent caregiver device (hybrid-joined 2026-06-05). Full set = phones + those 6 machines. All joined laptops show `isManaged=null` (auto-MDM-enroll did not fire — MDM user scope likely not =All, and only local logins so far). Intune is OPTIONAL: the allow-list is tag-based and works on Entra-join alone; Intune only needed for printer-push / a Windows compliance policy. Intune/MDM decision deferred until all devices on Win11 25H2. Enrollment account `devices@` (Cloud Device Admin), licensed Business Premium transiently (reclaim after batch).
|
||||
- **Cutover (low-risk, can be all-at-once):** verified no gap — only `CSC-` phones are compliant today and the allow-list also permits them, so enabling the allow-list ADDS the laptops without removing phone access; nobody on a phone gets locked out. Per-user go-live gate is the ALIS email-match + test sign-in (one at a time), not a CA change. Cutover = enable `CSC - Caregivers: allow-listed devices only` + disable `CSC - Block caregivers on non-compliant device`.
|
||||
- **Restricted vs privileged classification (2026-06-04):** Restricted/inside (SG-Caregivers) = the 38 + Veronica Feller (caretaker; inventory shows her remote/PA — confirm on-site) + Christine Nyanzunda (MC admin asst + PT medtech; uses ASSISTNURSE-PC; directory surname typo "Nyanzuda" to fix). Privileged/outside (NOT in SG-Caregivers; ALIS via SSO + offsite MFA) = Lois Lane, Karen Rossini, Christina DuPras, and all admins/directors/managers; nurses ruled OUTSIDE. Zachary Nelson is accounting/no-ALIS (not a caregiver). Still pending classification: Judith Palmer, Patricia Sandoval-Beck, Joey Ty, Alejandra Vallejo, Celia Lassey. Worklist: `clients/cascades-tucson/reports/2026-06-04-caregiver-alis-sso-worklist.md`.
|
||||
- **User<->computer map source:** Syncro `kabuto_information.last_user` (GuruRMM does not expose logged-in user). DuPras=ALASSIST-PC, Lois Lane=DESKTOP-KQSL232, Karen Rossini=DESKTOP-LPOPV30, shared medtech=ASSISTNURSE-PC, shared MemCare reception=MEMRECEPT-PC (excluded from caregiver allow-list, receptionist-only). CONTEXT.md GuruRMM roster stale (27->32) — refresh pending.
|
||||
- **Caregiver desktop app shortcuts:** ALIS (`https://cascadestucson.alisonline.com`), LinkRx (`https://pharmcare.linkrxnow.com/`), HelpAny (`https://app.safe-living.com/login`) — deploy via a Public-Desktop PowerShell script launching Edge `--app` mode (preserves SSO device-claim), pushed via GuruRMM to the 6 caregiver machines.
|
||||
- **Login UX:** Entra/Microsoft sign-in (and ALIS SSO) requires the full UPN — no bare-username option for cloud accounts. Minimize typing via Windows Hello PIN on laptops + silent ALIS SSO once signed in; pursue ALIS Login PINs (Medtelligent limited-release).
|
||||
- **Caregiver test rig (2026-06-05, validated):** Phased-test infra before promoting to all caregivers. `SG-Caregivers-DeviceTest` (`db5849ec`, USERS) carries the full caregiver rule set (off-network block + sign-in-freq + allow-list, excluded from compliance-block); `Cascades - Caregiver Devices` (`02c6f698`, STATIC devices) targets Intune profiles (NURSESTATION only for now); `SG-Intune-Enrollment` (`13d94f6e`, holds devices@) scopes MDM auto-enroll. Test acct `pilot.test@cascadestucson.com` (`d26e0e5a`, Business Premium, ephemeral). NURSESTATION-PC is **Hybrid Entra Joined** (re-domain-joined Win11 25H2; new deviceId `d3bf931f-f128-4261-8398-b46c34a4b342`, object id `de199a15-3f5d-4da3-8b17-3faade7f7dad`, trustType `ServerAd`). Intune profiles (idle-lock 5min + disable-WHfB OMA-URI) assigned to device group but **NOT yet applied** — `INTUNE_A: PendingInput` tenant-wide blocks enrollment on newly-licensed accounts (devices@, pilot.test); MS case open; does NOT block caregiver access (GPO path used instead). **PROVEN 2026-06-05:** pilot.test on NURSESTATION-PC -> ALIS opened via SSO with lockdown holding (off-network blocked, only allow-listed device passes). ALIS first threw CA 53003 because the `extensionAttribute1` tag takes >70 min to propagate into CA's device-filter cache; fixed by adding NURSESTATION's **deviceId** directly to the allow-list rule (immediate, lag-free) — for the small caregiver device set, **deviceId matching is the reliable lever**. Windows Hello does NOT auto-provision on hybrid-joined machines (`WillNotProvision: PolicyEnabled NO`). **GPOs deployed 2026-06-05:** `CSC - Caregiver Workstation` validated on pilot.test; `CSC - Caregiver Device Lockdown` deployed to `OU=Caregiver Devices` (activates on reboot). **Monday go-live:** promote allow-list + GPO filter from test group to `SG-Caregivers`; disable compliance-block; move real machines in one at a time.
|
||||
- **Threat model (confirmed 2026-06-05):** off-network + device allow-list specifically defeats remote credential abuse (hacker / bad employee from home) — stolen caregiver creds unusable off-site/off-device because CA blocks the cloud sign-in before ALIS/email. Risk-based MFA policies are inert (tenant has no Identity Protection P2 license).
|
||||
- **Caregiver device allow-list (2026-06-03 -- report-only):** `CSC - Caregivers: allow-listed devices only (REPORT-ONLY)` -- id `1b7fd025-1aad-47c8-9274-c32c3e0b163c`; state `enabledForReportingButNotEnforced`. Device filter (mode `exclude`): `(device.displayName -startsWith "CSC-") -or (device.extensionAttribute1 -eq "CSCCaregiverDevice")`. Includes: NURSESTATION-PC (deviceId `d3bf931f`), Laptop2, LAPTOP-DRQ5L558, LAPTOP-E0STJJE8, LAPTOP-8P7HDSEI, ASSISTNURSE-PC (needs re-join + re-tag after Win11 reinstall).
|
||||
- **GDAP exclusion:** CA policy 3 must exclude "Service provider users" (GDAP foreign principals) + `SG-External-Signin-Allowed` + `SG-Break-Glass`, otherwise ACG partner admins lose access at CA cutover.
|
||||
- **Known bug:** `Require MFA for all users` policy (`7e87a1c7...`) excludes `SG-Caregivers-Pilot` instead of the live `SG-Caregivers` (`8b8d9222`). Functionally harmless today (pilot group still exists), but must be corrected.
|
||||
- **Pilot cleanup required when done:** Delete `pilot.test@cascadestucson.com`, clean up `howard.enos@cascadestucson.com`, remove `SG-Caregivers-Pilot` from CA policy targets and delete the group. Source: `project_cascades_pilot_cleanup.md`.
|
||||
|
||||
### EXO / Message Trace
|
||||
|
||||
- **Get-MessageTrace is deprecated.** Use `Get-MessageTraceV2` instead. V2 has a 10-day max window — loop 9 consecutive windows to cover 90 days. A wildcard sender with a 30-day window returns false positives due to the window-limit violation; keep windows to 10 days and use specific sender domains.
|
||||
- **EXO access token auth:** When `Connect-ExchangeOnline -Credential` fails (MFA/modern auth block) and the app cert is not in the Windows cert store, use client_credentials flow to get an EXO-scoped token and pass it via `-AccessToken`. See access note in the Access section above.
|
||||
- **Get-MessageTrace is deprecated.** Use `Get-MessageTraceV2` instead. V2 has a 10-day max window -- loop 9 consecutive windows to cover 90 days.
|
||||
- **EXO access token auth:** When `Connect-ExchangeOnline -Credential` fails and the app cert is not in the Windows cert store, use client_credentials flow to get an EXO-scoped token and pass it via `-AccessToken`.
|
||||
|
||||
### Wireless / UniFi RF
|
||||
|
||||
- **Fleet (live audit 2026-06-15):** 77 U7-Pro APs, ~9 UniFi switches, ~574 concurrent wireless clients. Managed on shared UOS controller (172.16.3.29; see [[uos-server]]); Cascades site_id `685f39068e65331c46ef6dd2`.
|
||||
- **Primary pain band is 2.4 GHz.** Avg TX-retry 11.2%; cu_total 69–94% live; catastrophic neighbor BSSID density (ch6 ~33k BSSIDs, ch1 ~19k, ch11 ~17k). 27 of the 40 worst clients stuck on 2.4 GHz (retry 11–42%), mostly IoT/legacy hardware (Ring cameras, robotic cleaner, smart plugs, EPSON printer, Poly phone, handheld scanners, smartwatch). Root cause: ~75 2.4 GHz radios running at auto (full) TX power in extreme AP density.
|
||||
- **5 GHz — DFS is a resilience risk, not a throughput killer.** 76/77 radios on 80 MHz width (should be 40 MHz at this density). 55/77 radios on DFS channels (52–144). Davis-Monthan AFB + TUS airport radar are nearby → radar-detection events force channel-vacate + CAC silence → intermittent area-level client drops. Measured TX-retry rate on DFS radios (8.4%) is approximately equal to non-DFS (9.0%) — no throughput penalty observed. An earlier mid-session claim during the 2026-06-15 audit that DFS was the #1 problem was an artifact of tooling bugs (raw counter + 15-AP sample cap) and was withdrawn after correction. Do not repeat that claim.
|
||||
- **6 GHz is nearly unused.** 75 radios active; only 1 client of 574. Largest untapped, clean, non-DFS capacity. Band-steering 6E-capable clients to 6 GHz is the highest-ROI tuning opportunity.
|
||||
- **AP-level satisfaction 95–100 fleet-wide.** Network is healthy on average; pain is in the client tail, consistent with "bad for SOME users" reports.
|
||||
- **Config flags (remediation pending):** 6 APs have 2.4 min-RSSI OFF (615, 608, 505, 517, 622, salon); 4 APs off the 1/6/11 channel plan on auto (128, 108, 108U7 Pro, salon); 2.4 TX power auto on ~75 radios.
|
||||
- **Known hardware issues:** AP 108 (Floor 1) offline pending a new cable run (per Howard — expected). Stale duplicate controller object ("108" vs "108U7 Pro") to clean up separately.
|
||||
- **Tuning plan (prioritized — NOT yet applied; all writes gated on per-zone live validation):**
|
||||
1. 2.4 GHz TX power → Low, per-zone (one floor at a time; live before/after cu_total + retry% validation via `live-stats.sh`).
|
||||
2. Steer 6E-capable clients to 6 GHz (band-steering config).
|
||||
3. 5 GHz: 80 → 40 MHz width; non-DFS channel plan (UNII-1 36–48 + UNII-3 149–161) for resilience against radar events.
|
||||
4. Min data rates: kill 1–11 Mbps; 2.4 GHz floor 12/24 Mbps.
|
||||
5. 2.4 min-RSSI −75/−76 on the 6 APs where it is currently OFF; pin 4 off-plan APs to ch 1/6/11.
|
||||
- AP radio disables deferred until an AP-to-AP RF-neighbor table is built (requires rogue BSSID cross-ref vs each AP's vap_table via Mongo). Until then, power/channel/width are the safe levers. Writes require the RW cred (`infrastructure/uos-server-network-api-rw`; vaulted 2026-06-15).
|
||||
- **Tooling:** `unifi-wifi` skill + `live-stats.sh` (Network API, Plane 2) = live validation path. `uos-mongo.sh` (Mongo, Plane 1) = config/interference path. Creds: `infrastructure/uos-server-ssh-key` (SSH/Mongo), `infrastructure/uos-server-network-api-rw` (RW API), `clients/cascades-tucson/unifi-ap-ssh` (per-AP SSH, needs site VPN for L3 reach to 192.168.2.x).
|
||||
- **Prior diagnostic (2026-05-16):** cloud API only, read-only; identified 2.4 GHz saturation hypothesis. Controller access was blocked at the time. Live controller access gained 2026-06-15.
|
||||
- **Tooling note:** `live-stats.sh` had accuracy bugs fixed 2026-06-15: removed 15-AP head cap (was hiding 62 of 77 APs), switched satisfaction to device-level (per-radio always −1), switched TX-retries to `tx_retries_pct` rate field, sorted worst-client list by satisfaction. These bugs caused a mid-session misdiagnosis that was corrected before the session ended.
|
||||
- **Fleet (full audit 2026-06-16):** 77 U7-Pro APs, **12 switches**, ~587 wireless clients. Controller: UOS at 172.16.3.29, HTTPS 11443 (see [[uos-server]]); site short name `va6iba3v`, site_id `685f39068e65331c46ef6dd2`. No UniFi gateway (pfSense is the gateway). pfSense ruled out as WiFi factor 2026-06-16 (DHCP not exhausted, DNS up, WAN stable -- see Network section).
|
||||
- **Primary pain band is 2.4 GHz.** Avg TX-retry ~10%; cu_total 69-94% live; catastrophic neighbor BSSID density (ch6 ~33k BSSIDs, ch1 ~19k, ch11 ~17k). 27 of the 40 worst clients stuck on 2.4 GHz (retry 11-42%), mostly IoT/legacy hardware (Ring cameras, robotic cleaner, smart plugs, EPSON printer, Poly phone, handheld scanners, smartwatch). Root cause: ~75 2.4 GHz radios running at auto (full) TX power in extreme density. Experience splits by band: 5/6 GHz clients are fine; clients that land or stick on 2.4 GHz suffer.
|
||||
- **5 GHz -- DFS concern is theoretical; empirically clean.** 76/77 radios on 80 MHz width (should be 40 MHz at this density). 55/77 radios on DFS channels (52-144) near Davis-Monthan AFB + TUS airport radar. `dfs-check.sh` 2026-06-16: **ZERO real radar events fleet-wide** (55 DFS APs, full `dmesg` sweep, precise pattern match) -- DFS is empirically low-risk here. Measured TX-retry DFS (8.4%) ~= non-DFS (9.0%) -- no throughput penalty. Still recommended to move to non-DFS (UNII-1 36-48 + UNII-3 149-161) for resilience. NOTE: an earlier mid-session claim (2026-06-15 audit) that "DFS was the #1 problem" was an artifact of tooling bugs (raw counter + 15-AP head cap) and was corrected before session end -- do not repeat it.
|
||||
- **6 GHz is nearly unused.** 75 radios active; only 1 client. Largest untapped, clean, non-DFS capacity. Band-steering 6E-capable clients to 6 GHz is the highest-ROI tuning opportunity.
|
||||
- **Switch audit (2026-06-16):** ~25 ports linked at 100 Mbps but gig-capable (systematic cabling/NIC issue, 1st/2nd/3rd-floor switches; investigate after WiFi Phase A). PoE budgets healthy. 3 offline switches: Switch 2nd Floor #2, Switch 4th Floor #2, USW Pro Max 16. Port p38 (1st Floor USW) 4.0% tx-drop rate.
|
||||
- **AP-level satisfaction 95-100 fleet-wide.** Network is healthy on average; pain is in the client tail.
|
||||
- **Remediation status (as of 2026-06-16 evening):**
|
||||
- **Phase A (2.4 power-down to Low): PARTIALLY APPLIED.** Floor-4 pilot applied 2026-06-16 (14/15 radios to 6 dBm from ~23; avg retry 13.2->9.5%, cu_total 86->83%, clients retained -- no coverage loss). AP 445 lagged (left alone, harmless). Remaining floors 1-3, 5-6 + floor-2/misc mesh APs = staged, pending go-ahead per zone. AP 128 is disabled (intentionally, re-disable after any zone apply restores it).
|
||||
- **Phase C (disable 9 redundant 2.4 radios): NOT applied.** Data-backed disable list (each has >=2 active-2.4 SNR neighbors): 127->128, 229->128, 248->348, 330->128, 445->347/348/247, 428->128, 622->505/615/608, Kitchen->Memcare TV room, Dining Room->memcare piano. Excludes mesh-protected APs (2nd Floor Atrium, CC Bridge, salon, 206 U7 Pro) and Memcare TV room. APs 445/428 disables held pending further validation.
|
||||
- **Deferred levers (separate session):** min-data-rate raise (1->12 Mbps), band-steering (`apply-wlan bandsteer`), 2.4 min-RSSI on the 6 OFF APs (615, 608, 505, 517, 622, salon), 5 GHz 80->40 MHz + non-DFS channel plan, 6 GHz band-steering.
|
||||
- **Config flags:** 6 APs with 2.4 min-RSSI OFF (615, 608, 505, 517, 622, salon); 4 APs off the 1/6/11 plan (128 disabled, 108 offline, 108U7 Pro auto, salon auto).
|
||||
- **Mesh topology:** 2nd Floor Atrium is wireless-mesh parent for CC Bridge + salon (5 GHz backhaul ch36); 206 U7 Pro carries AP 108. These must NEVER be disabled or powered down via zone command -- coverage-thin auto-excludes them.
|
||||
- **Known hardware:** AP 108 (Floor 1) offline pending a new cable run (expected). Stale duplicate controller object ("108" vs "108U7 Pro") to clean up separately.
|
||||
- **AP-hang recovery:** use `device-control.sh cascades poe-cycle "<AP name>" --apply` (remote PoE port cycle via controller cmd/devmgr). Do NOT use `force-provision` -- it took AP 445 offline during the Floor-4 pilot and was removed from device-control.sh.
|
||||
- **Tooling (`unifi-wifi` skill -- feature-complete as of 2026-06-16):**
|
||||
- Collectors: `audit-site.sh` (config + neighbor density), `live-stats.sh` (live per-AP/client, Plane 2), `model-rank.sh`, `radio-usage.sh` (77-day 2.4 usage history per AP; confirms POWER-DOWN vs disable), `coverage-thin.sh` (mesh-aware 2.4 SNR dominating-set -- drives Phase C), `neighbor-collect.sh` (/proc/ui_neighbor AP-to-AP SNR matrix, non-disruptive, drives optimize-radios disables), `survey-collect.sh` (per-channel busy%/noise -> channel plan), `dfs-check.sh` (precise per-AP radar event history), `switch-audit.sh`, `gw-audit.sh`, `monitor-run.sh` (cron health digest, all sites), `sites.sh` (multi-client site list, ~49 UOS sites).
|
||||
- Apply (gated + rollback): `apply-radio.sh` (power/width/channel/minrssi/disable/enable, --zone/--ap), `apply-wlan.sh` (minrate/bandsteer/bands/steer/bsstm/dtim/isolation/etc.), `client-control.sh` (block/unblock/kick MAC), `device-control.sh` (poe-cycle; adopt/restart/locate/upgrade), `channel-plan.sh` (data-driven 2.4/5 GHz channel plan via neighbor + survey data).
|
||||
- pfSense: `pfsense-ssh.sh` (audit/dhcp/run -- SSH backend, no RESTAPI package needed; auth from `clients/<slug>/pfsense-firewall`; system OpenSSH via askpass). ROADMAP: gated control verbs (firewall rules, port forwards) -- deferred to Mike per SS E.
|
||||
- All scripts site-parameterized (work for any of ~49 UOS sites). Per-client AP-side creds via `clients/<slug>/unifi-ap-ssh`.
|
||||
- **Creds (vault refs only):** `infrastructure/uos-server-ssh-key` (SSH/Mongo), `infrastructure/uos-server-network-api-rw` (RW API), `clients/cascades-tucson/unifi-ap-ssh` (per-AP SSH, needs site VPN for L3 reach to 192.168.2.x/3.x), `clients/cascades-tucson/pfsense-firewall` (pfSense admin for pfsense-ssh.sh).
|
||||
- **Prior diagnostic (2026-05-16):** cloud API only, read-only; identified 2.4 GHz saturation hypothesis. Controller access was blocked at the time. Live controller access gained 2026-06-15 when Mike vaulted the SSH key and RW admin.
|
||||
- **Tooling note:** `live-stats.sh` accuracy bugs fixed 2026-06-15 (removed 15-AP head cap, switched satisfaction to device-level, switched TX-retries to `tx_retries_pct` rate field, sorted worst-client list by satisfaction). These bugs caused a mid-session misdiagnosis that was corrected before session end.
|
||||
|
||||
### Known Issues / Pending Hygiene (as of 2026-06-04)
|
||||
### Known Issues / Pending Hygiene (as of 2026-06-16)
|
||||
|
||||
- **[BUG] Stale exclude-group on MFA-all-users policy:** The `Require multifactor authentication for all users` policy (`7e87a1c7…`) currently excludes `SG-Caregivers-Pilot` (`0674f0bc…`) instead of the live `SG-Caregivers` (`8b8d9222…`). Functionally harmless today (pilot group still exists), but this is a known bug that must be corrected. Fix: PATCH `excludeGroups` to replace `SG-Caregivers-Pilot` with `SG-Caregivers`.
|
||||
- **[DESIGN] ALIS-native 2FA is not a perimeter control.** The `Require MFA for all users` policy excludes `AllTrusted` locations, so Entra never prompts on the Cascades network. A non-SSO ALIS user can reach ALIS from anywhere with only ALIS credentials — Entra never sees that login. The correct permanent model: force all ALIS logins through Entra SSO (SSO-only, credential fallback disabled), so Entra enforces onsite-seamless / offsite-MFA. Office/privileged users should be standardized onto ALIS SSO as a separate workstream; ALIS-native 2FA should then be disabled per-user then globally.
|
||||
- **[INFO] Android enrollment token expiry (2027-05-08) does NOT unenroll devices.** The `CSC - Android Shared Phones (Entra SDM)` enrollment token (`9a0fcc6d`) is a join key only. Existing enrolled devices (25 as of 2026-06-03) are unaffected by token expiry. Renewal is needed only before enrolling new devices after that date.
|
||||
- **[INFO] Chris Knight bill.com/BOK Financial emails (2026-06-04):** Zero bill.com or BOK Financial emails ever delivered to chris.knight@ or c.knight@ in 90 days. bill.com confirmed delivering to other Cascades users (no tenant-wide block). Root cause: bill.com and BOK Financial backends likely still have Chris Knight's old email address. Resolved externally by Howard. No tenant config changes needed.
|
||||
- **[BUG] Stale exclude-group on MFA-all-users policy:** The `Require multifactor authentication for all users` policy (`7e87a1c7...`) excludes `SG-Caregivers-Pilot` (`0674f0bc...`) instead of the live `SG-Caregivers` (`8b8d9222...`). Fix: PATCH `excludeGroups` to replace `SG-Caregivers-Pilot` with `SG-Caregivers`.
|
||||
- **[DESIGN] ALIS-native 2FA is not a perimeter control.** The correct permanent model: force all ALIS logins through Entra SSO (SSO-only, credential fallback disabled). Office/privileged users should be standardized onto ALIS SSO as a separate workstream; ALIS-native 2FA should then be disabled per-user then globally.
|
||||
- **[INFO] Android enrollment token expiry (2027-05-08) does NOT unenroll devices.** Renewal is needed only before enrolling new devices after that date.
|
||||
- **[WARN] ~25 switch ports at 100 Mbps but gig-capable.** Physical: re-terminate/replace cable or check NIC. Investigate after WiFi Phase A remediation is stable.
|
||||
- **[WARN] 3 offline switches** (Switch 2nd Floor #2, Switch 4th Floor #2, USW Pro Max 16). Root cause unknown; investigate onsite.
|
||||
|
||||
### Security Incidents (historical)
|
||||
|
||||
- **Megan Hiatt (2026-04-16):** Active credential-stuffing — 126 failed sign-ins, bursts from Belfast GB, Hamburg DE. Password reset and SMTP AUTH disable were action items. Mailbox was clean (not breached).
|
||||
- **John Trozzi (2026-04-16, 2026-04-20):** Investigated twice — both times NO BREACH. First: credential stuffing flag (clean). Second: inbound phishing email (clean). Reports in `clients/cascades-tucson/reports/`.
|
||||
- **Megan Hiatt (2026-04-16):** Active credential-stuffing -- 126 failed sign-ins, bursts from Belfast GB, Hamburg DE. Password reset and SMTP AUTH disable were action items. Mailbox was clean (not breached).
|
||||
- **John Trozzi (2026-04-16, 2026-04-20):** Investigated twice -- both times NO BREACH. First: credential stuffing flag (clean). Second: inbound phishing email (clean). Reports in `clients/cascades-tucson/reports/`.
|
||||
- **Crystal Rodriguez (2026-04-19):** Phishing investigation. Report: `clients/cascades-tucson/reports/2026-04-19-crystal-rodriguez-phish-investigation.md`.
|
||||
- **Canva email delivery (2026-05-20):** Alma Montt not receiving Canva invites. Resolved by adding canva.com domains to AllowedSenderDomains in EOP policies.
|
||||
- **ALIS AADSTS65001 (2026-06-03):** megan.hiatt, karen.rossini, memcarereceptionist could not sign in to ALIS on non-phone devices. Root cause: missing tenant-wide admin consent on ALIS SP (`e1cae4ad`). Resolved by granting `AllPrincipals` `User.Read` via Graph API. CA was NOT the cause — all failures showed `conditionalAccessStatus: success` from trusted IPs.
|
||||
- **dunedolly21@gmail.com:** External guest invited 2026-04-14 by Lauren Hasselman from mobile. Status unknown — confirm with Lauren. [unverified]
|
||||
- **Chris Knight bill.com / BOK email delivery (2026-06-04):** `chris.knight@cascadestucson.com` (alias: `c.knight@cascadestucson.com`) not receiving bill.com or BOK Financial emails. M365 mailbox confirmed healthy: 24 inbound messages traced over prior 48h, no inbox rules, no forwarding, no junk/quarantine hits, no transport rules or connectors blocking. Root cause: SENDER-SIDE, not M365. bill.com sends via SendGrid (`inform.bill.com`); the address was on SendGrid's ESP suppression list — mail dropped before SMTP, so nothing appeared in message trace and repeated resends never arrived. BOK diagnosis confirmed: correcting the email in BOK's portal produced a "Welcome to Exchange!" delivery from `alerts@exchange.bokfinancial.com` within minutes. **bill.com fix requires calling bill.com support** — the account email cannot be changed in the web UI (it is the locked login identity); support must update it AND clear the SendGrid suppression. Ticket #32383, 1.5h remote.
|
||||
- **ALIS AADSTS65001 (2026-06-03):** megan.hiatt, karen.rossini, memcarereceptionist could not sign in to ALIS on non-phone devices. Root cause: missing tenant-wide admin consent on ALIS SP (`e1cae4ad`). Resolved by granting `AllPrincipals` `User.Read` via Graph API.
|
||||
- **dunedolly21@gmail.com:** External guest invited 2026-04-14 by Lauren Hasselman from mobile. Status unknown -- confirm with Lauren. [unverified]
|
||||
- **Chris Knight bill.com / BOK email delivery (2026-06-04):** Root cause was SENDER-SIDE: bill.com address on SendGrid suppression list; BOK had wrong recipient email. Resolved externally by Howard. No tenant config changes needed. Ticket #32383, Resolved.
|
||||
|
||||
### HIPAA Compliance
|
||||
|
||||
- **Primary objective.** Cascades stores PHI on CS-SERVER and uses ALIS for clinical records.
|
||||
- **Critical open gaps:** No backup (§164.308(a)(7)); no audit logging on D:\Homes (§164.312(b)); Object Access auditing disabled; no SMB encryption on homes share; no file access auditing.
|
||||
- **Restored 7 deleted mailboxes (2026-04-25)** for HIPAA §164.316(b)(2) 7-year retention.
|
||||
- **Critical open gaps:** No audit logging on D:\Homes (SS164.312(b)); Object Access auditing disabled; no SMB encryption on homes share; no file access auditing. Audit retention infra (LAW 90d + Storage 6yr) approved but not yet built.
|
||||
- **Backup gap closed (2026-06-15):** Mike installed ACG cloud backup (MSP360/CloudBerry -> ACG-backup server) on CS-SERVER. Verify first full backup completes and set retention; confirm image-based / bare-metal + system-state for DC recoverability.
|
||||
- **Restored 7 deleted mailboxes (2026-04-25)** for HIPAA SS164.316(b)(2) 7-year retention.
|
||||
- **Termination policy established:** Convert to shared mailbox, hide from GAL, retain 7 years.
|
||||
|
||||
---
|
||||
|
||||
## Active Work
|
||||
|
||||
Primary active project as of 2026-05-24: dept-by-dept domain migration (Syncro #110680053). Syncro live pull 2026-06-15: **0 open tickets** (was one real open ticket — #32370 eFax/scanner onsite — as of 2026-06-13; verify/likely closed).
|
||||
Primary active project as of 2026-05-24: dept-by-dept domain migration (Syncro #110680053). Syncro live pull 2026-06-16: **0 open tickets.**
|
||||
|
||||
**Migration phase status (as of 2026-05-26):**
|
||||
|
||||
@@ -397,29 +371,33 @@ Primary active project as of 2026-05-24: dept-by-dept domain migration (Syncro #
|
||||
| RECEPTIONIST-PC (frontdesk) | Domain-joined 2026-05-22; loopback Replace mode, no folder redirect by design |
|
||||
| NURSESTATION-PC | Domain-joined, folder redirect complete |
|
||||
| Lauren Hasselman | Domain-joined, folder redirect complete 2026-05-23 |
|
||||
| Megan Hiatt (Marketing) | COMPLETE 2026-05-27 — domain joined via ProfWiz, folder redirection live, data on server |
|
||||
| DESKTOP-KQSL232 (Lois Lane — CareTakers) | Blocked — Lois Lane resistant to change; John Trozzi working with her |
|
||||
| Megan Hiatt (Marketing) | COMPLETE 2026-05-27 -- domain joined via ProfWiz, folder redirection live, data on server |
|
||||
| DESKTOP-KQSL232 (Lois Lane -- CareTakers) | Blocked -- Lois Lane resistant to change; John Trozzi working with her |
|
||||
| CHEF-PC, SALES4-PC, MDIRECTOR-PC | Not yet started |
|
||||
|
||||
**Blocking issues / pending:**
|
||||
- M365 relicensing: 31 Business Standard → Business Premium (SUSPENDED — time-critical, 31 SPB seats free)
|
||||
- Break-glass accounts: not created (confirmed 2026-05-27)
|
||||
- Audit retention infra: not built
|
||||
- M365 relicensing: 31 Business Standard -> Business Premium (SUSPENDED -- time-critical, 31 SPB seats free)
|
||||
- Break-glass accounts: not created (confirmed 2026-05-27); YubiKey arrival unconfirmed
|
||||
- Audit retention infra: approved 2026-04-29, not yet built
|
||||
- RECEPTIONIST-PC GuruRMM agent (9c91d324): flaky WebSocket, lagging fleet
|
||||
- Entra Connect: OU=Administrative not yet in sync scope; UPN suffix updates for that OU pending
|
||||
- NURSESTATION-PC: reboot required to activate `CSC - Caregiver Device Lockdown` GPO (deployed 2026-06-05, linked to `OU=Caregiver Devices`; startup script runs at boot — verify lock@3min, 90s warning, sign-out@15min, never-sleep)
|
||||
- #32370 — eFax/scanner onsite (Howard); verify/likely closed (Syncro live 2026-06-15 shows 0 open; was confirmed [New]/open 2026-06-13). No appointment scheduled as of 2026-06-02.
|
||||
- NURSESTATION-PC: reboot required to activate `CSC - Caregiver Device Lockdown` GPO (deployed 2026-06-05; verify lock@3min, 90s warning, sign-out@15min, never-sleep)
|
||||
- #32370 -- eFax/scanner onsite (Howard); verify/likely closed (Syncro live 2026-06-16 shows 0 open)
|
||||
- Caregiver device allow-list: ASSISTNURSE-PC needs re-join + re-tag after Win11 reinstall; LAPTOP-8P7HDSEI Win11 upgrade + join/tag still pending; then cutover (enable allow-list policy, disable compliance-block)
|
||||
- ALIS office/privileged standardization: move office/managers/nurses to ALIS SSO-only; disable ALIS-native 2FA per-user then globally (separate workstream)
|
||||
- Fix stale `SG-Caregivers-Pilot` exclude-group on `Require MFA for all users` policy (known bug, see Known Issues)
|
||||
- LAPTOP-8P7HDSEI: upgrade Win 10 → Win 11 before PHI use
|
||||
- Chris Knight bill.com/BOK Financial addresses: confirm updated in bill.com backend and at BOK Financial (resolved externally 2026-06-04 but no confirmation of actual address update on vendor side)
|
||||
- Edge UNC download bug (Chromium 149): decide fix path for Ashley Jensen + Lois Lane and fleet (see Patterns -> Browser / Edge); no fix applied as of 2026-06-08
|
||||
- ALIS app session timeout: lower from 20 to 15 min (Howard, ALIS admin) — PENDING
|
||||
- **Wireless RF tuning (staged, no changes applied as of 2026-06-15):** 2.4 GHz TX power → Low per-zone (Floor 4 pilot first; live cu_total + retry% before/after validation); 6 GHz band-steering for capable clients; 5 GHz 80→40 MHz + non-DFS channel plan (UNII-1+UNII-3); min data rates; min-RSSI + channel-plan fixes on 6 flagged APs. Gated: build AP-to-AP RF-neighbor table before any AP disables; pull radar-detection event history to confirm DFS avoidance need; site VPN `.ovpn` needed for `watch-ap.sh` live stream (pfSense OpenVPN Client Export).
|
||||
- **[CRITICAL] CS-SERVER degraded RAID-1 (2026-06-15):** OS mirror (C:) running on a single 320 GB laptop spindle, no redundancy — root cause of "server slow". Plan SSD rebuild-then-swap (image C: first); DC migration is the real fix. Cloud backup now installed/started — verify first full completes + set retention.
|
||||
- **[CLEANUP] CS-SERVER agent sprawl:** remove the previous MSP's leftover Datto RMM + Datto EDR/Infocyte stack (thrashing the degraded disk atop Syncro/GuruRMM/ScreenConnect/KPAX).
|
||||
- **[PLANNED] Voice VLAN (VLAN 30) for Vertical phones + remote desktop:** vendor email sent 2026-06-16; awaiting Richard's confirm (cloud-PBX, desktop static, VPN cert CN) + maintenance window, then execute. Runbook: `clients/cascades-tucson/docs/network/voice-vlan-cutover.md`.
|
||||
- ALIS office/privileged standardization: move office/managers/nurses to ALIS SSO-only; disable ALIS-native 2FA per-user then globally
|
||||
- Fix stale `SG-Caregivers-Pilot` exclude-group on `Require MFA for all users` policy
|
||||
- LAPTOP-8P7HDSEI: upgrade Win 10 -> Win 11 before PHI use
|
||||
- Edge UNC download bug (Chromium 149): decide fix path for Ashley Jensen + Lois Lane and fleet; no fix applied as of 2026-06-08
|
||||
- ALIS app session timeout: lower from 20 to 15 min (Howard, ALIS admin) -- PENDING
|
||||
- **[CRITICAL] CS-SERVER degraded RAID-1 (2026-06-15):** OS mirror (C:) running on a single 320 GB laptop spindle, no redundancy. Plan SSD rebuild-then-swap (image C: first, AFTER backup verifies). DC migration is the real fix. Cloud backup installed/started 2026-06-15 -- **verify first full completes + confirm image-based + set retention before any drive work.**
|
||||
- **[CLEANUP] CS-SERVER agent sprawl:** remove the previous MSP's leftover Datto RMM (CentraStage) + Datto EDR (Infocyte) stack (thrashing the degraded disk).
|
||||
- **[PLANNED] Voice VLAN (VLAN 30) for Vertical phones + remote desktop:** vendor email sent 2026-06-16, awaiting Richard Turner's confirm (cloud-PBX confirmed via recon, desktop static, VPN cert CN) + maintenance window, then execute. Runbook: `clients/cascades-tucson/docs/network/voice-vlan-cutover.md`.
|
||||
- **[IN PROGRESS] Wireless RF remediation (2.4 GHz):**
|
||||
- Phase A (power-down to Low): Floor-4 pilot APPLIED 2026-06-16 (retry 13.2->9.5%, no coverage loss). Remaining floors (1-3, 5-6 + floor-2/misc per-AP) = staged, awaiting go-ahead. Runbook: `clients/cascades-tucson/reports/2026-06-16-2.4ghz-remediation-runbook.md`.
|
||||
- Phase C (disable 9 redundant 2.4 radios): staged, awaiting Phase A validation + explicit go-ahead. APs 445/428 disables held; AP 128 disabled.
|
||||
- Deferred: min-data-rate, band-steering, 2.4 min-RSSI, 5 GHz 80->40 MHz + non-DFS, 6 GHz steering.
|
||||
- pfSense Phase A / gated controls: pfSense SSH backend (pfsense-ssh.sh) live 2026-06-16; firewall control verbs deferred to Mike (ROADMAP SS E).
|
||||
- **[VERIFY] ~25 switch ports at 100 Mbps but gig-capable** (switch-audit.sh 2026-06-16): systematic cabling/NIC issue. Investigate after WiFi Phase A stable.
|
||||
|
||||
---
|
||||
|
||||
@@ -439,64 +417,65 @@ Primary active project as of 2026-05-24: dept-by-dept domain migration (Syncro #
|
||||
| 2026-04-30 | CA rollout (Report-only mode): 3 caregiver policies created. SDM bootstrap. |
|
||||
| 2026-05-01 | Howard billed 33.5 hrs against prepaid block on Entra project ticket #32214 ($0 invoice). |
|
||||
| 2026-05-07-08 | SDM phone provisioning. SDM token success. ALIS SSO app registration values captured to vault. |
|
||||
| 2026-05-14-16 | Caregiver AD accounts created. Security groups always deliberate (no OU→group automation). Wireless diagnostic (read-only via cloud API; 2.4 GHz saturation hypothesis identified; local controller inaccessible at the time). |
|
||||
| 2026-05-14-16 | Caregiver AD accounts created. Security groups always deliberate (no OU->group automation). Wireless diagnostic (read-only via cloud API; 2.4 GHz saturation hypothesis identified; local controller inaccessible at the time). |
|
||||
| 2026-05-18 | Billing review. 39.5 hrs remaining before session. 7 hrs billed separately. |
|
||||
| 2026-05-20 | Canva email delivery resolved (canva.com domains added to EOP). |
|
||||
| 2026-05-21 | Crystal Rodriguez folder redirect confirmed working. Lauren Hasselman + Crystal Rodriguez domain join attempted — passwords didn't work initially. |
|
||||
| 2026-05-21 | Crystal Rodriguez folder redirect confirmed working. Lauren Hasselman + Crystal Rodriguez domain join attempted -- passwords didn't work initially. |
|
||||
| 2026-05-22 | Ashley Jensen domain-joined. RECEPTIONIST-PC domain-joined. GPO ILT fixes (FrontDesk printer + R: drive). cascadesDS auth failure diagnosed (workgroup collision) and deferred. |
|
||||
| 2026-05-14 | Entra Connect exited staging mode — actively syncing. CA pilot re-pointed to SG-Caregivers. |
|
||||
| 2026-05-14 | Entra Connect exited staging mode -- actively syncing. CA pilot re-pointed to SG-Caregivers. |
|
||||
| 2026-05-23 | Lauren Hasselman folder redirect complete. Megan Hiatt (Marketing) confirmed in AD, domain join pending. |
|
||||
| 2026-05-24 | RECEPTIONIST-PC GuruRMM agent noted as 0.6.37 straggler while fleet at 0.6.38. Flaky WebSocket. |
|
||||
| 2026-05-26 | Access control vendor meeting onsite (ticket #32324). 0.5h Howard + 0.5h Mike billed against prepaid block. Block at 28.0h. Remote diagnosis of UniFi controller confirmed impossible (no Tailscale route, GuruRMM WebSocket-only, pfSense SSH blocked). |
|
||||
| 2026-06-03 | ALIS AADSTS65001 diagnosed and resolved: granted tenant-wide admin consent (`AllPrincipals` `User.Read`) on ALIS SP `e1cae4ad`. Caregiver device allow-list CA policy created in report-only (`CSC - Caregivers: allow-listed devices only (REPORT-ONLY)`, id `1b7fd025`). Allow-list = CSC- phones + 5 tagged devices (NURSESTATION-PC, Laptop2, LAPTOP-8P7HDSEI, LAPTOP-DRQ5L558, LAPTOP-E0STJJE8). Cutover pending laptop Intune enrollment + validation. Three existing enforced caregiver CA policies left untouched. |
|
||||
| 2026-06-04 | Three same-day tickets: #32381 Tamra scanner (0.5h onsite), #32382 Megan file access (1.5h onsite), #32383 Chris Knight bill.com/BOK email delivery (1.5h remote). Chris Knight mailbox investigation: full EXO/EOP/quarantine/message trace analysis — no tenant config issues found. No Inky in tenant (confirmed). bill.com delivering to other users; zero delivery to chris.knight/c.knight in 90 days. Root cause: wrong address in bill.com/BOK backends + SendGrid suppression on bill.com side. BOK resolved by correcting email in portal (delivery within minutes). bill.com fix requires support call. Resolved externally by Howard; no tenant config changes needed. EXO access token auth method documented (cert not in BEAST cert store). Prepay block: 17.25 → 15.75 hrs. |
|
||||
| 2026-06-05 | NURSESTATION-PC localadmin login-screen issue: `SpecialAccounts\UserList` hide (`localadmin=0`) — removed via RMM (agent `f5a89784`); account was already enabled + admin. Vault hygiene: `sysadmin@` GA password vaulted (`clients/cascades-tucson/m365-sysadmin.sops.yaml`); voice MFA scoped group "MFA - Voice Call Scoped (sysadmin)" (`304f941e`) created; `alternateMobile` updated to +1 520-585-1310 (Howard). Caregiver test rig built: `SG-Caregivers-DeviceTest` (`db5849ec`, full rule set), `Cascades - Caregiver Devices` (`02c6f698`, static), `SG-Intune-Enrollment` (`13d94f6e`), `pilot.test@cascadestucson.com` (`d26e0e5a`, ephemeral). Hybrid Entra Join enabled in Entra Connect (SCP `ConfigureSCP.ps1`; `OU=Caregiver Devices` added to sync scope). NURSESTATION re-domain-joined (Win11 25H2) + hybrid-registered as `trustType: ServerAd`, new deviceId `d3bf931f-f128-4261-8398-b46c34a4b342` (object `de199a15`). Caregiver access model proven end-to-end on desktop: pilot.test + NURSESTATION — ALIS via silent SSO, CA off-network block + device allow-list holding. CA 53003 on `extensionAttribute1` tag lag (>70 min); resolved by adding deviceId directly to allow-list rule (immediate). Windows Hello does NOT auto-provision on hybrid-joined machines (`WillNotProvision: PolicyEnabled NO`). GPO `CSC - Caregiver Workstation` (`{3B5CD9A6-A278-4676-A9FD-9396D21A8261}`, User config GPP): 3 desktop shortcuts (ALIS, LinkRx, Helpany) + 6 `\\CS-SERVER\` printers with location-based default (Nurses for `SG-PC-MainTower`, MCMedTech for `SG-PC-MemoryCare`, computer-context ILT) + `LegacyDefaultPrinterMode=1` — built, linked at `OU=Caregivers`, security-filtered to `SG-Caregivers-Test` (pilot.test only), validated on NURSESTATION. GPO `CSC - Caregiver Device Lockdown` (`{E6174988-2721-4D96-ADF5-F5BB44E92769}`, computer-only): startup script (lock 3 min / auto sign-out 15 min / 90s warning / never sleep) + psscripts.ini in SYSVOL — deployed + linked at `OU=Caregiver Devices` (takes effect on next NURSESTATION reboot). Intune enrollment blocked tenant-wide (`INTUNE_A: PendingInput` on newly-licensed accounts); MS case open; GPO path used instead. Ticket #32303 billing reconciliation: work summary posted as customer-visible resolution note (comment 417582473); 7.0h onsite line item (42750851) + invoice #67782 ($0.00 prepaid); prepay block 15.75 → 8.75 hrs; ticket status → Invoiced. |
|
||||
| 2026-06-08 | **Chris Knight workstation setup (onsite).** Discovered his AD account `chris.knight` already existed (created 2026-05-27, OU=Administrative) but was incomplete; finished it to match Lauren Hasselman — `New-HomeFolder`, added to `SG-FolderRedirect`, set `mail`, reset AD password to `Cascades2026!` (change-at-logon cleared). Confirmed mailbox is cloud-only/unsynced (so are Lauren/Ashley/Meredith/Zachary/Alma — Entra Connect include-list is Caregivers+Groups+Caregiver Devices only; OU=Administrative NOT in scope). Machine **DESKTOP-N5G1ROO** domain-joined + GuruRMM-enrolled (agent `205025ee...`), Office installed, Chris logged in. **MAJOR: root-caused why folder redirection has failed on every machine** — the FR GPO's targets were in a misnamed `fdeploy1.ini`; Windows reads `fdeploy.ini` (absent) → empty path → silent no-op → manual registry workaround every time. Fixed by writing a correct `fdeploy.ini` to GPO `{512B43A4}` + version bump 917506→983042 (GPT.INI + AD versionNumber); backup at `C:\Windows\Temp\frfix-20260608-161144`. LE GPO found completely empty too. CS-SERVER live RMM agent is now `c39f1de7-...` (was `6766e973`). Billed 1.0h onsite (computer setup, ticket #111216087). |
|
||||
| 2026-06-08 | **ASSISTNURSE-PC reinstalled (Win10→Win11).** Howard did a clean Windows 11 install (machine was Win10 19045; in-place upgrade attempts failed, clean install the only option) using our key, then reinstalled the RMM agent. Claude (RMM): deleted the stale pre-reinstall agent `88891eb8` (Win10, offline) — HTTP 204; kept the new agent `62d108d6` (`Assistnurse-pc`, Win11 Pro for Workstations 24H2, v0.6.57, online). Deployed 3 caregiver app shortcuts as `.url` files to `C:\Users\Public\Desktop` (machine-wide) matching the team's GPP definitions: ALIS `https://cascadestucson.alisonline.com/Login`, LinkRx `https://pharmcare.linkrxnow.com/Login.aspx`, Helpany `https://app.safe-living.com/login`. Heads-up: reinstall = new Entra device object → needs re-join + re-tag `CSCCaregiverDevice` (+ clean old Entra record) at caregiver cutover. Billing for the 1.0h onsite reinstall: billed on #32303 (drew 57.75→56.75; implied by subsequent balance chain). |
|
||||
| 2026-06-08 | **Edge UNC download bug diagnosed (no fix applied).** Ashley Jensen (DESKTOP-U2DHAP0) and Lois Lane (DESKTOP-KQSL232) both on Edge 149.0.4022.52 could not open Office files (.xlsx, .docx) from the Edge download panel when Downloads is redirected via folder redirection to `\\cs-server\homes\<user>\Downloads`. Root cause: Chromium 149 regression (issue 519243472) in `LaunchShellExecuteViaExplorer` — prepends `\\?\` to UNC paths without converting to `\\?\UNC\`, producing malformed paths. PDF and text files unaffected (different launch path). Fix options documented in Patterns section; fix path decision left to Howard. Fleet-wide exposure for any Cascades user with Downloads folder-redirected to the Homes share on Edge 149. |
|
||||
| 2026-06-09 | **Accounting scan-to-folder built + billing reconciliation.** Created `D:\Shares\Accounting` + `\Scans` on CS-SERVER (NTFS locked to `lauren.hasselman`/`chris.knight`/`zachary.nelson` = Modify, no Everyone; `svc-scan` = Modify on `\Scans` only), shared as `\\CS-SERVER\AcctDept` (named AcctDept because a Canon MF455DW *printer* share already owns "Accounting" — restored that share after a grant collision). New vaulted AD service account `svc-scan` for the Brother's SMB auth. Brother MFC-L8900CDW (10.0.20.220) Scan-to-Network profile → `\\192.168.2.254\AcctDept\Scans` (NTLMv2, `cascades\svc-scan`); **test scan confirmed**. Found pfSense blocks main-LAN→VLAN-20 (can't reach VLAN-20 printer WBM from CS-SERVER; printer→server:445 open). Persistent drive maps to the share: Chris (Y:), Zachary on ACCT2-PC (Y:), Lauren (X:). Also reconciled crashed-session billing: #32330 (Chris Knight computer) was already invoiced (#67790) — fixed status Resolved→Invoiced; live prepay confirmed **57.75h** (prior 7.75 was pre-top-up). Updated machine inventory (ASSISTNURSE-PC reinstall, caregiver device table) in this wiki. |
|
||||
| 2026-06-10 | **Meredith Kuhn locked Word doc — stale owner files on cascadesDS.** Five orphaned Word `~$` owner files dated 2024 in `\\cascadesds\Public\Company Web Docs\Staff Trainings\` caused false "locked for editing" messages on training documents with no active session. Diagnosed and deleted all 5 via RMM in Meredith's `user_session` on ASSISTMAN-PC (agent `cf86fa5e`) — CS-SERVER SYSTEM cannot authenticate to cascadesDS (workgroup/Kerberos mismatch). Howard's post-reboot check on the Synology confirmed no live handles. Ticket #32403 (id 112502876), 0.5h remote, invoice $0.00 prepaid, block 56.75→56.25. |
|
||||
| 2026-06-12 | **Created shared mailboxes grievances@ + Surveys@ and delegated to Meredith & Ashley.** `grievances@cascadestucson.com` and `Surveys@cascadestucson.com` created as SharedMailbox (cloud-only, no license consumed), each delegated to Meredith Kuhn and Ashley Jensen with FullAccess (auto-mapping) + SendAs. Work done via ComputerGuru Exchange Operator MSP app cert auth (EXO module v3.10.0 installed on Howard-Home for this session). All 8 permission grants verified post-creation. Ticket #32417 (id 112597225), 0.5h remote, invoice #1650665832 $0.00 prepaid, block 56.25→55.75; ticket Invoiced. |
|
||||
| 2026-06-15 | **Wireless RF full audit — controller access gained.** Mike vaulted `infrastructure/uos-server-ssh-key` + `clients/cascades-tucson/unifi-ap-ssh`; `unifi-wifi` skill used end-to-end. Live audit via UOS Mongo (Plane 1) confirmed 77 U7-Pro APs, 574 clients, 2.4 GHz saturation as primary pain band (avg retry 11.2%, cu_total 69–94%, catastrophic neighbor density). Accuracy bugs in `live-stats.sh` found and fixed mid-session (15-AP head cap, wrong satisfaction/retry fields) — corrected the data and corrected a mid-session misdiagnosis that DFS was the #1 problem (withdrawn; DFS retry rate 8.4% ≈ non-DFS 9.0%). Mike also vaulted `infrastructure/uos-server-network-api-rw` (RW controller admin) same day; Plane 2 (Network API) re-audited and confirmed findings. DFS designated a resilience concern (near Davis-Monthan AFB + TUS radar), not a throughput concern. 6 GHz (1 client of 574) identified as largest untapped capacity. Tuning plan staged (see Patterns -> Wireless / UniFi RF); no changes applied. |
|
||||
| 2026-06-15 | **CS-SERVER slowness root-caused to a degraded RAID-1; backup started; OpenVPN password reset.** "CS-SERVER slow / check for infections" -> not RAM/CPU/disk (48 GB RAM ~72% free, 10-day uptime, clean infection sweep). Dell OMSA: PD 0:0:3 (320 GB WD SATA) Critical/Removed, Virtual Disk2 (C: mirror) Non-Critical/Degraded -> C: on a single 320 GB Hitachi 5400 spindle, no redundancy (root cause of slowness); 1.2 TB SAS "Ready" disk is the wrong size to rebuild. Found leftover Datto RMM + Datto EDR/Infocyte; CS-SERVER not in Bitdefender. Mike installed MSP360/CloudBerry cloud backup and started it (closes the no-backup HIPAA gap). Reset Howard's lost pfSense OpenVPN password (local-DB user `Howard`, userid 0) via `local_user_set_password()` PHP-exec driven from CS-SERVER over RMM (CS-SERVER reaches 192.168.0.1:443/22); verified AUTHOK and vaulted. |
|
||||
| 2026-06-16 | **Voice VLAN plan for Vertical phones (PLANNED, not executed).** Vertical's tech (Richard Turner) couldn't reach phones from the remote desktop (192.168.2.180) and phone IPs drift. UOS controller diagnosis: Poly phones (22, `48:25:67`) on WiFi/CSCNet PPSK -> VLAN 20; AudioCodes (8, `00:90:8f`) wired USW-16-PoE ports 1-8 on Default LAN; Vertical desktop wired port 16 on Default, static, no ACG login. CSCNet found to be a shared PPSK SSID (corrected the old "staff/VLAN 20" note). GuruRMM recon from CS-SERVER: desktop = RDP-only (not a PBX); CS-QB (192.168.2.228) = SMB-only, no SIP -> phones likely cloud PBX. Designed dedicated VLAN 30 VOICE (10.0.30.0/24) for all phones + the desktop (internet-only egress, isolated from VLAN 20/main LAN/PHI, OpenVPN scoped via CSO); wrote the cutover runbook (`docs/network/voice-vlan-cutover.md`); Howard sent the vendor email. Awaiting confirm + window. |
|
||||
| 2026-05-26 | Access control vendor meeting onsite (ticket #32324). 0.5h Howard + 0.5h Mike billed against prepaid block. Block at 28.0h. |
|
||||
| 2026-06-03 | ALIS AADSTS65001 diagnosed and resolved: granted tenant-wide admin consent on ALIS SP `e1cae4ad`. Caregiver device allow-list CA policy created in report-only (`CSC - Caregivers: allow-listed devices only (REPORT-ONLY)`, id `1b7fd025`). |
|
||||
| 2026-06-04 | Three same-day tickets: #32381 Tamra scanner (0.5h onsite), #32382 Megan file access (1.5h onsite), #32383 Chris Knight bill.com/BOK email delivery (1.5h remote). Root cause sender-side. EXO access token auth method documented. |
|
||||
| 2026-06-05 | NURSESTATION-PC localadmin login-screen issue (`SpecialAccounts\UserList` hide) -- removed via RMM. Vault hygiene: `sysadmin@` GA password vaulted; voice MFA scoped group created; `alternateMobile` updated to +1 520-585-1310 (Howard). Caregiver test rig built. Hybrid Entra Join enabled; NURSESTATION re-domain-joined + hybrid-registered (new deviceId `d3bf931f`). Caregiver access model proven end-to-end: pilot.test + NURSESTATION, ALIS via silent SSO. GPOs deployed: `CSC - Caregiver Workstation` validated; `CSC - Caregiver Device Lockdown` deployed to `OU=Caregiver Devices`. Ticket #32303 billed 7.0h, invoice #67782 ($0.00 prepaid). |
|
||||
| 2026-06-08 | **Chris Knight workstation setup (onsite).** AD account finished (OU=Administrative, home folder, SG-FolderRedirect, mail set). Machine DESKTOP-N5G1ROO domain-joined + GuruRMM-enrolled (`205025ee`), Office installed. **MAJOR: root-caused why folder redirection failed on every machine** -- FR GPO targets were in misnamed `fdeploy1.ini`; Windows reads `fdeploy.ini` (absent) -> empty path -> silent no-op. Fixed by writing correct `fdeploy.ini` to GPO `{512B43A4}` + version bump 917506->983042. Native FR now works for new users. **ASSISTNURSE-PC reinstalled (Win10->Win11).** |
|
||||
| 2026-06-08 | **Edge UNC download bug diagnosed (no fix applied).** Ashley Jensen + Lois Lane on Edge 149.0.4022.52 cannot open Office files from Edge download panel when Downloads is UNC-redirected. Root cause: Chromium 149 regression (issue 519243472) in `LaunchShellExecuteViaExplorer`. Fix path decision left to Howard. |
|
||||
| 2026-06-09 | **Accounting scan-to-folder built + billing reconciliation.** Created `D:\Shares\Accounting` + `\Scans` on CS-SERVER; shared as `\\CS-SERVER\AcctDept`; new vaulted AD service account `svc-scan`; Brother MFC-L8900CDW Scan-to-Network profile configured (NTLMv2; test scan confirmed). Found pfSense blocks main-LAN->VLAN-20. Persistent drive maps set for Chris (Y:), Zachary (Y:), Lauren (X:). Reconciled crashed-session billing; live prepay confirmed 57.75h. |
|
||||
| 2026-06-10 | **Meredith Kuhn locked Word doc -- stale owner files on cascadesDS.** Five orphaned `~$` files dated 2024 in `\\cascadesds\Public\Company Web Docs\Staff Trainings\` caused false lock messages. Diagnosed and deleted via RMM in Meredith's `user_session` on ASSISTMAN-PC. Ticket #32403, 0.5h remote, block 56.75->56.25. |
|
||||
| 2026-06-12 | **Created shared mailboxes grievances@ + Surveys@ and delegated to Meredith & Ashley.** Both SharedMailbox type (cloud-only, no license). FullAccess + SendAs granted. Work via ComputerGuru Exchange Operator cert auth (EXO module v3.10.0 installed on Howard-Home). All 8 permission grants verified. Ticket #32417, 0.5h remote, block 56.25->55.75; Invoiced. |
|
||||
| 2026-06-15 | **Wireless RF full audit -- controller access gained.** Mike vaulted `infrastructure/uos-server-ssh-key` + `clients/cascades-tucson/unifi-ap-ssh` + `infrastructure/uos-server-network-api-rw`. `unifi-wifi` skill used end-to-end. Live audit confirmed 77 U7-Pro APs, ~574->587 clients, 2.4 GHz saturation as primary pain band (avg retry ~10-11%, cu_total 69-94%, catastrophic neighbor density). `live-stats.sh` accuracy bugs found and fixed mid-session (15-AP head cap, wrong satisfaction/retry fields). DFS concern corrected: retry DFS 8.4% ~= non-DFS 9.0% -- no throughput penalty; mid-session misdiagnosis withdrawn. 6 GHz (1 client) identified as largest untapped capacity. Tuning plan staged; no live changes applied. |
|
||||
| 2026-06-15 | **CS-SERVER slowness root-caused to degraded RAID-1; backup started; pfSense OpenVPN password reset.** Dell OMSA: PD 0:0:3 (320 GB WD SATA) Critical/Removed, Virtual Disk2 (C: mirror) Degraded -> C: on a single 320 GB Hitachi 5400 RPM spindle (root cause of slowness). Mike installed MSP360/CloudBerry cloud backup on CS-SERVER (closes HIPAA backup gap). Reset Howard's lost pfSense OpenVPN password via Diagnostics PHP-exec from CS-SERVER (local_user_set_password() -> AUTHOK); vaulted at `clients/cascades-tucson/pfsense-openvpn-howard`. |
|
||||
| 2026-06-16 | **Voice VLAN plan for Vertical phones (PLANNED, not executed).** Diagnosed split voice gear: Poly phones (22, WiFi/CSCNet/VLAN 20), AudioCodes (8, wired USW-16-PoE/Default LAN), Vertical desktop (wired, static, no ACG login). CSCNet confirmed as shared PPSK SSID (not simple staff/VLAN-20). GuruRMM recon: desktop RDP-only (not a PBX); CS-QB SMB-only/no SIP; phones likely cloud PBX. Designed VLAN 30 VOICE (10.0.30.0/24, isolated, internet-only egress); wrote cutover runbook (`docs/network/voice-vlan-cutover.md`); vendor email sent. Awaiting Richard's confirm + window. |
|
||||
| 2026-06-16 | **pfSense confirmed as pfSense Plus 25.07-RELEASE; health verified; home-LAN shadow resolved.** Howard-Home renumbered from 192.168.0.0/24 to 10.137.42.0/24 (removed collision with Cascades 192.168.0.0/24). pfSense now reachable from Howard-Home over the site VPN. SSH health check: DHCP not exhausted, DNS up, WAN stable, states 28-31k/790k, load 0.6 -- gateway ruled out as WiFi factor. `pfsense-ssh.sh` backend built and validated live (SSH, no RESTAPI package needed). |
|
||||
| 2026-06-16 | **Floor-4 2.4 GHz power-down pilot applied (first production RF change).** 14/15 Floor-4 radios set to 6 dBm (from ~23); avg retry 13.2->9.5% (~28% fewer retransmits); clients retained, no coverage loss. AP 445 lagged (left alone, harmless). AP-hang recovery procedure learned: `device-control poe-cycle` (NOT force-provision -- took 445 offline; removed from the tool). `dfs-check.sh` confirmed ZERO real radar events fleet-wide (DFS empirically clean). `unifi-wifi` skill feature-complete (WiFi monitor/tune/apply + switch/gateway/pfSense-SSH + multi-client + channel-plan + cron health). |
|
||||
|
||||
---
|
||||
|
||||
## Compilation Notes
|
||||
|
||||
**Session logs read:** 28 root session logs + client-specific logs in `clients/cascades-tucson/session-logs/` (through 2026-06-15 wireless RF audit) + 7 memory files + 5 structured docs. Date range: 2026-03-06 through 2026-06-15.
|
||||
**Session logs read:** all prior sessions + new 2026-06-15/16 logs (wireless RF audit, CS-SERVER RAID + VPN reset, voice VLAN plan) + 2 reports (unifi-full-audit, 2.4ghz-remediation-runbook) + 8 memory files. Date range: 2026-03-06 through 2026-06-16.
|
||||
|
||||
**Client folder:** `clients/cascades-tucson/` (NOT `clients/cascades/` — that directory does not exist).
|
||||
**Client folder:** `clients/cascades-tucson/` (NOT `clients/cascades/` -- that directory does not exist).
|
||||
|
||||
**Open items flagged as unverified:**
|
||||
- Break-glass accounts + YubiKeys — confirmed not created as of 2026-05-27; YubiKey arrival unconfirmed
|
||||
- Audit retention infra — approved 2026-04-29, not yet built
|
||||
- dunedolly21@gmail.com guest invite — confirm with Lauren
|
||||
- Windows MDM auto-enroll scope — confirm in portal (Entra → Devices → Mobility → Microsoft Intune → MDM user scope)
|
||||
- #32381 / #32382 ticket details (Tamra scanner, Megan file access) — referenced in 2026-06-04 session log reference table only; full ticket details not documented in session logs
|
||||
- #32370 — verify/likely closed; Syncro live 2026-06-15 shows 0 open tickets (was confirmed [New]/open on 2026-06-13)
|
||||
- Edge UNC download bug fix path — no fix applied as of 2026-06-08; decision pending Howard
|
||||
- ALIS BAA with Medtelligent — not yet verified; confirm with Meredith
|
||||
- JD Martin (jd.martin@cascadestucson.com) — confirmed Syncro contact; role not yet documented
|
||||
- Wireless RF tuning — tuning plan staged 2026-06-15; no changes applied; per-zone execution pending RW cred + go-ahead
|
||||
- Break-glass accounts + YubiKeys -- confirmed not created as of 2026-05-27; YubiKey arrival unconfirmed
|
||||
- Audit retention infra -- approved 2026-04-29, not yet built
|
||||
- dunedolly21@gmail.com guest invite -- confirm with Lauren
|
||||
- Windows MDM auto-enroll scope -- confirm in portal (Entra -> Devices -> Mobility -> Microsoft Intune -> MDM user scope)
|
||||
- #32370 -- verify/likely closed; Syncro live 2026-06-16 shows 0 open tickets
|
||||
- Edge UNC download bug fix path -- no fix applied as of 2026-06-08; decision pending Howard
|
||||
- ALIS BAA with Medtelligent -- not yet verified; confirm with Meredith
|
||||
- JD Martin (jd.martin@cascadestucson.com) -- confirmed Syncro contact; role not yet documented
|
||||
- CS-SERVER cloud backup: verify first full completes, confirm image-based / bare-metal + system-state, set retention; only then proceed with RAID remediation
|
||||
- NURSESTATION-PC: verify `CSC - Caregiver Device Lockdown` GPO activated (requires reboot; verify lock@3min, 90s warning, sign-out@15min, never-sleep)
|
||||
- Wireless RF: Floors 1-3, 5-6 power-down + Phase C disables pending scope go-ahead from Howard
|
||||
|
||||
**Resolved since last compile (2026-06-13 → 2026-06-15):**
|
||||
- Wireless controller access unblocked (2026-06-15): `infrastructure/uos-server-ssh-key` + `infrastructure/uos-server-network-api-rw` + `clients/cascades-tucson/unifi-ap-ssh` vaulted by Mike; live RF audit completed with `unifi-wifi` skill; `live-stats.sh` accuracy bugs fixed; tuning plan staged.
|
||||
**Resolved since last compile (2026-06-15 -> 2026-06-16):**
|
||||
- Howard-Home LAN shadow: resolved 2026-06-16 (renumbered to 10.137.42.0/24; Cascades 192.168.0.x now reachable over VPN)
|
||||
- pfSense version: confirmed pfSense Plus 25.07-RELEASE (was listed as "pfSense 24.0")
|
||||
- pfSense gateway: ruled out as WiFi factor (health check 2026-06-16)
|
||||
- DFS empirically clean: dfs-check.sh confirmed ZERO radar events fleet-wide (was theoretical concern)
|
||||
- Floor-4 2.4 GHz power-down: applied (first production RF change; retry 13.2->9.5%)
|
||||
- unifi-wifi skill: feature-complete as of 2026-06-16 (WiFi/switch/gateway/pfSense-SSH, all gated writes validated)
|
||||
|
||||
**Carried forward from prior compile (2026-06-05 → 2026-06-13):**
|
||||
- New tiered remediation app suite — confirmed consented 2026-04-21 (all 6 apps active)
|
||||
- DMARC — confirmed upgraded to p=quarantine;pct=100
|
||||
- ALIS AADSTS65001 sign-in failures — resolved 2026-06-03 by granting admin consent
|
||||
- Chris Knight bill.com / BOK email delivery (#32383) — Resolved (confirmed live 2026-06-13); BOK corrected in portal 2026-06-04, bill.com fixed sender-side (support + SendGrid suppression clear)
|
||||
- `CSC - Caregiver Device Lockdown` GPO — deployed 2026-06-05 (was blocked/pending in prior compile)
|
||||
- Hybrid Entra Join on NURSESTATION-PC — proven 2026-06-05; Intune-to-GPO pivot complete; full caregiver desktop access model validated end-to-end
|
||||
- Ticket #32303 billing — 7.0h billed 2026-06-05, invoice #67782 ($0.00 prepaid); ASSISTNURSE-PC reinstall 1.0h billed on same ticket (implied by balance chain 57.75→56.75); ticket Invoiced
|
||||
- Folder redirection root cause found and fixed (2026-06-08): `fdeploy.ini` written to GPO `{512B43A4}`; native FR now works for new users
|
||||
- Stale Word owner files on cascadesDS cleared (2026-06-10): 5 orphaned `~$` files deleted via RMM ASSISTMAN-PC session; ticket #32403 Invoiced
|
||||
- Shared mailboxes grievances@ + Surveys@ created and delegated (2026-06-12): ticket #32417 Invoiced; prepay block now 55.75h (confirmed live pull 2026-06-13)
|
||||
**Carried forward from prior compile:**
|
||||
- Wireless controller access unblocked (2026-06-15): SSH/Mongo + RW API + AP creds all vaulted; live RF audit completed; tuning plan staged
|
||||
- CS-SERVER RAID degraded + cloud backup installed (2026-06-15)
|
||||
- Voice VLAN VLAN 30 plan + runbook (2026-06-16); vendor email sent; awaiting confirm
|
||||
- CSCNet SSID correction: shared PPSK SSID (~230 per-key->network mappings), not "staff/VLAN-20"
|
||||
- Shared mailboxes grievances@ + Surveys@ created and delegated (2026-06-12): ticket #32417 Invoiced; prepay block 55.75h
|
||||
|
||||
## Backlinks
|
||||
|
||||
- [[projects/gururmm]] — RECEPTIONIST-PC enrolled (site CascadesTucson); CS-SERVER enrolled
|
||||
- [[wiki/systems/uos-server]] — shared UOS controller hosts the Cascades UniFi site (site_id `685f39068e65331c46ef6dd2`); SSH/Mongo access via `infrastructure/uos-server-ssh-key`
|
||||
- [[projects/gururmm]] -- RECEPTIONIST-PC enrolled (site CascadesTucson); CS-SERVER enrolled
|
||||
- [[wiki/systems/uos-server]] -- shared UOS controller hosts the Cascades UniFi site (site_id `685f39068e65331c46ef6dd2`); SSH/Mongo access via `infrastructure/uos-server-ssh-key`
|
||||
|
||||
@@ -18,7 +18,7 @@ Run `/wiki-lint` to check for stale entries and broken backlinks.
|
||||
|
||||
| Article | Summary | Last Compiled |
|
||||
|---|---|---|
|
||||
| [Cascades of Tucson](clients/cascades-tucson.md) | Prepaid block $175/hr, **55.75 hrs remaining** (live 2026-06-16); senior living; active domain migration + HIPAA compliance project; single DC on aging R610 hardware; caregiver restricted-access model PROVEN 2026-06-05: Hybrid Entra Join + CA allow-list + ALIS SSO validated on NURSESTATION-PC/pilot.test; GPO `CSC - Caregiver Workstation` (shortcuts + printers) built + validated; GPO `CSC - Caregiver Device Lockdown` deployed (HIPAA auto-logoff, activates on reboot); INTUNE_A PendingInput tenant-wide (MS case open; GPO path used instead); folder-redirection root cause fixed 2026-06-08 (fdeploy.ini); shared mailboxes grievances@/Surveys@ created + delegated 2026-06-12 (#32417); Monday cutover to real caregivers pending; #32383 (bill.com/BOK chris.knight) Resolved; UniFi wifi RF audit 2026-06-15 (77 U7-Pro APs/~574 clients via UOS controller): 2.4GHz primary pain band, DFS=resilience risk near Davis-Monthan, 6GHz untapped — tuning plan staged, not applied; CS-SERVER OS RAID-1 degraded 2026-06-15 (data-loss risk; cloud backup now started); Voice VLAN (VLAN 30) consolidation planned 2026-06-16 for Vertical phones + remote desktop (CSCNet confirmed a shared PPSK SSID); Syncro 0 open tickets | 2026-06-16 |
|
||||
| [Cascades of Tucson](clients/cascades-tucson.md) | Prepaid block $175/hr, **55.75 hrs remaining** (live 2026-06-16); senior living; active domain migration + HIPAA compliance project; single DC on aging R610 hardware; caregiver restricted-access model PROVEN 2026-06-05: Hybrid Entra Join + CA allow-list + ALIS SSO validated on NURSESTATION-PC/pilot.test; GPO `CSC - Caregiver Workstation` (shortcuts + printers) built + validated; GPO `CSC - Caregiver Device Lockdown` deployed (HIPAA auto-logoff, activates on reboot); INTUNE_A PendingInput tenant-wide (MS case open; GPO path used instead); folder-redirection root cause fixed 2026-06-08 (fdeploy.ini); shared mailboxes grievances@/Surveys@ created + delegated 2026-06-12 (#32417); Monday cutover to real caregivers pending; #32383 (bill.com/BOK chris.knight) Resolved; UniFi wifi RF (77 U7-Pro APs/~587 clients via UOS controller): 2.4GHz over-coverage = primary pain; pfSense ruled out as cause; Floor-4 power-down pilot applied 2026-06-16 (retry 13.2->9.5%); coverage-thin disable plan + 2.4 remediation runbook staged; DFS empirically clean; 6GHz untapped; CS-SERVER OS RAID-1 degraded 2026-06-15 (data-loss risk; cloud backup now started); Voice VLAN (VLAN 30) consolidation planned 2026-06-16 for Vertical phones + remote desktop (CSCNet confirmed a shared PPSK SSID); Syncro 0 open tickets | 2026-06-16 |
|
||||
| [Dataforth Corporation](clients/dataforth.md) | Prepaid block ~$2,099/mo, 34.5 hrs remaining; signal conditioning manufacturer; 64 DOS test stations; 2025 crypto attack recovery + incomplete restore (files dropped across shares — migration-gap audit in progress); 2026-03-27 phishing incident + MFA rollout; active test datasheet pipeline project; Neptune Exchange colocated at D2; 2026-06-04 SP1366 file recovery (19/20 PDFs restored from HGHAUBNER pre-attack backup); GuruRMM fleet 13→45 agents; 2026-06-02 Syncro asset reconciliation (78→20 keep/21 flag/28 remove/9 verify); fleet-wide Syncro agent break ~2025-10-06; Bitdefender phase-off in progress | 2026-06-04 |
|
||||
| [Instrumental Music Center](clients/instrumental-music-center.md) | Prepaid block $175/hr, 12.5 hrs remaining; music retail/repair; AIMsi POS on SQL Server 2019; phantom DC causing slow logons; GuruRMM enrolled (IMC1) | 2026-05-24 |
|
||||
| [Valley Wide Plastering](clients/valleywide.md) | Prepaid block, 10 hrs remaining; plastering/stucco contractor; HP DL360 Gen10 + XenServer; VB6 app modernization project; RDWeb brute-force incident; 11 Yealink phones pending | 2026-06-14 |
|
||||
|
||||
Reference in New Issue
Block a user