azcomputerguru/claudetools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
8d975c1b44648c88f397a8b67ed63049ac8c2b61
claudetools/clients/cascades-tucson/docs/issues/log.md
Howard Enos 8d975c1b44 import: ingested 160 files from C:\Users\howar\Clients 
Howard's personal MSP client documentation folder imported into shared
ClaudeTools repo via /import command. Scope:

Clients (structured MSP docs under clients/<name>/docs/):
- anaise       (NEW)  - 13 files
- cascades-tucson     - 47 files merged (existing had only reports/)
- dataforth           - 18 files merged (alongside incident reports)
- instrumental-music-center - 14 files merged
- khalsa       (NEW)  - 22 files, multi-site (camden, river)
- kittle       (NEW)  - 16 files incl. fix-pdf-preview, gpo-intranet-zone
- lens-auto-brokerage (NEW) - 3 files (name matches SOPS vault)
- _client_template    - 13-file scaffold for new clients

MSP tooling (projects/msp-tools/):
- msp-audit-scripts/ - server_audit.ps1, workstation_audit.ps1, README
- utilities/         - clean_printer_ports, win11_upgrade,
                       screenconnect-toolbox-commands

Credential handling:
- Extracted 1 inline password (Anaise DESKTOP-O8GF4SD / david)
  to SOPS vault: clients/anaise/desktop-o8gf4sd.sops.yaml
- Redacted overview.md with vault reference pattern
- Scanned all 160 files for keys/tokens/connection strings -
  no other credentials found

Skipped:
- Cascades/.claude/settings.local.json (per-machine config)
- Source-root CLAUDE.md (personal, claudetools has its own)
- scripts/server_audit.ps1 and workstation_audit.ps1 at source root
  (identical duplicates of msp-audit-scripts versions)

Memory updates:
- reference_client_docs_structure.md (layout, conventions, active list)
- reference_msp_audit_scripts.md (locations, ScreenConnect 80-char rule)

Session log: session-logs/2026-04-16-howard-client-docs-import.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 19:43:58 -07:00

38 KiB
Raw Blame History

Issue Log

Record past issues and their resolutions here. This helps the AI learn from historical troubleshooting and avoid repeating failed approaches.

2026-03-04 - CS-SERVER on 16-Year-Old Dell R610 — Imminent Hardware Failure Risk

  • Reported By: AI audit of server systeminfo
  • Severity: Critical
  • Symptoms: CS-SERVER (192.168.2.254) is the PRIMARY DOMAIN CONTROLLER for cascades.local running on a Dell PowerEdge R610 (~2009 vintage). This single server hosts AD DS, DNS, DHCP, File Server, Hyper-V, RDS, IIS, and NPS. If this hardware fails — and at 16+ years old it is statistically overdue — ALL services go down simultaneously with NO backup and NO second domain controller.
  • Root Cause: Legacy hardware never replaced. No redundancy or DR plan implemented.
  • Resolution: OPEN — Priority actions:
    1. IMMEDIATE: Configure Synology Active Backup for Business to back up C: and D: nightly
    2. URGENT: Deploy a second domain controller (VM on Synology, or new hardware, or Azure AD DS)
    3. Plan full server migration to modern hardware (Dell PowerEdge T350/R350 or similar)
    4. Separate roles across multiple servers/VMs to reduce blast radius
  • Time to Resolve: Pending
  • Lessons Learned: N/A

2026-03-04 - No Backup Solution for Any System

  • Reported By: AI audit
  • Severity: Critical
  • Symptoms: No backup exists for CS-SERVER, workstations, or M365 data. The only DC has no backup. File shares on D: (1.09TB, ~470GB used) are unprotected. Ransomware, hardware failure, or accidental deletion = permanent data loss.
  • Root Cause: Backup solution never implemented.
  • Resolution: PLANNED — Migration Phase 0.1 + Phase 4.4
    1. Synology Active Backup for Business (free) → back up CS-SERVER C: and D: to Synology NAS (Phase 0)
    2. Configure Hyper Backup on Synology → replicate to Backblaze B2 or Wasabi for offsite copy (Phase 4)
    3. Add M365 backup (Datto SaaS Protection, Veeam, or Acronis) (future)
  • Time to Resolve: Phase 0 (Day 1 evening)
  • Lessons Learned: N/A

2026-03-04 - Single Domain Controller — No AD Redundancy

  • Reported By: AI audit of server roles
  • Severity: Critical
  • Symptoms: CS-SERVER is the only domain controller for cascades.local. No secondary DC exists. AD DS, DNS for domain resolution, and DHCP (Windows) all depend on this single machine.
  • Root Cause: Second DC was never deployed.
  • Resolution: OPEN — Deploy a second DC. Options:
    1. VM on Synology (if it supports VMs) running Server 2019/2022 as DC
    2. New physical server (budget permitting)
    3. Azure AD DS or a cloud-hosted DC with site-to-site VPN
  • Time to Resolve: Pending — DC promotion takes ~1 hour once hardware/VM is ready
  • Lessons Learned: N/A

2026-03-04 - DHCP Conflict Check (pfSense + Windows DHCP)

  • Reported By: AI audit of pfSense config + server roles
  • Severity: Medium RESOLVED
  • Symptoms: Both pfSense and CS-SERVER have DHCP server roles installed.
  • Root Cause: Windows DHCP role is installed but has ZERO scopes configured. pfSense handles all DHCP for all VLANs. No conflict.
  • Resolution: RESOLVED — Get-DhcpServerv4Scope returned no results. pfSense is the sole DHCP server. Windows DHCP role could optionally be uninstalled to reduce confusion.
  • Time to Resolve: Resolved 2026-03-04
  • Lessons Learned: Role installed ≠ role active. Always verify scopes.

2026-03-04 - CS-QB VoIP VM Running on Failing Hardware

  • Reported By: AI audit of Hyper-V VMs
  • Severity: Medium
  • Symptoms: CS-QB VM (VoIP server, 2.35 GB RAM, IP 192.168.2.228) runs on CS-SERVER. VoIP is not MSP-managed, but the VM infrastructure IS. If CS-SERVER hardware fails, the phone system goes down with it.
  • Root Cause: VoIP VM placed on the only server, which is on 16-year-old hardware.
  • Resolution: OPEN — When CS-SERVER is migrated to new hardware, ensure CS-QB VM is migrated or relocated. Coordinate with VoIP provider since they require static IPs.
  • Time to Resolve: Pending — tied to server migration project
  • Lessons Learned: N/A

2026-03-04 - Synology Sync Machine VM is Powered Off

  • Reported By: AI audit of Hyper-V VMs
  • Severity: Low RESOLVED
  • Symptoms: A VM called "Synology Sync machine" exists on CS-SERVER but is powered off.
  • Root Cause: VM was set up "just in case" but was never used.
  • Resolution: RESOLVED — Confirmed by Howard: VM does nothing, safe to delete. Scheduled for deletion in Migration Phase 5.2.
  • Time to Resolve: Resolved 2026-03-04 (deletion in Phase 5)
  • Lessons Learned: N/A

2026-03-04 - CS-SERVER Timezone Mismatch with pfSense

  • Reported By: AI audit
  • Severity: Low RESOLVED
  • Symptoms: CS-SERVER is set to Pacific Time (UTC-8, observes DST). pfSense is set to America/Phoenix (UTC-7, does NOT observe DST). During DST (March-November), these are the same time. Outside DST, they differ by 1 hour. This can cause Kerberos authentication issues, log correlation problems, and GPO scheduling mismatches.
  • Root Cause: Inconsistent timezone configuration.
  • Resolution: RESOLVED 2026-03-07 — Set CS-SERVER to (UTC-07:00) Arizona via Set-TimeZone -Id "US Mountain Standard Time".
  • Time to Resolve: Resolved 2026-03-07
  • Lessons Learned: Arizona does not observe DST — use "US Mountain Standard Time" not "Mountain Standard Time".

2026-03-04 - Floating Firewall Rule #4 Passes ALL IPv4 Traffic

  • Reported By: AI audit of pfSense config
  • Severity: High
  • Symptoms: Floating rule #4 (PASS, any interface, IPv4, any source, any destination) overrides per-interface rules. Breaks room-to-room isolation. Residents can potentially reach staff VLAN (10.0.20.0/24), management interfaces, VoIP phones, and other rooms.
  • Root Cause: Overly permissive floating rule — likely added as a quick fix or during initial setup and never scoped down.
  • Resolution: PLANNED — Migration Phase 1.3. Disable floating rule #4, replace with: ResidentsGroup → ! RFC1918 (rooms internet only). Replace INTERNAL rules with scoped AD/SMB/Print/ICMP allows. See migration/phase1-network.md.
  • Time to Resolve: Phase 1 (Day 1 evening)
  • Lessons Learned: N/A

2026-03-04 - 9 Offline Access Points

  • Reported By: AI audit of UniFi AP data
  • Severity: Medium
  • Symptoms: 9 APs showing offline in UniFi controller. Coverage gaps on floors 1-4.
  • Affected APs:
    • 108 (192.168.6.127) — wrong IP range, was mesh uplink
    • 121 (192.168.2.184) — mesh uplink
    • 128 (192.168.2.95) — no uplink detected
    • 204 (192.168.7.243) — wrong IP range, no uplink
    • 335 (192.168.2.206) — no uplink
    • 406 (192.168.2.4) — no uplink
    • 441 (192.168.2.200) — no uplink
    • 450 (192.168.6.207) — wrong IP range, no uplink
    • 4th Floor Atrium (192.168.3.28) — no uplink
  • Root Cause: Unknown — could be power loss, cable failure, or adoption issues. APs 108, 204, and 450 have IPs in 192.168.6.x/7.x which are outside the LAN DHCP range (192.168.2.x-3.x), suggesting they may have been misconfigured or adopted to wrong network.
  • Resolution: OPEN — Visit each AP physically. Check PoE power, cable, and switch port status. Re-adopt APs with wrong IPs. Prioritize 4th Floor Atrium (common area coverage).
  • Time to Resolve: Pending
  • Lessons Learned: N/A

2026-03-04 - Room 218 DHCP Scope Misconfigured (Single IP)

  • Reported By: AI audit of pfSense config
  • Severity: Medium RESOLVED
  • Symptoms: Room 218 DHCP scope range is 10.2.18.2 to 10.2.18.2. Only 1 IP can be leased. Any second device in the room will fail to get an address.
  • Root Cause: Typo or error in pfSense DHCP configuration. End of range should be 10.2.18.14 (matching the /28 pattern used by all other rooms).
  • Resolution: RESOLVED 2026-03-07 — Range end changed to 10.2.18.14.
  • Time to Resolve: Resolved 2026-03-07
  • Lessons Learned: N/A

2026-03-04 - Room 339 Interface Possibly Disabled

  • Reported By: AI audit of pfSense config
  • Severity: Medium
  • Symptoms: Room 339 VLAN interface (igc1.339, opt129) is missing the <enable> tag in the pfSense config. Room may have no network connectivity.
  • Root Cause: Unknown — may be intentional (vacant room) or a config oversight.
  • Resolution: OPEN — Verify if room 339 is occupied. If yes, enable the interface in pfSense under Interfaces > Room339. If intentionally disabled, document the reason.
  • Time to Resolve: Pending
  • Lessons Learned: N/A

2026-03-04 - 206 Health Services Printer Drops WiFi

  • Reported By: Staff / printer doc
  • Severity: Medium
  • Symptoms: Brother MFC-L8900CDW at 192.168.1.138 will not stay connected to wireless. Repeated disconnections.
  • Root Cause: Likely combination of: Brother deep sleep mode dropping WiFi, possible weak signal from nearest AP, and/or DHCP lease issues. IP is in 192.168.1.x range (valid within /22 LAN but unusual compared to most devices on 192.168.2.x-3.x).
  • Resolution: OPEN — Recommended fix priority:
    1. Wire the printer (run ethernet to nearest switch port)
    2. If wiring not possible: set DHCP reservation, disable deep sleep on printer, verify AP signal strength in room 206
  • Time to Resolve: Pending
  • Lessons Learned: N/A

2026-03-04 - Bizhub C368 Location and Status Unknown

  • Reported By: AI audit of printer inventory
  • Severity: Low
  • Symptoms: A Bizhub C368 is listed in the printer inventory but has no IP address, no location, and no status documented. Since printers are rentals, this unit needs to be accounted for.
  • Root Cause: Incomplete documentation.
  • Resolution: OPEN — Locate the Bizhub C368 physically. Document its IP, location, and connection type. Verify it's on the rental tracking list.
  • Time to Resolve: Pending
  • Lessons Learned: N/A

2026-03-04 - Room 130 Disabled Firewall Rule

  • Reported By: AI audit of pfSense config
  • Severity: Low RESOLVED
  • Symptoms: Room 130 has a disabled TCP PASS rule in pfSense. Dead rules add clutter and confusion to the firewall config.
  • Root Cause: Likely a test rule that was disabled and never removed.
  • Resolution: RESOLVED 2026-03-07 — Disabled rule deleted from Room130 interface.
  • Time to Resolve: Resolved 2026-03-07
  • Lessons Learned: N/A

2026-03-04 - SSH Enabled on pfSense — Verify Hardening

  • Reported By: AI audit of pfSense config
  • Severity: Low
  • Symptoms: SSH is enabled on pfSense. Not inherently a problem but should be hardened.
  • Root Cause: N/A — feature is intentionally enabled for remote management.
  • Resolution: OPEN — Verify: key-based auth only (no password auth), SSH access restricted to management VLAN or VPN only, not exposed on WAN.
  • Time to Resolve: Pending
  • Lessons Learned: N/A

2026-03-04 - RMM Data Not Documented

  • Reported By: AI audit
  • Severity: Low
  • Symptoms: The RMM documentation is blank. No visibility into what devices are monitored, what alerts are configured, or what patching policies are in place.
  • Root Cause: Documentation gap.
  • Resolution: OPEN — Fill in rmm/rmm.md with the RMM product, agent counts, monitoring policies, and patch schedule.
  • Time to Resolve: Pending
  • Lessons Learned: N/A

2026-03-05 - No Group Policy Configured

  • Reported By: AI audit of AD
  • Severity: High
  • Symptoms: gpresult /r shows no RSoP data on CRYSTAL-PC. No GPOs are linked to any OU. No password policy, no account lockout policy, no security baselines, no drive mappings — only Windows defaults apply.
  • Root Cause: GPOs were never created or configured.
  • Resolution: PLANNED — Migration Phase 2.5 + 3.2. Create 4 GPOs (Security Baseline, Drive Mappings, Printer Deployment, Windows Update). Link after first successful domain join. See migration/phase2-server-prep.md.
  • Time to Resolve: Phase 2-3 (Day 2-3)
  • Lessons Learned: N/A

2026-03-05 - RDS Licensing Not Configured — Compliance Risk

  • Reported By: AI audit of CS-SERVER
  • Severity: High
  • Symptoms: Get-RDLicenseConfiguration returns Mode=NotConfigured, no license servers. RDS roles (Connection Broker, Session Host, Web Access) are installed. Server was installed 8/4/2024 — the 120-day grace period expired ~17 months ago.
  • Root Cause: RDS CALs were never purchased or configured.
  • Resolution: PLANNED — Migration Phase 5.4. Check if anyone uses RDS. If yes → purchase CALs. If no → uninstall RDS role.
  • Time to Resolve: Phase 5 (Day 4)
  • Lessons Learned: N/A

2026-03-05 - Only 6 Computers Domain-Joined (4+ Staff PCs Missing)

  • Reported By: AI audit of AD computers
  • Severity: Medium
  • Symptoms: Get-ADComputer returns only 6 objects (CS-SERVER, CS-QB, CRYSTAL-PC, ACCT2-PC, DESKTOP-H6QHRR7, DESKTOP-1ISF081). Known staff PCs SALES4-PC, CHEF-PC, MDIRECTOR-PC, and DESKTOP-KQSL232 are NOT in AD. These machines are on the network but not domain-joined.
  • Root Cause: PCs were likely set up as workgroup machines and never joined to the domain.
  • Resolution: PLANNED — Migration Phase 3. Join in order: DESKTOP-KQSL232 → CHEF-PC → SALES4-PC → MDIRECTOR-PC. See migration/phase3-domain-join.md and scripts.
  • Time to Resolve: Phase 3 (Day 3)
  • Lessons Learned: N/A

2026-03-05 - Shared/Generic AD Accounts in Use

  • Reported By: AI audit of AD users
  • Severity: Medium
  • Symptoms: Three shared/generic accounts exist and are enabled: Culinary, Receptionist, saleshare. Shared accounts cannot be audited to a specific person — if something is deleted or accessed inappropriately, there's no way to trace who did it.
  • Root Cause: Created for convenience instead of individual accounts.
  • Resolution: PLANNED — Migration Phase 5.3. Replace with individual user accounts + security group membership.
  • Time to Resolve: Phase 5 (Day 4)
  • Lessons Learned: N/A

2026-03-05 - 5 Stale Disabled AD Accounts

  • Reported By: AI audit of AD users
  • Severity: Low
  • Symptoms: 5 disabled user accounts (Anna.Pitzlin, Nela.Durut-Azizi, Jodi.Ramstack, Monica.Ramirez, Jeff.Bristol) are still in AD. Disabled accounts with remaining permissions or group memberships can be a security risk if re-enabled.
  • Root Cause: Former employees disabled but not removed.
  • Resolution: PLANNED — Migration Phase 2.2. Delete after client confirmation. See migration/scripts/phase2-ad-setup.ps1 ($DeleteStaleAccounts flag).
  • Time to Resolve: Phase 2 (Day 2)
  • Lessons Learned: N/A

2026-03-05 - QuickBooks Pro 2024 Installed on Domain Controller

  • Reported By: AI audit of installed software
  • Severity: High
  • Symptoms: QuickBooks Pro 2024 (v34.0) is installed directly on CS-SERVER, the primary domain controller. QuickBooks Database Server is listening on port 6600. Business applications should never run on a DC — they increase attack surface, consume resources needed for AD services, and complicate server migration.
  • Root Cause: QuickBooks was installed on the only available server rather than a dedicated workstation or VM.
  • Resolution: OPEN — Migrate QuickBooks to a dedicated workstation or VM. The QuickBooks database files on D:\Shares can still be accessed via network share. QuickBooks Desktop can run in multi-user mode with the database on a file share.
  • Time to Resolve: Pending
  • Lessons Learned: N/A

2026-03-05 - Stale DNS Records in cascades.local Zone

  • Reported By: AI audit of Windows DNS
  • Severity: Medium RESOLVED
  • Symptoms: Multiple DNS A records in the cascades.local zone point to incorrect or outdated IPs:
    • cascades.local @ → 192.168.0.5 and 192.168.2.59 (DC is actually at 192.168.2.254)
    • CRYSTAL-PC → 192.168.5.115 (should be 10.0.20.205)
    • CS-QB → 192.168.5.29 (should be 192.168.2.228)
    • DESKTOP-1ISF081 → 192.168.5.30 (192.168.5.x is not a documented subnet)
    • DomainDnsZones/ForestDnsZones → 192.168.0.5 and 192.168.2.59
  • Root Cause: Dynamic DNS registrations from old IPs were never cleaned up. DNS scavenging was not configured.
  • Resolution: RESOLVED 2026-03-06 — All stale records removed. Correct @ record added (192.168.2.254). DomainDnsZones/ForestDnsZones fixed. DNS scavenging enabled (7-day interval). First scavenge available 3/13/2026.
  • Time to Resolve: Resolved 2026-03-06
  • Lessons Learned: Enable DNS scavenging on all AD-integrated zones. Set aging on zone + server-level scavenging.

2026-03-05 - No Reverse DNS Lookup Zones

  • Reported By: AI audit of Windows DNS
  • Severity: Low RESOLVED
  • Symptoms: No reverse lookup zones exist for production subnets (192.168.0.0/22, 10.0.20.0/24). Only auto-created placeholder zones (0, 127, 255). PTR lookups will fail for all internal hosts.
  • Root Cause: Reverse zones were never created.
  • Resolution: RESOLVED 2026-03-06 — 5 reverse lookup zones created: 0.168.192, 1.168.192, 2.168.192, 3.168.192.in-addr.arpa + 20.0.10.in-addr.arpa.
  • Time to Resolve: Resolved 2026-03-06
  • Lessons Learned: N/A

2026-03-05 - Guest WiFi on Same Network as Servers — No Isolation

  • Reported By: AI audit of UniFi WiFi config
  • Severity: High RESOLVED
  • Symptoms: The "Guest" SSID is mapped to the Native Network (Default LAN, 192.168.0.0/22). Guest users land on the same L2/L3 network as CS-SERVER (DC, 192.168.2.254), Synology NAS (192.168.0.120), pfSense management (192.168.0.1), and all LAN infrastructure.
  • Root Cause: Guest SSID was configured on the default/native network instead of an isolated VLAN.
  • Resolution: RESOLVED 2026-03-06 — Guest VLAN 50 created (10.0.50.0/24, igc1.50 GUEST interface). DHCP scope 10.0.50.50-239. Four firewall rules (block LAN, block 10.x, block 172.x, pass internet). Guest SSID reassigned to VLAN 50 in UniFi. Needs onsite test to confirm isolation.
  • Time to Resolve: Resolved 2026-03-06 (pending onsite verification)
  • Lessons Learned: Always isolate guest WiFi on a dedicated VLAN with explicit deny rules to RFC1918.

2026-03-05 - UniFi VLAN 10 "CSC Internal Network" Mismatch

  • Reported By: AI audit of UniFi WiFi config
  • Severity: Low
  • Symptoms: UniFi defines "CSC Internal Network" as VLAN 10, but pfSense has the INTERNAL staff interface on VLAN 20 (igc1.20, 10.0.20.0/24). UniFi also has "Internal" on VLAN 20 (correct). VLAN 10 may be orphaned or cause confusion.
  • Root Cause: Likely a leftover from initial setup or a mislabeled network.
  • Resolution: PLANNED — Migration Phase 1.5. Verify unused, delete from UniFi.
  • Time to Resolve: Phase 1 (Day 1 evening)
  • Lessons Learned: N/A

2026-03-07 - Non-IT Staff in Domain Admins Group

  • Reported By: AD export analysis (2026-03-07)
  • Severity: High
  • Symptoms: Domain Admins group contains 5 members. Meredith.Kuhn (administrative staff) and John.Trozzi (maintenance) are Domain Admins. Monica.Ramirez (DISABLED account) is still in Domain Admins. Non-IT users with DA privileges can accidentally or intentionally modify AD, GPOs, DNS, or any domain-joined system.
  • Root Cause: Overly permissive group membership, likely set up for convenience.
  • Resolution: PLANNED — Phase 2.2. Remove Monica.Ramirez, Meredith.Kuhn, and John.Trozzi from Domain Admins. Only Administrator and sysadmin should remain.
  • Time to Resolve: Pending
  • Lessons Learned: N/A

2026-03-07 - 3 Undocumented GPOs Found (Dec 2025)

  • Reported By: AD export analysis (2026-03-07)
  • Severity: Medium REVIEWED
  • Symptoms: Three GPOs exist that were not previously documented: CopyRoomPrinter (Dec 10, 2025), Nurses-Kiosk (Dec 15, 2025), MemCareMedTechPrinter (Dec 11, 2025). Unknown who created them, what settings they contain, and whether they are linked to any OU.
  • Root Cause: Created by previous MSP or sysadmin account, not documented.
  • Resolution: REVIEWED 2026-03-07 — GPO report exported and analyzed. All 3 GPOs are completely empty — no computer or user configuration settings defined, not linked to any OU. Safe to delete with zero impact. Delete during Phase 2.2 AD cleanup.
  • Time to Resolve: Reviewed 2026-03-07, deletion pending Phase 2.2
  • Lessons Learned: N/A

2026-03-07 - Account Lockout Policy Disabled (Default Domain Policy)

  • Reported By: GPO report analysis (2026-03-07)
  • Severity: High
  • Symptoms: Default Domain Policy has account lockout threshold set to 0 (disabled). This means unlimited failed password attempts are allowed with no lockout, enabling brute-force attacks against any AD account.
  • Root Cause: Default Domain Policy was never hardened — only basic password complexity was configured.
  • Resolution: PLANNED — Set account lockout threshold to 5 invalid attempts, lockout duration 30 minutes, reset counter after 30 minutes. Will be applied via the planned "CSC - Security Baseline" GPO in Phase 2.6.
  • Time to Resolve: Pending Phase 2.6
  • Lessons Learned: N/A

2026-03-07 - Synology NAS Uses ext4 — Active Backup for Business Blocked

  • Reported By: Backup setup attempt (2026-03-07)
  • Severity: Medium
  • Symptoms: Synology Active Backup for Business requires Btrfs filesystem. Synology Volume 1 is formatted as ext4. Cannot convert without wiping the volume.
  • Root Cause: NAS was set up with ext4 (older default) rather than Btrfs.
  • Resolution: PLANNED — Use Windows Server Backup on CS-SERVER to back up to a Synology SMB share instead. Alternative: Veeam Backup Free Edition.
  • Time to Resolve: Pending — next session
  • Lessons Learned: Always verify NAS filesystem before planning backup solutions. ABB requires Btrfs.

2026-03-07 - ARP Flapping: LG TV Dual-Connected (WiFi + Ethernet)

  • Reported By: ARP log analysis
  • Severity: High RESOLVED
  • Symptoms: 192.168.2.148 and 192.168.2.152 flapping every 30-60 seconds between MACs 58:96:0a:c6:1e:49 (ethernet) and e0:85:4d:4d:f0:3e (WiFi). Hundreds of ARP "moved from" entries daily. Constant since at least Feb 8.
  • Root Cause: LG webOS TV connected via both ethernet (1st Floor USW Port 18) and WiFi (CSC ENT, 5 GHz) simultaneously. Both interfaces claimed the same IP, causing pfSense ARP table to flip between MACs.
  • Resolution: RESOLVED 2026-03-07 — Disabled ethernet port (1st Floor USW Port 18) in UniFi. TV now WiFi-only. DHCP reservation recommended for WiFi MAC e0:85:4d:4d:f0:3e at 192.168.2.148.
  • Time to Resolve: Resolved 2026-03-07
  • Lessons Learned: Check for dual-homed devices (WiFi + ethernet) when ARP flapping is detected. Smart TVs often auto-connect to both.

2026-03-07 - ARP Flapping: Brother Printer at 192.168.2.53 Dual-Connected

  • Reported By: ARP log analysis
  • Severity: Medium
  • Symptoms: 192.168.2.53 flapping between MACs c8:a3:e8:a2:dd:9e (ethernet, MemCare switch Port 3) and 80:3f:5d:72:7b:ee (WiFi, CSC ENT 2.4 GHz). Hostname BRWC8A3E8A2DD9E confirms Brother printer.
  • Root Cause: Brother printer connected via both ethernet and WiFi simultaneously, same as the LG TV issue.
  • Resolution: OPEN — Needs onsite visit to determine whether to keep WiFi or ethernet connection. Disable the other.
  • Time to Resolve: Pending onsite visit
  • Lessons Learned: N/A

2026-03-07 - Minor ARP Flapping in Resident Rooms (307, 130) and iPhone MAC Randomization

  • Reported By: ARP log analysis
  • Severity: Low
  • Symptoms: Occasional ARP flapping: Room 307 (10.3.7.7, evenings), Room 130 (10.1.30.8, Feb 22 only), and 192.168.3.115 (iPhone, ~6 AM daily). Room 130 and 307 devices no longer on network. iPhone flapping caused by iOS private MAC randomization.
  • Root Cause: Room conflicts likely transient DHCP collisions from resident devices. iPhone conflict is normal iOS behavior.
  • Resolution: No action required. Room 307 noted for onsite check. Room 130 devices gone. iPhone MAC randomization is cosmetic.
  • Time to Resolve: N/A
  • Lessons Learned: N/A

2026-03-07 - AD OU Structure: 13 Junk Root-Level OUs + CN=Users Misplacement

  • Reported By: AD export analysis (2026-03-07)
  • Severity: Medium
  • Symptoms: 10 duplicate department OUs exist at root level AND under Departments OU. 3 additional empty OUs (Managment [misspelled], MemCare, Sales) at root. 20 user accounts sitting in the default CN=Users container instead of proper department OUs. 5 computers in default CN=Computers instead of a Workstations OU.
  • Root Cause: Previous admin created OUs at root level, then also under Departments. Accounts were created in the default container and never moved.
  • Resolution: PLANNED — Phase 2.1. Scripts created:
    1. phase2-ou-cleanup.ps1 — Audit root OUs (confirm empty), delete 13 root-level OUs, delete/disable stale accounts in CN=Users, flag Lupe.Sanchez for review
    2. phase2-ad-setup.ps1 — Create Workstations OU, move 4 staff PCs from CN=Computers
  • Time to Resolve: Pending — run on CS-SERVER via ScreenConnect
  • Lessons Learned: N/A

2026-03-07 - Lupe.Sanchez: Possible Duplicate of Guadalupe.Sanchez

  • Reported By: AD export analysis (2026-03-07)
  • Severity: Low
  • Symptoms: Lupe.Sanchez exists in CN=Users (enabled). Guadalupe.Sanchez already exists in Departments\Housekeeping. "Lupe" is a common nickname for "Guadalupe". One account may be a duplicate.
  • Root Cause: Unknown — may have been created by different admins at different times.
  • Resolution: OPEN — Flag for client review onsite. Check which account is actively used, delete the duplicate.
  • Time to Resolve: Pending onsite visit
  • Lessons Learned: N/A

2026-03-08 - M365 Licenses Fully Allocated (0 Available)

  • Reported By: M365 admin portal audit (2026-03-08)
  • Severity: High
  • Symptoms: Microsoft 365 Business Standard is 34/34 with 0 available. Cannot add new users (e.g., nick pavloff, new hire created 2026-03-07) without purchasing additional licenses.
  • Root Cause: 12 generic/role-based accounts (accounting@, frontdesk@, hr@, security@, memcarereceptionist@, boadmin@, accountingassistant@, Training@, etc.) each consume a full Business Standard license ($12.50/mo) instead of being shared mailboxes (free).
  • Resolution: PLANNED — Convert role-based accounts to shared mailboxes in Exchange Online. This frees up 12 licenses ($150/mo savings). Shared mailboxes don't require a license and still receive/send email. Users access them via permissions instead of logging in directly.
  • Time to Resolve: Pending — next M365 cleanup session
  • Lessons Learned: N/A

2026-03-08 - 12 Role-Based M365 Accounts Wasting Licenses

  • Reported By: M365 user export analysis (2026-03-08)
  • Severity: Medium
  • Symptoms: 12 generic/role-based accounts are set up as regular licensed users instead of shared mailboxes: accounting@, accountingassistant@, boadmin@, frontdesk@, hr@, memcarereceptionist@, security@, Training@, Kitchenipad@, medtech@, nurse@, transportation@. Each consumes a Business Standard or Exchange Online Essentials license.
  • Root Cause: Previous admin created role accounts as regular users rather than shared mailboxes.
  • Resolution: PLANNED — For each: convert to shared mailbox (Set-Mailbox -Type Shared), remove license, grant FullAccess permission to appropriate users. No data loss — mailbox contents are preserved during conversion.
  • Time to Resolve: Pending
  • Lessons Learned: N/A

2026-03-08 - AD ↔ M365 Identity Mismatch: Tamra Johnson → Matthews

  • Reported By: M365/AD cross-reference (2026-03-08)
  • Severity: Medium
  • Symptoms: AD account is Tamra.Johnson. M365 UPN is tamra.matthews@cascadestucson.com (name changed, likely married). The tamra.johnson@ alias still works for email but the display name and UPN are mismatched between AD and M365.
  • Root Cause: Name change updated in M365 but not in AD.
  • Resolution: PLANNED — Update AD: rename Tamra.Johnson to Tamra.Matthews (Set-ADUser, Rename-ADObject). Coordinate timing — may affect drive mappings, profile path, etc. if already domain-joined.
  • Time to Resolve: Pending
  • Lessons Learned: N/A

2026-03-08 - 13 AD Users Have No M365 Account

  • Reported By: M365/AD cross-reference (2026-03-08)
  • Severity: Low
  • Symptoms: 13 AD user accounts have no matching M365 account. These users cannot receive email at cascadestucson.com.
  • Root Cause: Hourly/floor staff (front desk, courtesy patrol, transportation) given AD accounts but not M365 licenses.
  • Resolution: CONFIRMED 2026-03-10 — HR verified all are current employees. Roles: Front Desk/Courtesy Patrol (Sebastian.Leon, Sheldon.Gardfrey, Cathy.Kingston, Ray.Rai), MC Front Desk (Michelle.Shestko), Transportation (Richard.Adams, Julian.Crim, Christopher.Holik), Housekeeping (Guadalupe.Sanchez — already has M365 as lupe.sanchez@). Shontiel.Nunn transferring soon. Alyssa.Brooks already has M365. Still need to determine if remaining users need email.
  • Time to Resolve: Pending — determine email needs, free licenses via role account cleanup first
  • Lessons Learned: N/A

2026-03-08 - nick pavloff: New M365 User, No AD Account

  • Reported By: M365 user export (2026-03-08)
  • Severity: Medium
  • Symptoms: nick.pavloff@cascadestucson.com was created 2026-03-07 with a Business Standard license. No matching AD account exists. User cannot log into domain-joined PCs or access file shares.
  • Root Cause: New hire added to M365 but AD account not yet created.
  • Resolution: PLANNED — Create AD account (Nick.Pavloff) in appropriate department OU. Determine department and group membership. Add to domain join list if they have a dedicated PC.
  • Time to Resolve: Pending
  • Lessons Learned: N/A

2026-03-08 - Kristiana Dowse: M365 Licensed, Not in AD

  • Reported By: M365 user export (2026-03-08)
  • Severity: Low
  • Symptoms: kristiana.dowse@cascadestucson.com has a Business Standard license (created 2022-03-02, password never changed from default). No matching AD account exists. Unknown if current employee or former.
  • Root Cause: Former employee whose M365 was never cleaned up.
  • Resolution: CONFIRMED 2026-03-10 — HR confirmed DELETE. Block sign-in, remove license, delete account. Frees 1 Business Standard license.
  • Time to Resolve: Pending — next M365 cleanup session
  • Lessons Learned: N/A

2026-03-08 - Sandra Fish Still Global Admin on M365 Tenant

  • Reported By: M365 admin portal audit (2026-03-08)
  • Severity: Medium
  • Symptoms: admin@NETORGFT4257522.onmicrosoft.com (Sandra Fish) holds the Entra ID P2 license and appears to be the original/primary global admin. Sandra Fish is listed as previous owner/manager.
  • Root Cause: Tenant was created under Sandra Fish's account and admin access was never transitioned.
  • Resolution: OPEN — Verify with Howard: should Sandra Fish retain global admin? If she is no longer involved, create a new break-glass admin account, transfer global admin, and downgrade or remove her access.
  • Time to Resolve: Pending client discussion
  • Lessons Learned: N/A

2026-03-08 - 3 Former Employee Shared Mailboxes Still Active

  • Reported By: M365 admin portal (2026-03-08)
  • Severity: Low
  • Symptoms: Shared mailboxes exist for Anna Pitzlin, Jeff Bristol, and Nela Durut-Azizi — all confirmed former employees. Jeff Bristol and Nela Durut-Azizi have sign-in blocked. Anna Pitzlin's is unlicensed.
  • Root Cause: Accounts were converted to shared mailboxes (or created as such) but never cleaned up.
  • Resolution: CONFIRMED 2026-03-10 — HR says Anna and Nela were forwarded to Meredith but can now be deleted. Jeff Bristol still pending (Lauren Hasselman replaced him — may need forwarding).
  • Time to Resolve: Pending — next M365 cleanup session
  • Lessons Learned: N/A

2026-03-08 - "howaed" Typo Guest Account in M365

  • Reported By: M365 user export (2026-03-08)
  • Severity: Low
  • Symptoms: External guest account howaed@azcomputerguru.com exists alongside the correct howard@azcomputerguru.com. The "howaed" account is a typo.
  • Root Cause: Typo when inviting external guest.
  • Resolution: PLANNED — Delete the howaed@azcomputerguru.com guest account from Entra ID.
  • Time to Resolve: Quick fix — next M365 session
  • Lessons Learned: N/A

2026-03-08 - AD and M365 Are Completely Separate Identity Systems

  • Reported By: M365/AD cross-reference (2026-03-08)
  • Severity: Medium
  • Symptoms: DirSync/Entra Connect is not configured. AD accounts and M365 accounts are completely independent — users have different passwords, no SSO, and no automatic provisioning. Changes in one system (name changes, new hires, terminations) must be manually replicated to the other.
  • Root Cause: Entra Connect was never set up.
  • Resolution: OPEN — Evaluate Entra Connect for hybrid identity. Consider: does the environment benefit from SSO? With only 6 domain-joined PCs currently, the value is limited. Revisit after Phase 3 domain joins bring more PCs online. For now, document the manual mapping in cloud/m365.md.
  • Time to Resolve: Pending — evaluate after Phase 3
  • Lessons Learned: N/A

2026-03-10 - Synology NAS Stores PHI Without Audit Controls

  • Reported By: HIPAA compliance review (2026-03-10)
  • Severity: Critical
  • Symptoms: Synology NAS (cascadesDS, 192.168.0.120) stores PHI (resident/facility data). Filesystem is ext4 — cannot enable Windows audit logging. Access via SMB but no read/write audit trail. HIPAA §164.312(b) requires audit controls on PHI storage.
  • Root Cause: Synology was never designed with compliance in mind. ext4 cannot support NTFS audit logging.
  • Resolution: PLANNED — Migration Phase 4: Move all Synology shares to CS-SERVER NTFS volumes. Enable Advanced Audit Logging on PHI shares. Retire Synology from PHI scope (becomes backup target only).
  • Time to Resolve: Phase 4
  • Lessons Learned: PHI storage must support audit logging. NTFS + AD group permissions provide compliance-grade access control.

2026-03-10 - No Microsoft BAA Signed for M365

  • Reported By: HIPAA compliance review (2026-03-10)
  • Severity: High
  • Symptoms: M365 email (cascadestucson.com) may contain PHI in messages and attachments. No Business Associate Agreement (BAA) has been signed with Microsoft. HIPAA §164.308(b)(1) requires BAA with all business associates who handle PHI.
  • Root Cause: BAA never signed by previous MSP or management.
  • Resolution: PLANNED — Sign Microsoft HIPAA BAA via M365 Admin Center → Settings → Org Settings → Security & Privacy. Also verify BAA exists with ALIS (go-alis.com).
  • Time to Resolve: Quick — can be done in M365 admin portal
  • Lessons Learned: N/A

2026-03-10 - No MFA Enabled on M365

  • Reported By: HIPAA compliance review (2026-03-10)
  • Severity: High
  • Symptoms: No MFA (Security Defaults or Conditional Access) configured on M365 tenant. Staff accounts that access ALIS and email with PHI are protected only by passwords. HIPAA §164.312(d) requires person authentication controls.
  • Root Cause: MFA never enabled by previous MSP.
  • Resolution: PLANNED — Enable Security Defaults in Entra ID (free). All users will be prompted to set up MFA on next login.
  • Time to Resolve: Quick — 5 minutes to enable, users set up on next login
  • Lessons Learned: N/A

2026-03-10 - Kitchen iPads and Thermal Printers Not Inventoried or Isolated

  • Reported By: HIPAA compliance review (2026-03-10)
  • Severity: Medium
  • Symptoms: 9 kitchen iPads are on INTERNAL VLAN (10.0.20.x) with full access to staff resources. They are food-service only (NOT medical — used for taking food orders). Kitchen thermal receipt printers exist but are not inventoried (count, IP, model unknown). iPads should be restricted to kitchen printers only.
  • Root Cause: Kitchen operations not segmented from staff network.
  • Resolution: PLANNED — Phase 1.1b:
    1. Onsite: inventory kitchen thermal printers (count, model, IP, MAC, connection type)
    2. Create firewall rules restricting kitchen iPad MACs to kitchen printer IPs only
    3. Block iPad access to servers, NAS, and staff resources
    4. Allow internet (80/443) for app updates
  • Time to Resolve: Pending onsite visit
  • Lessons Learned: Non-PHI devices should still be isolated to prevent lateral movement into PHI networks.

2026-03-10 - ALIS BAA Status Unknown

  • Reported By: HIPAA compliance review (2026-03-10)
  • Severity: Medium
  • Symptoms: Nurses and medtechs access clinical records via ALIS (go-alis.com) through web browsers. ALIS is a cloud SaaS handling PHI. HIPAA §164.308(b)(1) requires a BAA with ALIS as a business associate. Unknown if Cascades has a signed BAA with ALIS.
  • Root Cause: Documentation gap — BAA status never verified.
  • Resolution: OPEN — Ask management if a BAA exists with ALIS. If not, one must be executed before continuing to use the service.
  • Time to Resolve: Pending management verification
  • Lessons Learned: N/A

2026-03-19 - Lauren Hasselman Needs Sales Share Access

  • Reported By: Howard
  • Severity: Low
  • Symptoms: Lauren Hasselman (Business Office Director) does not have access to the Sales share on CS-SERVER.
  • Root Cause: Permissions not granted when Lauren replaced Jeff Bristol.
  • Resolution: OPEN — Grant Lauren access to \CS-SERVER\Sales share. Add her AD account to the appropriate security group or add NTFS permissions directly.
  • Time to Resolve: Pending
  • Lessons Learned: N/A

2026-03-19 - Long Print Lag in Room 405

  • Reported By: Howard (staff report)
  • Severity: Medium
  • Symptoms: Resident room 405 experiencing long delays when printing. Specific printer unknown — likely a resident room printer.
  • Root Cause: Unknown — could be WiFi signal, printer sleep mode, DNS resolution delay, or network path issue through per-room VLAN routing.
  • Resolution: OPEN — Investigate onsite:
    1. Identify which printer room 405 is printing to
    2. Check network path (VLAN 10.4.5.0/28 → printer IP)
    3. Test ping/latency from room AP to printer
    4. Check printer sleep/power save settings
    5. Verify DNS resolution if printing by hostname
  • Time to Resolve: Pending onsite visit
  • Lessons Learned: N/A
Reference in New Issue View Git Blame Copy Permalink