# Instrumental Music Center (IMC) — Project State

> READ THIS before starting work on this client.
> UPDATE THIS when you begin work (claim a lock) and when you finish (release lock + log changes).
> Last updated: 2026-06-28

---

## Active Session Locks

| Session | Working On | Status | Started |
|---------|-----------|--------|---------|
| _(none active)_ | | | |

**How to claim a lock:** Add a row before starting work. Remove it when done. Locks older than 2 hours with no update are considered stale.

---

## Current State

**Status:** ACTIVE
**Last Activity:** 2026-04-28

Music retail + repair shop running AIMsi POS on-prem. Primary server IMC1 (Dell R720, Windows Server 2016, DC for IMC.local). **Production AIM database lives on `IMC1\SQLEXPRESS` (TCP 61151), DB `IMCAIM`, service account `IMC\AIM` — note the instance name is misleading: `SERVERPROPERTY('Edition')` confirms SQL Server 2019 Standard, not Express.** A second instance `IMC1\AIMSQL` (true SQL 2019 Express, port 63116) is an orphan from a 2023 conversion test — hosts only legacy `AIM` / `IMC` / `TestConv61223` DBs, zero active clients (verified 2026-05-06). AIMsi DBs on dedicated SSD (`S:\SQL\Data\`). Local SQL backups nightly; Cloudberry off-site backups configured.

**Personnel:** Manda is the new General Manager (replacing Michael Santander, already deactivated). Manda's new laptop `DESKTOP-KRHQ5TS` provisioned 2026-04-28 (AIMsi `USER#=4` per Leslie).

**Known issues:**
- **[UPDATE HOLD — 2026-06-28] Windows Update quality updates are intentionally blocked on IMC1.** The component-store corruption (malformed ETW manifest, provider GUID `{9c2a37f3-e5fd-5cae-bcd1-43dafeee1ff0}`) makes EVERY monthly Server 2016 CU stage S_OK then fail apply-on-boot (`HRESULT 15010` -> `CBS_E_INSTALLERS_FAILED` -> full rollback), leaving the box ~2 hrs in failed-apply + restore. Per Mike: CU **KB5094122** (2026-06 CU, the current offender; KB5075999 from Feb is superseded) was **hidden** via the WU COM API, AND `HKLM\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate\AU\NoAutoUpdate=1` set so no quality update auto-installs. **REVERSE this hold (delete NoAutoUpdate / unhide) once the component store is repaired or IMC1 is migrated to Server 2019.** Manual `Check for updates` / install still works for deliberate patching.
- `IMC1` component store corruption (0x80073701) blocking RDS role removal — Server 2019 migration decision pending. **Root cause of the update hold above.**
- **`ServerIMC` (192.168.0.63) — phantom/broken DC.** Registered as a DC in DNS (A + SRV records), responds to ICMP, but **TCP/389 LDAP and TCP/88 Kerberos refuse connections.** The DC locator round-robins between IMC1 and ServerIMC; clients picking ServerIMC time out. **This degrades authentication for every domain user at IMC** — intermittent slow logons, GPO failures, and was the root cause of the 2026-04-22 remote domain-join failure for `DESKTOP-KRHQ5TS`. Needs investigation: real-but-broken DC (repair AD services) or ghost from a demoted DC (`ntdsutil` metadata cleanup). Was flagged as "unclear" on 2026-04-13; promoted to confirmed issue 2026-04-28.

---

## Infrastructure / Access

| Resource | Address | Notes |
|----------|---------|-------|
| IMC1 (primary server) | 192.168.0.2 | Windows Server 2016, DC (IMC.local), Dell R720 |
| SQL instance — PRODUCTION | IMC1\SQLEXPRESS (TCP 61151) | **SQL 2019 Standard** (misleading instance name). DB `IMCAIM`. Service account `IMC\AIM`. Workstations connect here. |
| SQL instance — orphan | IMC1\AIMSQL (TCP 63116) | SQL 2019 Express GDR 15.0.2165.1. Holds legacy 2023 conversion DBs (`AIM`, `IMC`, `TestConv61223`). No active clients. Consolidation candidate. |
| SQL instance — system | IMC1\MICROSOFT##WID | Windows Internal DB (WSUS / AD RMS). Pages out under host pressure (17890 events). Consolidation candidate if WSUS unused. |

**SSH:** `ssh IMC\guru@192.168.0.2` (ed25519 key, PowerShell default shell)
**VPN:** OpenVPN (.ovpn profile) — disconnect Tailscale first (192.168.0.0/24 conflict). **Avoid remote domain-join over OpenVPN if your local LAN is also `192.168.0.0/24`** — DNS multi-homing race + the ServerIMC phantom-DC make it unreliable. Go onsite instead.
**Domain admin:** `IMC\guru` — also SQL sysadmin (added via single-user recovery 2026-04-12)
**Syncro customer ID:** 7088508 (prepay block: 12.5 hrs as of 2026-04-28)
**Primary contact:** Leslie Stirm — leslie@imc-az.com (Syncro contact_id 731730)
**Credentials:** vault `clients/imc/imc1.sops.yaml`

**Disks:** C: (OS, 77% full), E: (SQL backups + installers), F: (Windows Image Backups), S: (SSD — AIMsi DBs)
**Backup:** Nightly SQL at 22:00 to `E:\SQL\...\Backup\`; retention script `C:\Scripts\Clean-AimsiBackups.ps1` (14 dailies + 1st-of-month); Cloudberry at `C:\ProgramData\Online Backup\`

**Known issues:**
- Component store corrupted (0x80073701) — blocks RDS role removal; ETW manifest error on reboot
- C: drive at ~77% full — monitor
- SMB1 still enabled — disable when time permits
- `TestConv61223` DB (on orphan `IMC1\AIMSQL`, leftover from 2023 conversion test) safe to drop — verify first
- **AIM "connection broken" recurrence under host memory pressure.** SQLEXPRESS has no `max server memory` cap; under pressure Windows trims working sets and reaps idle Telerik pool slots, causing user-facing `Telerik.OpenAccess` connection-broken errors. First seen 2026-05-05 (Station 1 = `IMC-STATION1`, **192.168.0.50**), recurred 2026-05-06 ~12:14 PM (same workstation). Recommended fix: cap `max server memory` on SQLEXPRESS (~12-14 GB), WID (~512 MB), and AIMSQL (~256 MB). Long-term: dedicated DB host or Server 2019 migration. Note: misleading instance name burned ~1 day of triage — do NOT stop SQLEXPRESS thinking it's idle.

---

## Pending / Next Up

**High priority — AD hygiene**
- [ ] **Open ticket for `ServerIMC` (192.168.0.63) phantom-DC investigation.** SRV/A records claim it's a DC; LDAP/Kerberos refuse connections. Either repair (if a real-but-broken DC) or `ntdsutil` metadata cleanup (if a ghost demoted DC). Degrades authentication for all domain clients. Confirmed root cause of 2026-04-22 remote join failure.

**Documentation cleanup (Manda rollout)**
- [ ] Update `docs/overview.md` Speedway workstation table with `DESKTOP-KRHQ5TS` / Manda / AIM `USER#=4` entry
- [ ] Confirm Manda's full name in AD; add to overview/contacts

**Pre-existing**
- [ ] Decide Server 2019 migration path (in-place upgrade vs. clean build + migrate)
- [ ] Verify Cloudberry backup continuity
- [ ] Drop `TestConv61223` DB after confirming nothing references it
- [ ] Disable SMB1 on IMC1
- [ ] Clean up stale AD computer objects `IMC2` (last logon 2023), `IMC-VM` (last logon 2021)
- [ ] Follow up on disk management items from 2026-04-13 session

---

## Recent Changes

| Date | By | Change | Status |
|------|-----|--------|--------|
| 2026-06-28 | Howard | **Blocked failing Windows CU on IMC1** (per Mike): hid CU **KB5094122** (2026-06 Server 2016 CU) via WU COM API + set `NoAutoUpdate=1` policy, restarted `wuauserv`. Both verified. Stops the recurring ~2-hr stage->reboot->apply-fail->rollback caused by the component-store corruption. **NOTE: IMC1 GuruRMM agent UUID re-enrolled to `88cbf7c0-abfa-4f12-846c-96274f718bff`** (old `fa99e913-...` is stale). Component-store repair planned next. | DEPLOYED |
| 2026-05-06 | Howard | AIM connection-broken recurrence (Station 1, ~12:14 PM). Re-enumerated all 3 SQL instances on IMC1; corrected wrong-instance diagnosis from 2026-05-05 (production AIM is on `IMC1\SQLEXPRESS` Standard, not `IMC1\AIMSQL` Express). Confirmed AIMSQL is orphan (zero active clients, 2023-era DBs only). Unregistered the daily 02:30 AIMSQL restart task (it was restarting the wrong instance). Audit artifacts left on disk at `C:\Windows\Temp\aimsql-restart.{ps1,log}`. No service touches. | DIAGNOSED |
| 2026-05-05 | Howard | Initial AIM "connection broken" diagnosis on Station 1. GuruRMM client/site provisioned, IMC1 enrolled (agent `fa99e913-1027-4e33-a928-7695e31068e7`). Scheduled `MSSQL$AIMSQL` restart for 02:30 — fired clean but had no effect (wrong instance, see 2026-05-06 entry). | SUPERSEDED |
| 2026-04-28 | Howard | Provisioned `DESKTOP-KRHQ5TS` for Manda (new GM): joined to imc.local onsite, AD account created, Outlook M365 configured, Office activated, AIMsi `USER#=4` per Leslie. Ticket #32218 invoiced, 1.5 hrs from prepay (14.0 → 12.5). Confirmed `ServerIMC` (192.168.0.63) is a real authentication-degrading phantom DC (SRV/A claim DC, LDAP/Kerberos refuse). | DEPLOYED |
| 2026-04-22 | Howard | Attempted remote domain-join of `DESKTOP-KRHQ5TS` over OpenVPN — abandoned after subnet overlap (home Wi-Fi same /24 as IMC) + phantom-DC SRV pollution defeated NRPT/hosts/locator workarounds. Went onsite instead (see 2026-04-28 entry). | ABANDONED |
| 2026-04-12 | Mike | IMC1 cleanup and SQL move: AIMsi DBs moved to S: SSD, backup script + retention task deployed, sysadmin access restored | DEPLOYED |

---

## How to Update

**When starting:** Add your session to Active Session Locks.
**When finishing:** Remove your lock row, add entries to Recent Changes, update Current State if needed.
