# IMC1 Maintenance — Ticket Notes (2026-04-11 through 2026-04-13)

**Client:** Instrumental Music Center (IMC)
**Server:** IMC1 (Windows Server 2016, AD DC + AIMsi SQL host + RDS), 192.168.0.2
**Dates worked:** Saturday 2026-04-11, Sunday 2026-04-12, finalized Monday 2026-04-13

---

## Original request

Remove RDS role from IMC1 in preparation for a planned Server 2019 upgrade.

## Outcome (short version)

RDS role **could not be removed** — blocked by an underlying Windows component-store corruption that also affects Cumulative Update apply-on-boot. The 2019 upgrade path has to be reconsidered.

Parallel/opportunistic work completed while diagnosing: **716 GB recovered on E:** (stale SQL backup cleanup), **GFS retention automated**, **AIMsi databases moved to dedicated SSD (S:)**, **out-of-band SSH access set up** for future work.

---

## Work performed — billable

### 1. Remote-access groundwork
- Installed OpenSSH Server on IMC1 from official GitHub release (Windows built-in `Add-WindowsCapability` install was a ghost — binaries never landed, also a symptom of the component-store corruption)
- Registered `sshd` + `ssh-agent` services, opened TCP/22 in the firewall
- Configured key-based auth (ed25519) via `C:\ProgramData\ssh\administrators_authorized_keys` with proper ACLs
- Set PowerShell as default SSH shell
- Identified & documented a routing conflict: Tailscale's `pfsense-2` subnet-router was advertising `192.168.0.0/24` with lower metric than the OpenVPN tunnel, making IMC1 unreachable via VPN. Workaround: disconnect Tailscale when using IMC OpenVPN.

### 2. SQL backup cleanup — 716 GB freed on E:
- Inventoried `E:\SQL\MSSQL14.SQLEXPRESS\MSSQL\Backup\`: **66 nightly full backups, 905 GB total**, covering 2026-02-01 through 2026-04-11
- Verified the off-site Cloudberry/MSP360 backup was intact before deleting locally
- Applied GFS retention manually: kept 14 dailies + 1st-of-month (16 files, 189 GB); deleted the other 50 (716 GB)
- Noted the per-backup size dropping from ~15 GB to ~11 GB around 2026-03-28 (someone archived or cleaned source data that day; unrelated to this work)

### 3. Automated backup retention — going forward
- Wrote `C:\Scripts\Clean-AimsiBackups.ps1` implementing the same GFS policy (14 dailies + monthlies on day-1)
- Hard safety guards: minimum of 3 newest files always kept, filename-pattern validation, per-run logs to `C:\Scripts\Logs\aimsi-retention-YYYYMM.log`
- Registered Windows Scheduled Task `IMC AIMsi Backup Retention`: runs daily 23:30 as SYSTEM, 1-hour execution limit
- Test run completed successfully; script will keep backup footprint stable automatically from now on

### 4. AIMsi SQL database relocation (C: → S:)
- Elevated `IMC\guru` to sysadmin on the `AIMSQL` instance via single-user-mode recovery (only Windows admins had sysadmin by default; `IMC\guru` wasn't in that group historically)
- Moved the three user databases from `C:` to the dedicated Samsung 850 PRO SSD (`S:\SQL\Data\`):
  - `AIM.mdf` — 8.6 GB — production AIMsi database
  - `IMC.mdf` — 9.8 GB — legacy (usage unclear; kept for safety)
  - `TestConv61223.mdf` — 8.8 GB — leftover from a 2023-06-12 migration test; candidate for drop
- Moved `tempdb` + cleaned up its orphaned files on C:
- Left system DBs (`master`, `model`, `msdb`) on C: — moving `master` requires modifying SQL Server startup parameters and the benefit is marginal vs risk
- Verified AIM client launch after the move; 4 typical concurrent users connected fine
- **Disk impact:** `C:` 322 → 278 GB used (−44 GB); `S:` 27 → 53 GB used (+26 GB)

### 5. RDS removal (parked — deeper than scoped)
Root error during `Uninstall-WindowsFeature RDS-RD-Server`: `0x80073701 ERROR_SXS_ASSEMBLY_MISSING`.

Things we tried (in order):
1. `DISM /Online /Cleanup-Image /RestoreHealth` → failed Error 14 (internally `E_OUTOFMEMORY 0x8007000e` from an oversized 168 MB `COMPONENTS` registry hive — normal is 30–50 MB)
2. Same with explicit `/ScratchDir` → failed `E_ACCESSDENIED` because BITS + `wuauserv` were stopped
3. Started BITS/wuauserv and retried → BITS idles-and-auto-stops on Server 2016; DISM still couldn't fetch payloads
4. `/Source:WIM:E:\W2016\sources\install.wim:2 /LimitAccess` (install media on E:) → failed `CBS_E_SOURCE_MISSING`. The E:\W2016 image is RTM (14393.0); the damaged assembly is from a post-RTM Cumulative Update, so the RTM source doesn't have the right version
5. Extracted KB5075999 (Feb 2026 CU) from a local MSU and ran `DISM /Online /Add-Package` → **staged successfully (S_OK)**, but on reboot the apply phase failed with `HRESULT_FROM_WIN32(15010) ERROR_EVT_INVALID_EVENT_DATA` at `onecore\admin\wmi\events\config\manproc.cpp:733` — the ETW event manifest for provider GUID `{9c2a37f3-e5fd-5cae-bcd1-43dafeee1ff0}` is malformed → `CBS_E_INSTALLERS_FAILED` → full rollback

**Decision:** parked. The component-store corruption is isolated (server otherwise healthy) but it prevents both RDS removal AND CU application. It's deeper than the originally scoped RDS-removal work, and needs a planning conversation before committing more time.

### 6. Minor housekeeping
- Re-created the missing `C:\Users\guru\Downloads` folder (registry pointed there, the folder itself was gone)

---

## Current state of IMC1

| Area | State |
|---|---|
| AIMsi production | healthy, clients connecting |
| SQL instance | running; user DBs on S:, system DBs on C: |
| SQL backups | retained by scheduled script; 189 GB footprint (vs prior 905 GB) |
| C: drive | 278/419 GB used (~66%) — comfortable |
| S: drive | 53/238 GB used |
| RDS role | still installed (removal blocked) |
| OS updates | latest successful CU prior to this session; KB5075999 rolled back |
| SSH access | working, key-based, admin group |

---

## Open items / recommendations (for the next conversation)

**Decision needed on the Server 2019 upgrade path.** Three realistic options:

| Path | Rough effort | Notes |
|---|---|---|
| **A. Repair the component store, then retry RDS removal + in-place 2019 upgrade** | 2–4 hours, uncertain | Next step is to identify which KB owns the malformed provider GUID `{9c2a37f3-e5fd-5cae-bcd1-43dafeee1ff0}`, re-register its manifest via `wevtutil im`, retry. If the hive is just too damaged, this path fails the same way. |
| **B. In-place Server 2019 upgrade without fixing the corruption** | 2–3 hours attempt; high rollback risk | Microsoft's in-place upgrade rewrites system files wholesale; in practice it often resolves corruptions like this one, but if the upgrade itself encounters the same provider-manifest issue, we're worse off. |
| **C. Clean Server 2019 build + AD / SQL / file share / RDS migration** | 6–10 hours | Lowest risk of mid-flight breakage. Most predictable. Requires a cutover window. This is what I'd recommend if the upgrade is planned anyway. |

**Also pending (independent of the upgrade):**
- Verify the `IMC` database (9.8 GB) is actively being used; drop if not
- Drop `TestConv61223` (8.8 GB leftover from 2023-06-12 migration test) once confirmed unused
- Disable SMB1: `Set-SmbServerConfiguration -EnableSMB1Protocol $false` (currently enabled — security hygiene)
- `IMC2`, `IMC-VM` in AD — last-logon 2023 and 2021 respectively, likely decommissioned; clean up if confirmed
- `SERVERIMC` (192.168.0.63) — its role/status is unclear from AD, worth verifying what it's doing

---

## Key paths and references (for the next tech)

- SSH: `ssh IMC\guru@192.168.0.2` (ed25519 key, PowerShell shell)
- Authorized keys file: `C:\ProgramData\ssh\administrators_authorized_keys`
- Retention script: `C:\Scripts\Clean-AimsiBackups.ps1`
- Retention logs: `C:\Scripts\Logs\aimsi-retention-YYYYMM.log`
- Scheduled task: `IMC AIMsi Backup Retention` (daily 23:30, SYSTEM)
- SQL instance: `IMC1\AIMSQL` (MSSQL15 = SQL Server 2019 Express, despite folder name)
- SQL data: `S:\SQL\Data\` (user DBs); `C:\Program Files\Microsoft SQL Server\MSSQL15.AIMSQL\MSSQL\DATA\` (system DBs)
- SQL backups: `E:\SQL\MSSQL14.SQLEXPRESS\MSSQL\Backup\` (yes, MSSQL14 folder hosting a MSSQL15 instance's backups — legacy)
- DISM scratch + extracted KB payload: `C:\DISMScratch\`
- Server 2016 install media (RTM): `E:\W2016\sources\install.wim`
- Full session log: `clients/instrumental-music-center/session-logs/2026-04-12-imc1-cleanup-and-sql-move.md`

## Credentials

- Domain admin + AIMSQL sysadmin: `IMC\guru` (password handled verbally, not stored in this file)
- Local admin + SSH: same account
- `sa` on AIMSQL: exists and enabled; password unknown (tried one candidate, wrong, no lockout triggered)
