# IMC1 Maintenance — Ticket Notes (2026-04-11 through 2026-04-13) **Client:** Instrumental Music Center (IMC) **Server:** IMC1 (Windows Server 2016, AD DC + AIMsi SQL host + RDS), 192.168.0.2 **Dates worked:** Saturday 2026-04-11, Sunday 2026-04-12, finalized Monday 2026-04-13 --- ## Original request Remove RDS role from IMC1 in preparation for a planned Server 2019 upgrade. ## Outcome (short version) RDS role **could not be removed** — blocked by an underlying Windows component-store corruption that also affects Cumulative Update apply-on-boot. The 2019 upgrade path has to be reconsidered. Parallel/opportunistic work completed while diagnosing: **716 GB recovered on E:** (stale SQL backup cleanup), **GFS retention automated**, **AIMsi databases moved to dedicated SSD (S:)**, **out-of-band SSH access set up** for future work. --- ## Work performed — billable ### 1. Remote-access groundwork - Installed OpenSSH Server on IMC1 from official GitHub release (Windows built-in `Add-WindowsCapability` install was a ghost — binaries never landed, also a symptom of the component-store corruption) - Registered `sshd` + `ssh-agent` services, opened TCP/22 in the firewall - Configured key-based auth (ed25519) via `C:\ProgramData\ssh\administrators_authorized_keys` with proper ACLs - Set PowerShell as default SSH shell - Identified & documented a routing conflict: Tailscale's `pfsense-2` subnet-router was advertising `192.168.0.0/24` with lower metric than the OpenVPN tunnel, making IMC1 unreachable via VPN. Workaround: disconnect Tailscale when using IMC OpenVPN. ### 2. SQL backup cleanup — 716 GB freed on E: - Inventoried `E:\SQL\MSSQL14.SQLEXPRESS\MSSQL\Backup\`: **66 nightly full backups, 905 GB total**, covering 2026-02-01 through 2026-04-11 - Verified the off-site Cloudberry/MSP360 backup was intact before deleting locally - Applied GFS retention manually: kept 14 dailies + 1st-of-month (16 files, 189 GB); deleted the other 50 (716 GB) - Noted the per-backup size dropping from ~15 GB to ~11 GB around 2026-03-28 (someone archived or cleaned source data that day; unrelated to this work) ### 3. Automated backup retention — going forward - Wrote `C:\Scripts\Clean-AimsiBackups.ps1` implementing the same GFS policy (14 dailies + monthlies on day-1) - Hard safety guards: minimum of 3 newest files always kept, filename-pattern validation, per-run logs to `C:\Scripts\Logs\aimsi-retention-YYYYMM.log` - Registered Windows Scheduled Task `IMC AIMsi Backup Retention`: runs daily 23:30 as SYSTEM, 1-hour execution limit - Test run completed successfully; script will keep backup footprint stable automatically from now on ### 4. AIMsi SQL database relocation (C: → S:) - Elevated `IMC\guru` to sysadmin on the `AIMSQL` instance via single-user-mode recovery (only Windows admins had sysadmin by default; `IMC\guru` wasn't in that group historically) - Moved the three user databases from `C:` to the dedicated Samsung 850 PRO SSD (`S:\SQL\Data\`): - `AIM.mdf` — 8.6 GB — production AIMsi database - `IMC.mdf` — 9.8 GB — legacy (usage unclear; kept for safety) - `TestConv61223.mdf` — 8.8 GB — leftover from a 2023-06-12 migration test; candidate for drop - Moved `tempdb` + cleaned up its orphaned files on C: - Left system DBs (`master`, `model`, `msdb`) on C: — moving `master` requires modifying SQL Server startup parameters and the benefit is marginal vs risk - Verified AIM client launch after the move; 4 typical concurrent users connected fine - **Disk impact:** `C:` 322 → 278 GB used (−44 GB); `S:` 27 → 53 GB used (+26 GB) ### 5. RDS removal (parked — deeper than scoped) Root error during `Uninstall-WindowsFeature RDS-RD-Server`: `0x80073701 ERROR_SXS_ASSEMBLY_MISSING`. Things we tried (in order): 1. `DISM /Online /Cleanup-Image /RestoreHealth` → failed Error 14 (internally `E_OUTOFMEMORY 0x8007000e` from an oversized 168 MB `COMPONENTS` registry hive — normal is 30–50 MB) 2. Same with explicit `/ScratchDir` → failed `E_ACCESSDENIED` because BITS + `wuauserv` were stopped 3. Started BITS/wuauserv and retried → BITS idles-and-auto-stops on Server 2016; DISM still couldn't fetch payloads 4. `/Source:WIM:E:\W2016\sources\install.wim:2 /LimitAccess` (install media on E:) → failed `CBS_E_SOURCE_MISSING`. The E:\W2016 image is RTM (14393.0); the damaged assembly is from a post-RTM Cumulative Update, so the RTM source doesn't have the right version 5. Extracted KB5075999 (Feb 2026 CU) from a local MSU and ran `DISM /Online /Add-Package` → **staged successfully (S_OK)**, but on reboot the apply phase failed with `HRESULT_FROM_WIN32(15010) ERROR_EVT_INVALID_EVENT_DATA` at `onecore\admin\wmi\events\config\manproc.cpp:733` — the ETW event manifest for provider GUID `{9c2a37f3-e5fd-5cae-bcd1-43dafeee1ff0}` is malformed → `CBS_E_INSTALLERS_FAILED` → full rollback **Decision:** parked. The component-store corruption is isolated (server otherwise healthy) but it prevents both RDS removal AND CU application. It's deeper than the originally scoped RDS-removal work, and needs a planning conversation before committing more time. ### 6. Minor housekeeping - Re-created the missing `C:\Users\guru\Downloads` folder (registry pointed there, the folder itself was gone) --- ## Current state of IMC1 | Area | State | |---|---| | AIMsi production | healthy, clients connecting | | SQL instance | running; user DBs on S:, system DBs on C: | | SQL backups | retained by scheduled script; 189 GB footprint (vs prior 905 GB) | | C: drive | 278/419 GB used (~66%) — comfortable | | S: drive | 53/238 GB used | | RDS role | still installed (removal blocked) | | OS updates | latest successful CU prior to this session; KB5075999 rolled back | | SSH access | working, key-based, admin group | --- ## Open items / recommendations (for the next conversation) **Decision needed on the Server 2019 upgrade path.** Three realistic options: | Path | Rough effort | Notes | |---|---|---| | **A. Repair the component store, then retry RDS removal + in-place 2019 upgrade** | 2–4 hours, uncertain | Next step is to identify which KB owns the malformed provider GUID `{9c2a37f3-e5fd-5cae-bcd1-43dafeee1ff0}`, re-register its manifest via `wevtutil im`, retry. If the hive is just too damaged, this path fails the same way. | | **B. In-place Server 2019 upgrade without fixing the corruption** | 2–3 hours attempt; high rollback risk | Microsoft's in-place upgrade rewrites system files wholesale; in practice it often resolves corruptions like this one, but if the upgrade itself encounters the same provider-manifest issue, we're worse off. | | **C. Clean Server 2019 build + AD / SQL / file share / RDS migration** | 6–10 hours | Lowest risk of mid-flight breakage. Most predictable. Requires a cutover window. This is what I'd recommend if the upgrade is planned anyway. | **Also pending (independent of the upgrade):** - Verify the `IMC` database (9.8 GB) is actively being used; drop if not - Drop `TestConv61223` (8.8 GB leftover from 2023-06-12 migration test) once confirmed unused - Disable SMB1: `Set-SmbServerConfiguration -EnableSMB1Protocol $false` (currently enabled — security hygiene) - `IMC2`, `IMC-VM` in AD — last-logon 2023 and 2021 respectively, likely decommissioned; clean up if confirmed - `SERVERIMC` (192.168.0.63) — its role/status is unclear from AD, worth verifying what it's doing --- ## Key paths and references (for the next tech) - SSH: `ssh IMC\guru@192.168.0.2` (ed25519 key, PowerShell shell) - Authorized keys file: `C:\ProgramData\ssh\administrators_authorized_keys` - Retention script: `C:\Scripts\Clean-AimsiBackups.ps1` - Retention logs: `C:\Scripts\Logs\aimsi-retention-YYYYMM.log` - Scheduled task: `IMC AIMsi Backup Retention` (daily 23:30, SYSTEM) - SQL instance: `IMC1\AIMSQL` (MSSQL15 = SQL Server 2019 Express, despite folder name) - SQL data: `S:\SQL\Data\` (user DBs); `C:\Program Files\Microsoft SQL Server\MSSQL15.AIMSQL\MSSQL\DATA\` (system DBs) - SQL backups: `E:\SQL\MSSQL14.SQLEXPRESS\MSSQL\Backup\` (yes, MSSQL14 folder hosting a MSSQL15 instance's backups — legacy) - DISM scratch + extracted KB payload: `C:\DISMScratch\` - Server 2016 install media (RTM): `E:\W2016\sources\install.wim` - Full session log: `clients/instrumental-music-center/session-logs/2026-04-12-imc1-cleanup-and-sql-move.md` ## Credentials - Domain admin + AIMSQL sysadmin: `IMC\guru` (password handled verbally, not stored in this file) - Local admin + SSH: same account - `sa` on AIMSQL: exists and enabled; password unknown (tried one candidate, wrong, no lockout triggered)