Session log: IMC SQL move + DISM repair attempt, VWP RDWeb brute-force incident, Dataforth API planning
- IMC: document 716 GB SQL backup cleanup, retention scheduled task, DB move C:->S:, sysadmin grant via single-user recovery, parked RDS removal after KB5075999 apply rolled back on ETW manifest error - Valleywide: document RDWeb brute-force incident on VWP-QBS, UDM port forward closure, 30-day audit showing no breach, lockout policy restoration - Dataforth: capture Swagger API review and Hoffman Zoom call prep
This commit is contained in:
59
clients/instrumental-music-center/README.md
Normal file
59
clients/instrumental-music-center/README.md
Normal file
@@ -0,0 +1,59 @@
|
||||
# Instrumental Music Center (IMC)
|
||||
|
||||
Music retail + repair shop running AIMsi point-of-sale on-prem.
|
||||
|
||||
## Infrastructure
|
||||
|
||||
### Primary server: IMC1 (192.168.0.2)
|
||||
- **OS:** Windows Server 2016 Standard (build 14393.7426)
|
||||
- **Role:** Domain Controller (IMC.local), file server, AIMsi SQL host, RDS host
|
||||
- **Hardware:** Dell R720, 4 physical cores
|
||||
- **Disks:**
|
||||
- `C:` — OS + IIS + a few apps (419 GB, ~77% full as of 2026-04-13)
|
||||
- `E:` — SQL backups, app installers, Server 2016 install media (`E:\W2016`)
|
||||
- `F:` — Windows Image Backups
|
||||
- `S:` — Dedicated SSD (Samsung 850 PRO 256 GB), now holding AIMsi SQL DBs
|
||||
|
||||
### Access
|
||||
- **SSH:** `ssh IMC\guru@192.168.0.2` (ed25519 key auth; PowerShell default shell)
|
||||
- **VPN:** OpenVPN `.ovpn` profile (subnet issues with Tailscale 192.168.0.0/24 overlap — disconnect Tailscale first)
|
||||
- **Domain admin:** `IMC\guru`
|
||||
- **AIMSQL sysadmin:** `IMC\guru` (added 2026-04-12 via single-user recovery)
|
||||
|
||||
### AIMsi / SQL
|
||||
- **Instance:** `IMC1\AIMSQL` (MSSQL15 = SQL Server 2019 Express, despite folder name)
|
||||
- **Databases on `S:\SQL\Data\`:**
|
||||
- `AIM.mdf` (~8 GB) — production AIMsi database
|
||||
- `IMC.mdf` (~9 GB) — legacy, usage unclear (kept out of caution)
|
||||
- `TestConv61223.mdf` (~8 GB) — leftover from 2023-06-12 migration test; safe to drop
|
||||
- `tempdb.mdf`
|
||||
- **System DBs remain on** `C:\Program Files\Microsoft SQL Server\MSSQL15.AIMSQL\MSSQL\DATA\` (master, model, msdb)
|
||||
|
||||
### Backups
|
||||
- **Local SQL backups:** `E:\SQL\MSSQL14.SQLEXPRESS\MSSQL\Backup\IMCAIM_*.bak` (nightly at 22:00)
|
||||
- **Retention:** Automated via `C:\Scripts\Clean-AimsiBackups.ps1` scheduled task `IMC AIMsi Backup Retention` (daily 23:30, runs as SYSTEM)
|
||||
- **Policy:** Last 14 dailies + 1st-of-month; safety override keeps 3 newest regardless
|
||||
- **Off-site:** Cloudberry/MSP360 "Online Backup" at `C:\ProgramData\Online Backup\`
|
||||
|
||||
### AIM client share
|
||||
- `\\IMC1\AIM` → `S:\AIM` (4 connected users typical)
|
||||
- AIM.exe is a 128 KB launcher; real work happens against `IMC1\AIMSQL`
|
||||
- `RequireSecuritySignature = True` in SMB server config — adds auth overhead
|
||||
|
||||
### Known issues
|
||||
- **Component store corrupted** (0x80073701 during RDS role removal). KB5075999 re-apply succeeds but rolls back on reboot due to ETW manifest error (HRESULT 15010, provider GUID `{9c2a37f3-e5fd-5cae-bcd1-43dafeee1ff0}`)
|
||||
- `RDS removal is blocked` → pending 2019 migration strategy (in-place vs. clean)
|
||||
- Oversized `COMPONENTS` hive (~168 MB, normal is 30-50 MB)
|
||||
- `SMB1 enabled` on server — should disable as security hygiene
|
||||
|
||||
### Other servers in AD
|
||||
- `IMC2` — 2016 Essentials, last logon 2023, likely decommissioned
|
||||
- `IMC-VM` — 2016 Standard, last logon 2021, dead
|
||||
- `SERVERIMC` (192.168.0.63) — SSH-only, 2016 Essentials per AD, state unclear
|
||||
|
||||
## Open work
|
||||
|
||||
- Decide Server 2019 migration path (in-place vs. clean build + migrate)
|
||||
- Consider dropping `TestConv61223` DB after verifying nothing references it
|
||||
- Disable SMB1
|
||||
- Add IMC vault entry for SSH/SQL/domain credentials
|
||||
@@ -0,0 +1,77 @@
|
||||
# Session Log: 2026-04-12 — IMC1 Cleanup, SSH Setup, SQL Move
|
||||
|
||||
## Summary
|
||||
|
||||
Originally engaged to help remove RDS from IMC1 as prep for a Server 2019 upgrade. Removal failed with `0x80073701` (component store corruption). Spent most of the session setting up SSH access, diagnosing the corruption, performing SQL backup cleanup and DB relocation, and ultimately parking the RDS removal as a deeper problem than scoped.
|
||||
|
||||
## Work Completed
|
||||
|
||||
### Remote access
|
||||
- Installed OpenSSH Server on IMC1 via GitHub release (built-in `Add-WindowsCapability` install was a ghost — binaries never landed due to component store corruption)
|
||||
- Registered `sshd` and `ssh-agent` services, opened firewall port 22
|
||||
- Added public key to `C:\ProgramData\ssh\administrators_authorized_keys` with correct ACLs (inheritance off, Administrators + SYSTEM full control)
|
||||
- Set PowerShell as default SSH shell via registry
|
||||
- Diagnosed routing conflict: Tailscale's `pfsense-2` was advertising `192.168.0.0/24` with lower metric than OpenVPN; disconnecting Tailscale restored IMC reachability
|
||||
|
||||
### SQL backup cleanup
|
||||
- Inventoried `E:\SQL\MSSQL14.SQLEXPRESS\MSSQL\Backup\`: 66 AIMsi nightly fulls totaling **905 GB** (Feb 1 → Apr 11, 2026)
|
||||
- Confirmed Cloudberry off-site exists before deletion
|
||||
- Applied GFS retention manually: kept 14 dailies + 1st-of-month (16 files / 189 GB); deleted 50 files / **716 GB freed on E:**
|
||||
- Noted size drop from ~15 GB → ~11 GB around 2026-03-28 suggests someone purged/archived data that day
|
||||
|
||||
### Automated retention
|
||||
- Wrote `C:\Scripts\Clean-AimsiBackups.ps1` implementing GFS policy
|
||||
- Safety: 3-newest override, filename-pattern guard, log to `C:\Scripts\Logs\aimsi-retention-YYYYMM.log`
|
||||
- Registered scheduled task `IMC AIMsi Backup Retention`: daily 23:30, SYSTEM, highest privileges, 1h execution limit
|
||||
- Test ran successfully
|
||||
|
||||
### SQL database relocation (C: → S:)
|
||||
- Elevated `IMC\guru` to sysadmin on `AIMSQL` instance via single-user recovery mode (net stop → `net start MSSQL$AIMSQL /mSQLCMD` → `ALTER SERVER ROLE sysadmin ADD MEMBER` → normal restart)
|
||||
- Moved user databases via `ALTER DATABASE ... SET OFFLINE / MODIFY FILE / SET ONLINE`:
|
||||
- `AIM` (8.6 GB)
|
||||
- `IMC` (9.8 GB)
|
||||
- `TestConv61223` (8.8 GB) — still hanging on; candidate for drop
|
||||
- Moved `tempdb` via `ALTER DATABASE tempdb MODIFY FILE` + service restart; cleaned up orphaned files on C:
|
||||
- Left system DBs (master, model, msdb) on C: — moving `master` requires startup-parameter changes, marginal benefit
|
||||
- **Result:** C: 322→278 GB used, S: 27→53 GB used; AIM client launch tested working
|
||||
|
||||
### Minor fix
|
||||
- Recreated missing `C:\Users\guru\Downloads` folder (registry pointed there, folder didn't exist)
|
||||
|
||||
## RDS Removal / Component Store (parked)
|
||||
|
||||
Root error: `0x80073701 ERROR_SXS_ASSEMBLY_MISSING` on RDS role removal.
|
||||
|
||||
Attempts made:
|
||||
1. `DISM /Online /Cleanup-Image /RestoreHealth` — failed Error 14 (really `E_OUTOFMEMORY 0x8007000e` from oversized 168 MB COMPONENTS hive)
|
||||
2. With explicit `/ScratchDir` — failed `E_ACCESSDENIED` (BITS + wuauserv were stopped; DISM couldn't fetch payloads)
|
||||
3. Started BITS/wuauserv, retried — failed again; BITS idle-auto-stops on Server 2016 (known)
|
||||
4. `/Source:WIM:E:\W2016\sources\install.wim:2 /LimitAccess` — failed `CBS_E_SOURCE_MISSING` (E:\W2016 is RTM 14393.0 media; damaged assembly is from a post-RTM CU)
|
||||
5. Extracted KB5075999 (Feb 2026 CU) from local MSU at `C:\Users\guru\Documents\Downloads\` → `DISM /Add-Package` → **staged successfully (S_OK)** but on reboot, apply phase failed with `HRESULT_FROM_WIN32(15010) ERROR_EVT_INVALID_EVENT_DATA` at `onecore\admin\wmi\events\config\manproc.cpp line 733` — ETW event manifest for provider GUID `{9c2a37f3-e5fd-5cae-bcd1-43dafeee1ff0}` is malformed → `CBS_E_INSTALLERS_FAILED` → full rollback
|
||||
|
||||
Decision: deeper than scoped. Server otherwise healthy. RDS removal is blocking a planned 2019 upgrade.
|
||||
|
||||
## Next actions (for next session)
|
||||
|
||||
- **Decide 2019 upgrade strategy:**
|
||||
- Path A: identify specific KB owning provider GUID `{9c2a37f3-e5fd-5cae-bcd1-43dafeee1ff0}`, re-register its manifest via `wevtutil im`, retry CU apply
|
||||
- Path B: try in-place Server 2019 upgrade despite corruption — OS files get rewritten wholesale
|
||||
- Path C: clean 2019 build + AD/SQL/file/RDS migration
|
||||
- Verify whether `IMC` database (9.8 GB) is actively used; drop if not
|
||||
- Verify `TestConv61223` can be dropped safely (leftover migration test from 2023-06-12)
|
||||
- Disable SMB1 (security hygiene): `Set-SmbServerConfiguration -EnableSMB1Protocol $false`
|
||||
- Add IMC entry to SOPS vault
|
||||
|
||||
## Key Files and Paths
|
||||
|
||||
- SSH key authorized: `C:\ProgramData\ssh\administrators_authorized_keys` (ed25519 `guru@DESKTOP-0O8A1RL`)
|
||||
- Retention script: `C:\Scripts\Clean-AimsiBackups.ps1`
|
||||
- Retention logs: `C:\Scripts\Logs\aimsi-retention-YYYYMM.log`
|
||||
- DISM scratch: `C:\DISMScratch`
|
||||
- Expanded KB5075999 payload: `C:\DISMScratch\KB5075999\`
|
||||
- Local Server 2016 media: `E:\W2016\sources\install.wim` (RTM 14393.0, index 2 = Standard Desktop Experience)
|
||||
|
||||
## Credentials Referenced
|
||||
|
||||
- `IMC\guru` — domain admin, AIMSQL sysadmin. Password handled verbally, not stored here.
|
||||
- `sa` on `AIMSQL` — exists, enabled, password unknown (tried one candidate, failed — no lockout policy was hit)
|
||||
Reference in New Issue
Block a user