azcomputerguru/claudetools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

imc: Manda laptop provision (DESKTOP-KRHQ5TS) + ServerIMC phantom-DC confirmed

Browse Source 
- New laptop provisioned onsite at IMC Speedway: joined to imc.local, AD
  account created for Manda (incoming GM), Outlook bound to her M365
  mailbox, Office activated via retail key, AIMsi USER#=4 per Leslie.
- Syncro ticket #32218 invoiced — 1.5 hrs Onsite Business labor debited
  from IMC's prepay block (14.0 -> 12.5 hrs).
- ServerIMC (192.168.0.63) confirmed as a real authentication-degrading
  phantom DC: SRV/A records claim it's a DC; LDAP/Kerberos refuse
  connections. Promoted from "unclear, worth verifying" (2026-04-13) to
  confirmed AD hygiene issue. Was the root cause of the 2026-04-22 remote
  domain-join failure. Needs follow-up ticket: repair or ntdsutil cleanup.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Howard Enos
2026-04-28 11:10:29 -07:00
parent 00fa539e4f
commit b83c024ad2
2 changed files with 109 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
clients/instrumental-music-center/PROJECT_STATE.md
View File

@@ -2,7 +2,7 @@
> READ THIS before starting work on this client. > READ THIS before starting work on this client.
> UPDATE THIS when you begin work (claim a lock) and when you finish (release lock + log changes). > UPDATE THIS when you begin work (claim a lock) and when you finish (release lock + log changes).
> Last updated: 2026-04-20 > Last updated: 2026-04-28
--- ---
@@ -19,9 +19,15 @@
## Current State ## Current State
**Status:** ACTIVE **Status:** ACTIVE
**Last Activity:** 2026-04-13 **Last Activity:** 2026-04-28
Music retail + repair shop running AIMsi POS on-prem. Primary server IMC1 (Dell R720, Windows Server 2016, DC for IMC.local). SQL Server 2019 Express (`IMC1\AIMSQL`). AIMsi DBs moved to dedicated SSD (`S:\SQL\Data\`). Local SQL backups running nightly; Cloudberry off-site backups configured. Known issue: component store corruption (0x80073701) blocking RDS role removal — Server 2019 migration decision pending. Music retail + repair shop running AIMsi POS on-prem. Primary server IMC1 (Dell R720, Windows Server 2016, DC for IMC.local). SQL Server 2019 Express (`IMC1\AIMSQL`). AIMsi DBs moved to dedicated SSD (`S:\SQL\Data\`). Local SQL backups running nightly; Cloudberry off-site backups configured.
**Personnel:** Manda is the new General Manager (replacing Michael Santander, already deactivated). Manda's new laptop `DESKTOP-KRHQ5TS` provisioned 2026-04-28 (AIMsi `USER#=4` per Leslie).
**Known issues:**
- `IMC1` component store corruption (0x80073701) blocking RDS role removal — Server 2019 migration decision pending.
- **`ServerIMC` (192.168.0.63) — phantom/broken DC.** Registered as a DC in DNS (A + SRV records), responds to ICMP, but **TCP/389 LDAP and TCP/88 Kerberos refuse connections.** The DC locator round-robins between IMC1 and ServerIMC; clients picking ServerIMC time out. **This degrades authentication for every domain user at IMC** — intermittent slow logons, GPO failures, and was the root cause of the 2026-04-22 remote domain-join failure for `DESKTOP-KRHQ5TS`. Needs investigation: real-but-broken DC (repair AD services) or ghost from a demoted DC (`ntdsutil` metadata cleanup). Was flagged as "unclear" on 2026-04-13; promoted to confirmed issue 2026-04-28.
--- ---
@@ -33,9 +39,11 @@ Music retail + repair shop running AIMsi POS on-prem. Primary server IMC1 (Dell
| SQL instance | IMC1\AIMSQL | MSSQL 2019 Express. DBs on S:\SQL\Data\ | | SQL instance | IMC1\AIMSQL | MSSQL 2019 Express. DBs on S:\SQL\Data\ |
**SSH:** `ssh IMC\guru@192.168.0.2` (ed25519 key, PowerShell default shell) **SSH:** `ssh IMC\guru@192.168.0.2` (ed25519 key, PowerShell default shell)
**VPN:** OpenVPN (.ovpn profile) — disconnect Tailscale first (192.168.0.0/24 conflict) **VPN:** OpenVPN (.ovpn profile) — disconnect Tailscale first (192.168.0.0/24 conflict). **Avoid remote domain-join over OpenVPN if your local LAN is also `192.168.0.0/24`** — DNS multi-homing race + the ServerIMC phantom-DC make it unreliable. Go onsite instead.
**Domain admin:** `IMC\guru` — also SQL sysadmin (added via single-user recovery 2026-04-12) **Domain admin:** `IMC\guru` — also SQL sysadmin (added via single-user recovery 2026-04-12)
**Credentials:** Vault entry needed (`clients/imc/` — not yet created as of 2026-04-13) **Syncro customer ID:** 7088508 (prepay block: 12.5 hrs as of 2026-04-28)
**Primary contact:** Leslie Stirm — leslie@imc-az.com (Syncro contact_id 731730)
**Credentials:** vault `clients/imc/imc1.sops.yaml`
**Disks:** C: (OS, 77% full), E: (SQL backups + installers), F: (Windows Image Backups), S: (SSD — AIMsi DBs) **Disks:** C: (OS, 77% full), E: (SQL backups + installers), F: (Windows Image Backups), S: (SSD — AIMsi DBs)
**Backup:** Nightly SQL at 22:00 to `E:\SQL\...\Backup\`; retention script `C:\Scripts\Clean-AimsiBackups.ps1` (14 dailies + 1st-of-month); Cloudberry at `C:\ProgramData\Online Backup\` **Backup:** Nightly SQL at 22:00 to `E:\SQL\...\Backup\`; retention script `C:\Scripts\Clean-AimsiBackups.ps1` (14 dailies + 1st-of-month); Cloudberry at `C:\ProgramData\Online Backup\`
@@ -50,11 +58,19 @@ Music retail + repair shop running AIMsi POS on-prem. Primary server IMC1 (Dell
## Pending / Next Up ## Pending / Next Up
**High priority — AD hygiene**
- [ ] **Open ticket for `ServerIMC` (192.168.0.63) phantom-DC investigation.** SRV/A records claim it's a DC; LDAP/Kerberos refuse connections. Either repair (if a real-but-broken DC) or `ntdsutil` metadata cleanup (if a ghost demoted DC). Degrades authentication for all domain clients. Confirmed root cause of 2026-04-22 remote join failure.
**Documentation cleanup (Manda rollout)**
- [ ] Update `docs/overview.md` Speedway workstation table with `DESKTOP-KRHQ5TS` / Manda / AIM `USER#=4` entry
- [ ] Confirm Manda's full name in AD; add to overview/contacts
**Pre-existing**
- [ ] Decide Server 2019 migration path (in-place upgrade vs. clean build + migrate) - [ ] Decide Server 2019 migration path (in-place upgrade vs. clean build + migrate)
- [ ] Verify Cloudberry backup continuity - [ ] Verify Cloudberry backup continuity
- [ ] Drop `TestConv61223` DB after confirming nothing references it - [ ] Drop `TestConv61223` DB after confirming nothing references it
- [ ] Disable SMB1 on IMC1 - [ ] Disable SMB1 on IMC1
- [ ] Add IMC vault entry for SSH/SQL/domain credentials - [ ] Clean up stale AD computer objects `IMC2` (last logon 2023), `IMC-VM` (last logon 2021)
- [ ] Follow up on disk management items from 2026-04-13 session - [ ] Follow up on disk management items from 2026-04-13 session
--- ---
@@ -63,6 +79,8 @@ Music retail + repair shop running AIMsi POS on-prem. Primary server IMC1 (Dell
| Date | By | Change | Status | | Date | By | Change | Status |
|------|-----|--------|--------| |------|-----|--------|--------|
| 2026-04-28 | Howard | Provisioned `DESKTOP-KRHQ5TS` for Manda (new GM): joined to imc.local onsite, AD account created, Outlook M365 configured, Office activated, AIMsi `USER#=4` per Leslie. Ticket #32218 invoiced, 1.5 hrs from prepay (14.0 → 12.5). Confirmed `ServerIMC` (192.168.0.63) is a real authentication-degrading phantom DC (SRV/A claim DC, LDAP/Kerberos refuse). | DEPLOYED |
| 2026-04-22 | Howard | Attempted remote domain-join of `DESKTOP-KRHQ5TS` over OpenVPN — abandoned after subnet overlap (home Wi-Fi same /24 as IMC) + phantom-DC SRV pollution defeated NRPT/hosts/locator workarounds. Went onsite instead (see 2026-04-28 entry). | ABANDONED |
| 2026-04-12 | Mike | IMC1 cleanup and SQL move: AIMsi DBs moved to S: SSD, backup script + retention task deployed, sysadmin access restored | DEPLOYED | | 2026-04-12 | Mike | IMC1 cleanup and SQL move: AIMsi DBs moved to S: SSD, backup script + retention task deployed, sysadmin access restored | DEPLOYED |
--- ---

85
clients/instrumental-music-center/session-logs/2026-04-28-howard-manda-laptop-provision.md Normal file
View File

@@ -0,0 +1,85 @@
# 2026-04-28 — IMC: New laptop provision for Manda (DESKTOP-KRHQ5TS)
## User
- **User:** Howard Enos (howard)
- **Machine:** Howard-Home (then onsite at IMC Speedway)
- **Role:** tech
## Session Summary
Provisioned `DESKTOP-KRHQ5TS` for Manda, the incoming General Manager at IMC. Joined to `imc.local`, AD account created, Outlook configured against her existing M365 mailbox, Office activated via retail license, AIMsi station number set per Leslie. Ticket #32218 created and invoiced (1.5 hrs against IMC's prepay block).
## Work Performed (onsite at IMC Speedway, 7063 E Speedway Blvd)
- **Domain join** — `DESKTOP-KRHQ5TS` joined to `imc.local`. Original attempt was made remotely from Howard's home over OpenVPN (CG office to IMC) on 2026-04-22; that attempt was abandoned after a stack of DNS/locator issues (see "Issues encountered remotely" below). Onsite join completed without incident.
- **AD account** — Manda created in AD under `imc.local`. Logged in successfully, Windows profile built.
- **Outlook profile** — Configured against Manda's existing M365 mailbox (IMC mixes Google + Microsoft per user; Manda is on the M365 side).
- **Office activation** — Activated using IMC's retail license key.
- **AIMsi station ID** — Set Windows User environment variable `USER#` to `4` per Leslie's instruction. Per Tri-Tech (https://www.tritechretail.com/topic/aim), this is the per-machine workstation identifier AIMsi requires for POS polling and licensing.
## Personnel context (kept out of customer-facing ticket)
Manda is the new General Manager, replacing **Michael Santander**. Michael's domain account and other software access were already deactivated (no offboarding work required this session).
## Issues encountered remotely (not blocking — onsite path worked)
When attempting the domain join remotely over OpenVPN from Howard's home network, Windows would not resolve `imc.local` reliably and `Add-Computer` failed with "domain could not be contacted." Diagnosis surfaced two real environmental problems at IMC, plus one local conflict:
1. **Phantom DC `ServerIMC` (192.168.0.63)** — registered in DNS as a domain controller (A record + SRV records for `_ldap._tcp.dc._msdcs.imc.local` and `_kerberos._tcp.imc.local` both list it alongside `IMC1`), responds to ICMP ping, but **TCP/389 (LDAP) and TCP/88 (Kerberos) refuse connections.** The DC locator round-robins between the two DCs and times out roughly half the time when it picks `ServerIMC`. **This degrades client authentication for every domain user at IMC**, not just remote/VPN clients — intermittent slow logons, GPO failures, etc. Worth a follow-up ticket: investigate whether `ServerIMC` is a real machine that needs repair (AD services down) or a ghost from a demoted DC that needs `ntdsutil` metadata cleanup. Was previously flagged in the 2026-04-13 IMC ticket notes as "role/status unclear, worth verifying" — now confirmed as a real authentication-degrader.
2. **Subnet overlap** — Howard's home Wi-Fi network is `192.168.0.0/24`, the same as IMC's LAN. OpenVPN's pushed routes won (the laptop reached IMC1 via the tunnel correctly) but Windows multi-homed DNS resolution was racing between Wi-Fi's DNS (`192.168.0.1`) and the OpenVPN-set DNS (`192.168.0.2` = IMC1). Wi-Fi's DNS responded with NXDOMAIN faster, and Windows cached the negative answer.
3. Attempted fixes that **did not** resolve it: NRPT rule for `.imc.local`, hosts file entry for `imc1.imc.local`, `Add-Computer -Server <ip>`, `nltest /dsgetdc /force`. The phantom-DC + subnet-overlap combination was beyond what these client-side workarounds could paper over from outside the network.
**Playbook lesson for the future:** when remote-joining a laptop to a client domain over a tunnel, if your local network's subnet overlaps with the client's LAN, go onsite. The route conflicts and DNS race conditions on the laptop add up faster than they're worth fixing.
## Configuration Changes
### IMC AD
- New user `Manda` (full name TBD — verify in AD next session if not already documented)
- New computer object `DESKTOP-KRHQ5TS`
### Laptop `DESKTOP-KRHQ5TS`
- Joined to `imc.local`
- AIMsi `USER#` user environment variable = `4`
- Outlook profile bound to Manda's M365 mailbox
- Microsoft Office activated (retail license)
### Repo
- This session log
- `PROJECT_STATE.md` updated (workstation table + Recent Changes + ServerIMC follow-up)
- `docs/overview.md` to be updated next session with the workstation table entry once the AIM #4 mapping is confirmed at IMC
## Syncro
| Field | Value |
|---|---|
| Ticket # | 32218 (id 109542334) |
| Customer | Instrumental Music Center (id 7088508) |
| Subject | New User / Workstation Deployment - Manda (DESKTOP-KRHQ5TS) |
| Status | Invoiced |
| Contact | Leslie's House — leslie@imc-az.com (id 731730) |
| Billing | 1.5 hrs × `26118` Labor - Onsite Business @ $175/hr → debited from prepay block (14.0 → 12.5 hrs remaining) |
| Invoice # | 67468 (id 1650078027) — total $0 (prepay applied) |
## Pending / Follow-up
### High priority — IMC AD hygiene
- [ ] Open ticket: investigate `ServerIMC` (192.168.0.63) phantom DC. SRV/A records claim it's a DC; LDAP/Kerberos refuse connections; degrades authentication for all domain clients. Either repair (if a real-but-broken DC) or `ntdsutil` metadata cleanup (if a ghost). Was previously flagged 2026-04-13 with status "unclear" — promote to a real ticket.
### Documentation cleanup
- [ ] Update `docs/overview.md` Speedway workstation table with `DESKTOP-KRHQ5TS` / Manda / AIM USER#=4 entry
- [ ] Confirm Manda's full name in AD and add to overview/contacts
- [ ] Add IMC's Syncro customer ID `7088508` to `docs/overview.md` Contract Details (it wasn't documented there; took a Syncro API search to find it this session)
### Earlier IMC pending (carried forward from 2026-04-13)
- Decision on Server 2019 upgrade path for IMC1 (component-store corruption blocks RDS removal + CU apply)
- Verify `IMC` database (9.8 GB) usage, drop if not in use
- Drop `TestConv61223` (8.8 GB leftover migration test) once confirmed unused
- Disable SMB1 on IMC1
- Clean up stale AD computer objects `IMC2`, `IMC-VM` (last-logon 2023 / 2021)
## Reference
- Tri-Tech AIMsi workstation variable docs: https://www.tritechretail.com/topic/aim — variable name `WORKSTATION_ID` (current versions) or `USER#` (older versions). IMC is on the `USER#` convention per Leslie.
- IMC1 (DC + AIMsi SQL host): `192.168.0.2`, Windows Server 2016, domain `imc.local`
- IMC Speedway address: `7063 E Speedway Blvd, Tucson AZ 85710`
- Earlier domain-join troubleshooting (remote, before going onsite): documented in conversation but not committed as a separate session log; key findings preserved above.