Files

Mike Swanson b583aeed21 wiki: seed Instrumental Music Center + Valley Wide Plastering articles

instrumental-music-center.md — AIMsi POS on SQL Server 2019 (Standard
under misleading SQLEXPRESS instance name); phantom DC ServerIMC causing
slow logons; GuruRMM enrolled (IMC1 fa99e913); OpenVPN subnet-overlap
hazard; $175/hr prepaid, 12.5 hrs remaining; SQL max server memory fix
approved but unverified applied.

valleywide.md — Valley Wide Plastering; HP DL360 Gen10 VM host + XenServer;
VB6/Access 97 app modernization (130 tables, 791 Crystal Reports, certified
payroll); RDWeb brute-force incident (contained); 11 Yealink phones pending;
iLO requires paramiko (legacy ssh-rsa); $175/hr prepaid, 10 hrs remaining.

wiki/index.md — both clients added to Clients table and Cross-Reference.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-05-24 19:30:27 -07:00

19 KiB

Raw Blame History

type, name, display_name, last_compiled, compiled_by, sources, backlinks

type

name

display_name

last_compiled

compiled_by

sources

backlinks

client

valleywide

Valley Wide Plastering

2026-05-24

DESKTOP-0O8A1RL/claude-main

clients/valleywide/README.md

clients/valleywide/PROJECT_STATE.md

clients/valleywide/session-logs/2026-04-13-rdweb-brute-force-incident.md

clients/valleywide/session-logs/2026-04-22-hp-server-nvram-corruption-emergency.md

clients/valleywide/session-logs/2026-05-12-session.md

clients/valleywide/docs/yealink-phones.md

clients/valleywide/docs/yealink-t54w-recovery-procedure.md

clients/valleywide/app-modernization/CONTEXT.md

clients/valleywide/app-modernization/session-logs/2026-04-27-session.md

clients/valleywide/app-modernization/research/schema-analysis.md

clients/valleywide/app-modernization/source-analysis/D-drive-2026-05-16/SUMMARY.md

clients/valleywide/app-modernization/source-analysis/drive2-2026-05-16/SUMMARY.md

clients/valleywide/app-modernization/source-analysis/drive3-2026-05-16/SUMMARY.md

Valley Wide Plastering

Plastering / stucco subcontractor based in Arizona. Active ACG client. Primary work has been incident response (RDWeb brute-force, power outage recovery) and an ongoing app modernization project for their custom VB6/Access construction ERP.

Profile

Company type: Construction subcontractor (plastering / stucco)
Domain / site identifier: VWP (vwp.local internal AD domain, vwp.us registered external domain, valleywideplastering.com M365 domain)
Contract type: Prepaid hour block
Hours remaining: 10.0 hrs as of 2026-05-12 (after billing 1.5 hrs for HP server emergency). Always live-check Syncro before billing.
Billing rate: $150/hr remote labor (product 1190473 — Labor - Remote Business)
Emergency surcharge pattern: Bill as two line items — 1.0 hr normal + 0.5 hr surcharge. Use product 1190473 for both (NOT product 26184, which bakes in a 1.5x dollar rate that would double-charge prepaid block customers). Results in 1.5 hr block deduction = 150% charge.
Key contact: Shelly Dooley / Valley Wide P (Syncro customer display name)
Syncro customer ID: 31694734
Syncro ticket (2026-05-12 emergency): #32269 (ID: 110159277) — HP server powered off, ADSRVR unreachable. Invoiced; invoice #67594 (ID: 1650271395). Ticket status: Invoiced.
M365 tenant ID: 5c53ae9f-7071-4248-b834-8685b646450f
M365 domain: valleywideplastering.com

Infrastructure

Servers & Services

Host	IP	Role	OS	Notes
HP ProLiant DL360 Gen10 (SN: MXQ80400X4)	(LAN — no static IP documented)	Hypervisor / VM host for ADSRVR	—	iLO at 172.16.9.125 (SSH port 22, legacy ssh-rsa key). Power outage 2026-04-22 caused NVRAM corruption + factory iLO reset. Was found powered-off 2026-05-12; powered on remotely via iLO.
HP iLO	172.16.9.125	Out-of-band management for HP ProLiant	—	SSH port 22. Requires legacy RSA algorithms — modern OpenSSH rejects it. Use paramiko with `disabled_algorithms={'pubkeys': ['rsa-sha2-256', 'rsa-sha2-512']}`. Credentials in vault: `clients/valleywide/`
VWP_ADSRVR	192.168.0.25	Domain Controller for `vwp.local`	Windows Server 2019 Standard (build 17763)	VM on HP ProLiant DL360 Gen10. SSH enabled, key auth working for `vwp\guru` (ed25519, added 2026-04-13). Default shell is cmd.exe — use `powershell -NoProfile -Command` wrappers.
VWP-QBS	172.16.9.169	QuickBooks server + RDS/RemoteApp host	Windows Server 2022 Standard	Physical Dell server (NOT a VM). Has DRAC. Runs IIS (RD Web Access, RD Gateway). Reach from ADSRVR via `Invoke-Command -ComputerName VWP-QBS -Credential` with `vwp\sysadmin` PSCredential — no direct SSH; Kerberos does not forward over SSH double-hop. WinRM on 5985.
Dell DRAC (VWP-QBS)	[undocumented]	Out-of-band management for VWP-QBS Dell	—	DRAC functional as of 2026-04-22; used to force manual boot after power outage. IP not yet documented.
DC1	172.16.9.2	Domain Controller	—	Confirmed up 2026-05-12. Separate from ADSRVR.
XenServer (older Dell)	192.168.0.104	VM hypervisor — hosts BACKUP-SRV, Server 2012 R2, Server 2003	XenServer	Older Dell hardware. Was offline after 2026-04-22 power outage; status resolved. Credentials: `root` / see vault.
UDM (UniFi Dream Machine)	172.16.9.1	Perimeter firewall, OpenVPN server, DHCP, DNS, site router	UniFi OS	DNS override: `vwp-qbs.vwp.us` → 172.16.9.169 (static record in UDM dnsmasq). VPN pushes DNS=192.168.4.1 (UDM). WireGuard site-to-site peers present (wgsts1001, wgsts1003, wgsts1005 — likely UniFi SiteMagic).

[WARNING] No UPS on HP ProLiant DL360. The 2026-04-22 power outage caused NVRAM corruption. A UPS assessment is an outstanding priority item — hardware failure from power event is a proven risk.

Email & Identity

M365 tenant: valleywideplastering.com | Tenant ID: 5c53ae9f-7071-4248-b834-8685b646450f
On-prem AD domain: vwp.local (internal). External registered domain: vwp.us (used for internal FQDNs like vwp-qbs.vwp.us).
MFA status: [unverified] — No M365 CA or MFA configuration documented. Not investigated.
MX / mail flow: [unverified] — M365 tenant confirmed but mail flow not audited.

Network

ISP / WAN: Public WAN IP 98.168.18.21 (observed via Yealink YMCS last-seen registrar)
Firewall / Router: UniFi Dream Machine at 172.16.9.1
VPN: OpenVPN on UDM. Client pool: 192.168.4.0/24. Pushes routes for 172.16.9.0/24, 192.168.0.0/24, 192.168.3.0/24. DNS pushed as 192.168.4.1 (UDM).
Subnets:
- 172.16.9.0/24 — primary internal network (servers, Dell VWP-QBS, UDM, iLO)
- 192.168.0.0/24 — secondary internal (AD server, Yealink phones) [WARNING: conflicts with IMC's LAN — be careful when switching VPN contexts between clients]
- 192.168.4.0/24 — OpenVPN client pool
Static DNS (UDM): vwp-qbs.vwp.us → 172.16.9.169 (fixed typo from qwp-qbs.vwp.us on 2026-04-16)

RDS / RemoteApp

Session host: VWP-QBS (Windows Server 2022)
Mode: VPN-only (direct connect, no RD Gateway). Gateway was removed from the deployment 2026-04-16 after the RDWeb public exposure was closed. RDP manifests write gatewayusagemethod:i:0.
RDS Licensing: Per User mode. License server pointed at vwp-qbs.vwp.us (the same box — RDS-Licensing role was installed and activated on 2026-04-16 but had no real CALs).
[WARNING] RDS CALs not purchased. VWP-QBS license server has only the Built-in TS Per Device CAL placeholder. Users will start seeing "no licenses available" errors once grace period expires. Action: purchase Windows Server 2022 RDS Per User CALs, sized to active user count (check distinct interactive logons last 30 days via licmgr.msc).
Application: QuickBooks RemoteApp. VPN clients resolve vwp-qbs.vwp.us via UDM dnsmasq override and connect directly.

Voice / IP Phones

Fleet: 16x Yealink SIP-T54W color IP phones (OUIs 805e0c and 44dbd2)
YMCS portal: https://us.ymcs.yealink.com/manager/sip-product/sipManage — account: Valleywide Plastering (VWP)
YMCS admin password: vault — clients/valleywide/ (Yealink password documented 2026-04-22)
Status as of 2026-04-22: 5 phones previously provisioned (Offline in YMCS), 11 pending first boot
Named phones: 214-ValleyWidePlastering (extension 214), Reception (front desk, 192.168.0.17)
Phone subnet: 192.168.0.0/24 — phones on DHCP, IPs observed at .17, .54, .130, .140, .222
[WARNING] Known-bad firmware: 96.86.0.20 is a documented T54W brick-maker. Confirm YMCS firmware policy is NOT pushing this version before any mass provisioning.
Recovery procedure: TFTP recovery documented in clients/valleywide/docs/yealink-t54w-recovery-procedure.md. Use Tftpd64 with laptop at 192.168.81.100, phone at 192.168.81.10. Multiple recovery file sets may be needed (NEW RM → OLD RM → SPEAKER variant).

Access

SSH to VWP_ADSRVR: ssh vwp\guru@192.168.0.25 (ed25519 key auth — key added 2026-04-13)
Double-hop to VWP-QBS: Via WinRM — Invoke-Command -ComputerName VWP-QBS -Credential $cred using vwp\sysadmin PSCredential from ADSRVR. SSH won't forward Kerberos for domain double-hop.
HP iLO power management: Paramiko required (not system OpenSSH). SSH to 172.16.9.125:22. Use disabled_algorithms={'pubkeys': ['rsa-sha2-256', 'rsa-sha2-512']}. Command: start system1 to power on.
VWP-QBS DRAC: IP undocumented — needs to be recorded. DRAC functional.
VPN: Connect to VWP OpenVPN (UDM) first; this provides access to both the 172.16.9.0/24 and 192.168.0.0/24 subnets.
Vault paths: clients/valleywide/ (confirmed entries: adsrvr, dc1, udm, xenserver, quickbooks-server-idrac). Access via bash "$VAULT" get-field clients/valleywide/<entry> <field>.

App Modernization Project

VWP's core business application is a custom-built construction ERP. The original developer (known as "Darv") is deceased. The app is hitting the 2GB Jet/Access database file size limit. ACG was engaged to assess modernization feasibility.

Application Stack (Confirmed)

Layer	Technology	Evidence
Frontend / logic	Visual Basic 6.0	`frmPayroll.frm` source file, `.frx` resource files, `VB5!` header in exe
Compilation	P-Code (not Native Code)	Entry point `PUSH+CALL` to ThunRTMain by ordinal — not native binary
Database	MS Access Jet 3.x (.mdb)	`VWP.mdb` version byte 0x00, Access 97 format
Reporting	Crystal Reports 8.5	791 `.rpt` files (per 2026-04-27 archive); Crystl32.OCX import; SCR85Dev installer found
Installer	InstallShield Denali 2021	`Denali2021v1` folder on server
OCX controls	TABCTL32, mscomct2, comdlg32, Flp32a30, odg7, todg7	PE import table

P-Code is the best possible outcome for decompilation. VB Decompiler Pro (~$200) can recover 70-80% of source including form layouts, procedure names, string literals, and all SQL queries. Decompilation was approved as the next step.

Database: VWP.mdb

Current size: 938 MB (last written 2026-04-24). Growth: 671 MB (2020) → 761 MB (2022) → 938 MB (2026). Approaching the 2 GB Jet hard limit.
Format: Jet 3.x / Access 97. Modern ACE/DAO drivers refuse to open it — binary scan was used for schema extraction.
Scale: ~130 production tables spanning a full construction ERP.

Domain Coverage

Domain	Key Tables
Projects & Jobs	tblPROJECT, tblLOTINFO, tblPLANS, tblCHANGE, tblSZONE
Work Orders & Estimating	tblORDERS, tblTAKE, tblMEASURE, tblPlanBill
Inventory & Purchasing	tblINVPRICE, tblINVTRY, tblSUPPLIER, tblPOrder, tblYardOrder
Crew & Payroll	tblCREW, tblHRDAILY, tblPAYHEADER, tblPAYROLL, tblCREWRATE
Certified Payroll	tblCERTIFIED — government / prevailing wage work. HARD requirement.
Accounts Receivable	tblARMASTER, tblARINVOICE, tblARTRANS
Accounts Payable	tblAPMASTER, tblAPTRANS, tblJOBCOST, tblCHECKREC
Positive Pay (3 banks)	tblPosPayVWP, tblPosPayCRD, tblPosPaySWI — fraud-prevention bank integration. HARD dependency.
Scaffold	tblScaffold, tblSC_Crew
Repairs	tblREPAIR, tblRepList
System / Config	tblSECURITY, tblSYSInfo, tblGLAcct

Modernization complexity: HIGH. 791 Crystal Reports files, certified payroll (legal compliance — cannot be dropped), positive pay integration with 3 banks, and full AR/AP/Payroll.

Source Code Status

The production exe (Orders_10A.exe, 13.4 MB) has four shortcuts pointing to it. The original source was on Darv's personal development machine — only one form file (frmPayroll.frm, 32 KB) was found on the server at C:\Users\sysadmin\Desktop\Darv\Source\VWP\. The remainder of C:\Users\sysadmin\Desktop\Darv\ (13,231 files, 15.6 GB) includes Darv's installer projects, Crystal Reports, and personal files. VB6 source (.vbp, .frm) was scanned across multiple server drives (D: and two additional drives as of 2026-05-16). Substantial VB6 source exists across the drives (thousands of .frm and .vbp files); Mike was searching to confirm which are for the VWP application specifically.

Project Status (as of 2026-04-27)

Task	Status
Stack identification	Complete — VB6 P-Code + Jet 3.x confirmed
Schema mapping (table names)	Complete (~130 tables via binary scan)
Full schema with field types	Pending — needs Access 97/2000 environment or Jet 3.x → Jet 4.x conversion
VB6 source search across server drives	In progress — Mike searching
VB Decompiler Pro purchase and run	Pending ($200 investment)
Crystal Reports audit (791 .rpt files)	Pending
VWP staff workflow interviews	Pending
Feasibility / modernization report	Pending

Patterns & Known Issues

iLO Access (Non-Standard)

The HP ProLiant iLO at 172.16.9.125 uses legacy SSH host key algorithms (ssh-rsa/ssh-dss) that are rejected by modern OpenSSH on Windows by default. Do not use system OpenSSH to connect. Use Python paramiko with:

transport.disabled_algorithms = {'pubkeys': ['rsa-sha2-256', 'rsa-sha2-512']}

Power-on command: start system1.

RDS Double-Hop Pattern

SSH to ADSRVR (192.168.0.25) works fine with ed25519 key. But you cannot forward Kerberos over SSH to reach VWP-QBS — the WinRM double-hop must be done inside the SSH session using explicit PSCredential:

$cred = Get-Credential  # vwp\sysadmin
Invoke-Command -ComputerName VWP-QBS -Credential $cred -ScriptBlock { ... }

192.168.0.0/24 Subnet Conflict

VWP's AD/phone subnet (192.168.0.0/24) is the same RFC1918 range as IMC (another ACG client). When switching between client VPN contexts, verify which 192.168.0.x addresses are being targeted. This is a silent risk — wrong subnet = wrong client.

Syncro Billing for Prepaid Block Emergency

Do not use product 26184 (Labor - Emergency) for prepaid block customers. That product has the 1.5x rate baked in, which would result in double-charging when combined with the surcharge line item pattern. Always use product 1190473 for both normal and surcharge line items.

AD Account: `scanner`

The scanner AD account is used by some device or process (original purpose unknown). Its password was last set 2024-10-17. During the 2026-04-13 brute-force incident, it was being locked out every ~20 minutes by attacker attempts through the public-facing RDWeb. Password rotation is an outstanding hygiene item.

LastLogonDate Anomaly

VWP-QBS AD object showed LastLogonDate: 9/28/2049 — flagged as a time-skew artifact during 2026-04-13 incident. Likely cosmetic.

Active Work (as of 2026-05-12)

Item	Status	Priority
App modernization: VB Decompiler Pro run against Orders_10A.exe	Pending — decompiler not yet purchased	High
App modernization: Full schema extraction with field types	Pending — needs Access 97/2000 environment	High
App modernization: VB6 source search across server drives	In progress	High
RDS CAL purchase (Windows Server 2022 Per User, sized to user count)	Outstanding — grace period may expire	High
HP iLO reconfiguration (post factory-reset 2026-04-22)	[unverified — may have been configured during 2026-04-22 onsite; confirm credentials in vault]	Medium
UPS assessment for HP ProLiant	Outstanding since 2026-04-22	Medium
Yealink phone fleet provisioning (11 pending phones)	Outstanding — 11 of 16 phones never connected to YMCS	Medium
`scanner` AD account password rotation	Outstanding since 2026-04-13	Low
UDM UPnP audit	Outstanding since 2026-04-13	Low
DRAC IP documentation for VWP-QBS	Not yet recorded	Low

Security Posture

2026-04-13: RDWeb Brute-Force Incident

RDWeb (https://VWP-QBS/RDWeb/Pages/login.aspx) was publicly exposed via UDM port-forward on port 443. A distributed brute-force botnet (residential proxy infrastructure, IPs from China, Belarus, UAE, and others) was hammering POST /RDWeb/Pages/en-US/login.aspx at ~6 req/min, hitting usernames scanner, Guest, Receptionist. This triggered AD lockouts every ~20 minutes (lockout threshold 5, 16-min window) which initially appeared to be a stale internal credential problem.

Resolution: UDM port-forward removed (same day), IIS reset to drain in-flight sessions, lockout policy restored. 30-day audit of Event 4624 confirmed zero successful external logons — no compromise.

Current state: RDWeb accessible from VPN and internal LAN only (port 443 on VWP-QBS, 172.16.9.0/24). Not reachable from public internet.

Outstanding recommendation: If RDWeb must be re-exposed publicly, require: IPBan (https://github.com/DigitalRuby/IPBan), firewall restriction to known source IPs, and 2FA/Conditional Access.

2026-04-22: Power Outage / NVRAM Corruption

Power outage caused HP ProLiant NVRAM corruption (BIOS/iLO factory reset). VWP-QBS Dell server had a boot retry loop (resolved via DRAC). XenServer (older Dell) was offline. All recovered onsite. Root cause: no UPS on HP server.

History Highlights

Date	Event
2026-04-13	RDWeb brute-force incident discovered and contained. SSH key deployed to ADSRVR. 30-day audit — no compromise.
2026-04-13	Domain lockout policy temporarily disabled during diagnosis (threshold=0), restored to 5/16min/16min. 15-minute window of reduced lockout protection.
2026-04-16	RDS reconfigured to VPN-only (gateway removed). UDM DNS typo fixed (`qwp-qbs` → `vwp-qbs`). RDS licensing mode set Per User, pointed at local license server.
2026-04-22	Emergency onsite: power outage, HP ProLiant NVRAM corruption + iLO factory reset, VWP-QBS boot loop (DRAC), XenServer offline. All resolved ~12:00 MST.
2026-04-22	Yealink SIP-T54W fleet (16 devices) added to YMCS device management. 5 previously-provisioned, 11 pending.
2026-04-27	App modernization project initiated. VB6 P-Code + Jet 3.x stack confirmed. ~130 table schema extracted via binary scan. Crystal Reports 8.5 (791 .rpt files) documented.
2026-05-12	HP ProLiant found powered-off (ADSRVR unreachable). Powered on remotely via iLO paramiko. Syncro ticket #32269, invoice #67594, 1.5 hr block deduction (10.0 hrs remaining).

Compilation Notes

Date range covered: 2026-04-13 through 2026-05-12.

Items flagged [unverified]:

M365 MFA and mail flow configuration — never investigated
HP iLO credentials post factory-reset — should be confirmed via vault; iLO was accessible 2026-05-12 so credentials were re-established at some point
XenServer resolution detail after 2026-04-22 outage — session log notes it offline/critical, subsequent sessions confirm it was up by 2026-05-12
DRAC IP for VWP-QBS — functional but undocumented
Yealink provisioning status — 11 phones still pending as of 2026-04-22; no follow-up session
RDS CAL grace period expiry timing — unknown; may have already expired

19 KiB Raw Blame History

Valley Wide Plastering

Profile

Infrastructure

Servers & Services

Email & Identity

Network

RDS / RemoteApp

Voice / IP Phones

Access

App Modernization Project

Application Stack (Confirmed)

Database: VWP.mdb

Domain Coverage

Source Code Status

Project Status (as of 2026-04-27)

Patterns & Known Issues

iLO Access (Non-Standard)

RDS Double-Hop Pattern

192.168.0.0/24 Subnet Conflict

Syncro Billing for Prepaid Block Emergency

AD Account: scanner

LastLogonDate Anomaly

Active Work (as of 2026-05-12)

Security Posture

2026-04-13: RDWeb Brute-Force Incident

2026-04-22: Power Outage / NVRAM Corruption

History Highlights

Compilation Notes

19 KiB

Raw Blame History

AD Account: `scanner`