Files
claudetools/wiki/clients/valleywide.md

21 KiB

type, name, display_name, last_compiled, compiled_by, sources, backlinks
type name display_name last_compiled compiled_by sources backlinks
client valleywide Valley Wide Plastering 2026-06-14 GURU-5070/claude-main
clients/valleywide/README.md
clients/valleywide/PROJECT_STATE.md
clients/valleywide/session-logs/2026-04-13-rdweb-brute-force-incident.md
clients/valleywide/session-logs/2026-04-22-hp-server-nvram-corruption-emergency.md
clients/valleywide/session-logs/2026-05-12-session.md
clients/valleywide/docs/yealink-phones.md
clients/valleywide/docs/yealink-t54w-recovery-procedure.md
clients/valleywide/app-modernization/CONTEXT.md
clients/valleywide/app-modernization/session-logs/2026-04-27-session.md
clients/valleywide/app-modernization/research/schema-analysis.md
clients/valleywide/app-modernization/source-analysis/D-drive-2026-05-16/SUMMARY.md
clients/valleywide/app-modernization/source-analysis/drive2-2026-05-16/SUMMARY.md
clients/valleywide/app-modernization/source-analysis/drive3-2026-05-16/SUMMARY.md
clients/valleywide/session-logs/2026-05-16-source-code-recovery-from-backup-drives.md
clients/valleywide/session-logs/2026-06/2026-06-13-mike-vwp-server3-migration-and-orders-source-recovery.md
clients/valleywide/session-logs/2026-06/2026-06-13-mike-vwp-gpo-disable.md
wiki/projects/valleywide-orders-modernization.md
projects/valleywide-orders-modernization

Valley Wide Plastering

Plastering / stucco subcontractor based in Arizona. Active ACG client. Primary work has been incident response (RDWeb brute-force, power outage recovery), infrastructure migration (G: file share off XenServer to new Hyper-V file server), and an ongoing app modernization project for their custom VB6/Access construction ERP.


Profile

  • Company type: Construction subcontractor (plastering / stucco)
  • Domain / site identifier: VWP (VWP.US AD domain — NetBIOS VWP; valleywideplastering.com M365 domain; vwp.us also registered external domain used for internal FQDNs)
  • Contract type: Prepaid hour block
  • Hours remaining: 20.5 hrs as of 2026-06-14 (after billing 3.5 hrs for G: migration on #32418). Always live-check Syncro before billing.
  • Managed assets (Syncro): 28
  • Billing rate: $150/hr remote labor (product 1190473 — Labor - Remote Business)
  • Emergency surcharge pattern: Bill as two line items — 1.0 hr normal + 0.5 hr surcharge. Use product 1190473 for both (NOT product 26184, which bakes in a 1.5x dollar rate that would double-charge prepaid block customers). Results in 1.5 hr block deduction = 150% charge.
  • Key contact: Shelly Dooley / Valley Wide P (Syncro display name)
  • Syncro customer ID: 31694734
  • M365 tenant ID: 5c53ae9f-7071-4248-b834-8685b646450f
  • M365 domain: valleywideplastering.com

Infrastructure

Servers & Services

Host IP Role OS Notes
HP ProLiant DL360 Gen10 (SN: MXQ80400X4) ESXi mgmt 192.168.3.24 (VLAN 99); iLO 172.16.9.125 VMware ESXi 8.0.2 host — runs most of VWP's server fleet (~12 VMs) ESXi 8.0.2 (build 22380479) 40 cores / 512 GB RAM; datastore Tesst (VMFS-6) ~14 TB, 65% full (~4.9 TB free) after the 2026-06-14 cleanup. SSH on :22, vault clients/vwp/esxi (root). Hosts ADSRVR, VWP-SERVER, VWP-FIN, WIN-Acct, WIN-AD2, Server-97, SERVER19, WINFileSvr, etc. — see VM inventory below. Power outage 2026-04-22 caused NVRAM corruption + factory iLO reset.
HP iLO 172.16.9.125 Out-of-band management for HP ProLiant SSH port 22. Requires legacy RSA algorithms — modern OpenSSH rejects it. Use paramiko with disabled_algorithms={'pubkeys': ['rsa-sha2-256', 'rsa-sha2-512']}. Credentials: vault clients/valleywide/.
VWP_ADSRVR 192.168.0.25 Domain Controller for VWP.US (secondary DC / SSH entry point) Windows Server 2019 Standard (build 17763) VM on HP ProLiant DL360 Gen10. SSH enabled, key auth working for vwp\guru (ed25519, added 2026-04-13). Default shell is cmd.exe — use powershell -NoProfile -Command wrappers. Old Net (VLAN 2).
VWP-DC1 172.16.9.2 PDC emulator for VWP.US, NPS/RADIUS Windows Server 2019 FQDN VWP-DC1.VWP.US. Confirmed up through all sessions. ADWS on this host not reachable over the SSH double-hop from ADSRVR (use LDAP cmdlets instead).
VWP-QBS 172.16.9.169 QuickBooks server + RDS/RemoteApp host Windows Server 2022 Standard Physical Dell server (NOT a VM). Has DRAC. Runs IIS (RD Web Access). WinRM on 5985. Reach from ADSRVR via Invoke-Command -ComputerName VWP-QBS -Credential with vwp\sysadmin PSCredential.
Dell DRAC (VWP-QBS) [undocumented] Out-of-band management for VWP-QBS Dell DRAC functional as of 2026-04-22. IP not yet documented. Vault: clients/valleywide/quickbooks-server-idrac.
VWP-HYPERV1 172.16.9.184 Hyper-V host — primary VM host for new infrastructure Windows Server 2025 Dell R740, 112 vCPU / 255 GB RAM, C: 10.7 TB. One external vSwitch on Intel 10G NIC. VHDs in C:\VHD. GuruRMM agent bdc3e142-.... Added 2026-06-13.
VWP-FILES 172.16.9.132 (primary) + 192.168.0.20 (VLAN 2) G: file share server (19 SMB shares) Windows Server 2019 Gen2 VM on VWP-HYPERV1 Block-migrated from SERVER3 G: VDI (100 GB, ~88 GB used). Dual-homed: primary on 172.16.9.0/24; secondary vNIC tagged VLAN 2 holds 192.168.0.20 for IP-based stragglers (see Patterns). DNS registration disabled on the .20 NIC. GuruRMM enrolled (site Main Office, agent 8e02fbbc-...). MSP360 backup running green.
XenServer 192.168.0.104 VM hypervisor — hosts remaining VMs XenServer 7.6 (PowerEdge R720) SERVER3 VM (the old "server 2003", upgraded in-place to 2008) is now powered off and retired; snapshots retained for rollback. Vault: clients/vwp/xenserver.
WINFileSvr 192.168.0.35 File server — serves O: (Office_Archive, ~570 GB / 138K files) + P: (Estimating Archive = F: root, ~545 GB / 142K files), both GPO-mapped to all staff; actively used daily Windows Server 2019 Old Net (VLAN 2). VMware VM on the ESXi host (VMID 11, WINFilrSrvr) — see ESXi inventory. ~1.1 TB live data. Holds F:\Darv\Darv.rar (51 GB Darv dev-machine backup) + F:\Darv\Darv-rar (extract, trimmed 135→26 GB on 2026-06-14). GuruRMM 62db0264-.... Candidate to consolidate into VWP-FILES (retire the VM). Do not delete Darv.rar until VB6 source verified to compile.

[WARNING] No UPS on HP ProLiant DL360. The 2026-04-22 power outage caused NVRAM corruption. UPS assessment is an outstanding priority.

VMware ESXi Host & VM Inventory (192.168.3.24)

The HP ProLiant DL360 Gen10 runs VMware ESXi 8.0.2 (mgmt 192.168.3.24, VLAN 99; SSH :22; vault clients/vwp/esxi, root). 40 cores / 512 GB RAM. Single datastore Tesst (VMFS-6, ~14 TB, 65% full / ~4.9 TB free (after the 2026-06-14 cleanup; was 87% / 1.9 TB free). Documented 2026-06-14 — the cred had been mis-filed as infrastructure/vmware-workstation ("VMware Workstation"); relocated to clients/vwp/esxi. (Naming is messy — datastore "Tesst", typo'd VM names.) 9 VMs remain after cleanup.

VMID VM name State Guest Notes
4 VWP_AD_Srvr on 2019 = VWP_ADSRVR / DC (192.168.0.25)
12 VWP-SERVER on 2019
6 VWP-FIN on 2019 .vmx dir VWP-AD-Server2
1 Server-97 on 2019
8 WIN-AD2 on 2019
7 WIN-Acct on Win10/11
2 SERVER 19 on 2012 R2
3 VWIN7-2-PC.VWP.US on Win7
11 WINFilrSrvr on 2019 The live WINFileSvr (WINFileSvr.VWP.US, 192.168.0.35). 3 disks ~4.4 TB provisioned (C: + O: 570 GB + F:/Estimating 545 GB). Had a 2.5-yr snapshot chain (ROOT "WINFILESERVER" 2023-12-30 → "VWP-FileSvr" 2024-01-13, ~440 GB delta) — consolidated 2026-06-14 via vim-cmd vmsvc/snapshot.removeall 11.

2026-06-14 cleanup (Mike's decommission batch). Three VMs powered off together on 2026-05-18 were confirmed retired and destroyed 2026-06-14, reclaiming ~3.05 TB (datastore 87% → 65%):

  • WINFileSrvr (VMID 10) — old single-disk file server, 1.5 TB (superseded by the live VMID 11).
  • WIN-QB2 (VMID 9) — old virtualized QuickBooks, 1.4 TB (live QB is the physical VWP-QBS Dell).
  • VWP-BackupSVR (VMID 5) — backup server, 150 GB. Verified zero AD entanglement before deletion (not a DC, no FSMO, no AD computer object, no DNS record; the two real DCs are ADSRVR + VWP-DC1, FSMO split across them).

Then the live WINFileSvr (VMID 11) snapshot chain was consolidated (see its row). Remaining opportunity: consolidating WINFileSvr → VWP-FILES would move ~1.1 TB of live data off this host and let the VM be retired.

Email & Identity

  • M365 tenant: valleywideplastering.com | Tenant ID: 5c53ae9f-7071-4248-b834-8685b646450f
  • On-prem AD domain: VWP.US (NetBIOS VWP, PDC = VWP-DC1.VWP.US). [NOTE: earlier notes said vwp.local — the actual AD DNS root is VWP.US. SYSVOL: C:\Windows\SYSVOL\sysvol\vwp.us\Policies\.]
  • MFA status: [unverified] — No M365 CA or MFA configuration documented.
  • MX / mail flow: [unverified] — M365 tenant confirmed but mail flow not audited.

Network

  • ISP / WAN: Public WAN IP 98.168.18.21 (observed via Yealink YMCS)
  • Firewall / Router: UniFi Dream Machine at 172.16.9.1
  • VPN: OpenVPN on UDM. Client pool: 192.168.4.0/24. Pushes routes for 172.16.9.0/24, 192.168.0.0/24, 192.168.3.0/24. DNS pushed as 192.168.4.1 (UDM).
  • Subnets:
    • 172.16.9.0/24 — primary internal network (new servers, VWP-QBS, UDM, iLO, HYPERV1, VWP-FILES primary NIC); untagged
    • 192.168.0.0/24"Old Net" = VLAN 2 on UDM (gw 172.16.9.1, DHCP .100-.199, DNS → 192.168.0.25 + 8.8.8.8). Hosts: VWP_ADSRVR (.25), WINFileSvr (.35), XenServer (.104), Yealink phones (.17/.54/.130/.140/.222), VWP-FILES secondary NIC (.20). [WARNING: conflicts with IMC's LAN — verify client context when switching VPNs.]
    • 192.168.3.0/24 — Management VLAN 99
    • 192.168.4.0/24 — OpenVPN client pool
  • Static DNS (UDM): vwp-qbs.vwp.us172.16.9.169 (typo qwp-qbs fixed 2026-04-16)
  • GPOs (domain VWP.US, as of 2026-06-13): MappedDrives — G: map → \\VWP-FILES\G-drive; Syncro + Datto RMM Agent install by immediate scheduled task — both AllSettingsDisabled (flags=3); Default Domain Policy, Enable SMB1 Client, Default Domain Controllers Policy.

RDS / RemoteApp

  • Session host: VWP-QBS (Windows Server 2022)
  • Mode: VPN-only (direct connect, no RD Gateway since 2026-04-16). RDP manifests write gatewayusagemethod:i:0.
  • RDS Licensing: Per User mode. License server pointed at vwp-qbs.vwp.us.
  • [WARNING] RDS CALs not purchased. Only the Built-in TS Per Device CAL placeholder exists. Grace period may have expired. Purchase Windows Server 2022 RDS Per User CALs sized to active user count.
  • Application: QuickBooks RemoteApp.

Voice / IP Phones

  • Fleet: 16x Yealink SIP-T54W (OUIs 805e0c and 44dbd2)
  • YMCS portal: https://us.ymcs.yealink.com/manager/sip-product/sipManage — account: Valleywide Plastering (VWP). Credentials: vault clients/valleywide/.
  • Phone subnet: Old Net (VLAN 2) 192.168.0.0/24; phones on DHCP, IPs at .17, .54, .130, .140, .222
  • Status as of 2026-04-22: 5 phones provisioned (Offline in YMCS), 11 pending first boot.
  • [WARNING] Known-bad firmware: 96.86.0.20 is a documented T54W brick-maker. Confirm YMCS firmware policy is NOT pushing this version before any mass provisioning.
  • Recovery procedure: TFTP recovery in clients/valleywide/docs/yealink-t54w-recovery-procedure.md. Laptop at 192.168.81.100, phone at 192.168.81.10.

Access

  • SSH to VWP_ADSRVR: ssh vwp\guru@192.168.0.25 (ed25519 key auth — added 2026-04-13). Default shell cmd.exe; wrap PS commands.
  • Double-hop to VWP-QBS: Via WinRM — Invoke-Command -ComputerName VWP-QBS -Credential $cred using vwp\sysadmin PSCredential from inside ADSRVR SSH session.
  • HP iLO power management: Paramiko required (not system OpenSSH). SSH to 172.16.9.125:22, disabled_algorithms={'pubkeys': ['rsa-sha2-256', 'rsa-sha2-512']}. Power-on: start system1.
  • VWP-QBS DRAC: IP undocumented — needs to be recorded. DRAC functional.
  • VPN: Connect to VWP OpenVPN (UDM) first; provides access to both 172.16.9.0/24 and 192.168.0.0/24.
  • GPO changes over SSH (VWP_ADSRVR): GPMC (Get-GPO/Set-GPO) fails with 0x80072020 over SSH double-hop. Use LDAP cmdlets (Get-ADObject, Set-ADObject) instead.
  • Vault paths: clients/valleywide/ (entries: adsrvr, dc1, udm, xenserver, quickbooks-server-idrac, domain-sysadmin). Read via bash "$VAULT" get-field clients/vwp/<entry> <field>.

App Modernization Project

Dedicated article: projects/valleywide-orders-modernization — full stack detail, source locations, modernization strategy, and history.

VWP's core business application is a custom construction ERP called ORDERS (Orders_10A.exe). The original developer ("Darv") is deceased. The app runs VB6 + Jet/Access and is approaching the 2 GB database file-size limit. ACG engaged to assess modernization feasibility.

Source recovery status (2026-06-13): COMPLETE. The full VB6 source (ORDERS_C.vbp, 2020-06-09) was recovered from Darv's machine backup (F:\Darv\Darv.rar on WINFileSvr 192.168.0.35). 12.2 MB of pure source (147 .frm, 4 .bas, 5 .vbp) is staged in the repo at clients/valleywide/app-modernization/source-code/Orders-VWP_Current-2020/. VB Decompiler Pro is no longer needed — modernization proceeds from real 2020 source. See the dedicated project article for detail.

Tracking ticket: Syncro #32280 — Source Code Data Recovery (New).


Patterns & Known Issues

iLO Access (Non-Standard)

The HP ProLiant iLO at 172.16.9.125 uses legacy SSH host key algorithms (ssh-rsa/ssh-dss) that are rejected by modern OpenSSH on Windows by default. Do not use system OpenSSH. Use Python paramiko with:

transport.disabled_algorithms = {'pubkeys': ['rsa-sha2-256', 'rsa-sha2-512']}

Power-on command: start system1.

RDS Double-Hop Pattern

SSH to ADSRVR (192.168.0.25) works fine with ed25519 key. Kerberos cannot be forwarded over SSH to reach VWP-QBS — the WinRM double-hop must be done inside the SSH session using explicit PSCredential:

$cred = Get-Credential  # vwp\sysadmin
Invoke-Command -ComputerName VWP-QBS -Credential $cred -ScriptBlock { ... }

Same double-hop constraint applies to GPMC (Get-GPO/Set-GPO) — fails 0x80072020. Use LDAP cmdlets (Get-ADObject, Set-ADObject) for GPO status changes over SSH.

192.168.0.0/24 Subnet Conflict

VWP's Old Net (VLAN 2, 192.168.0.0/24) is the same RFC1918 range as IMC (another ACG client). When switching between client VPN contexts, verify which 192.168.0.x addresses are targeted. This is a silent risk.

VWP-FILES Dual-NIC / Asymmetric Routing

VWP-FILES is dual-homed: 172.16.9.132 (primary, new net) + 192.168.0.20 (VLAN 2, Old Net — for IP-based stragglers whose UNC paths hard-code .20). DNS registration is disabled on the .20 NIC so that name resolution always returns .132. Asymmetric routing applies: cross-subnet or VPN clients cannot reach .20 (VWP-FILES replies via its .132 NIC); only same-VLAN Old Net devices can use .20 directly. Use 172.16.9.132 for all management and file pulls from outside Old Net.

Syncro Billing for Prepaid Block Emergency

Do not use product 26184 (Labor - Emergency) for prepaid block customers. That product has the 1.5x rate baked in. Always use product 1190473 for both normal and surcharge line items.

AD Account: scanner

The scanner AD account is used by some device or process (original purpose unknown). During the 2026-04-13 brute-force incident, it was being locked out every ~20 minutes by attacker attempts through the public-facing RDWeb. Password rotation is an outstanding hygiene item.

LastLogonDate Anomaly

VWP-QBS AD object showed LastLogonDate: 9/28/2049 — flagged as a time-skew artifact during 2026-04-13 incident. Likely cosmetic.


Active Work (as of 2026-06-14)

Ticket / Item Status Priority
#32280 — Source Code Data Recovery / App modernization New — source recovered; next: stand up VB6 build env, confirm ORDERS_C.vbp compiles High
#32418 — G-Drive Migration Invoiced — 3.5 h billed, prepay 24.0→20.5 Closed
#32396 — Printer Waiting Medium
#32375 — New Phone Install New Medium
#32348 — Bizhub print New Medium
#32208 — Folder access New Medium
#32039 — Onsite setup New Medium
RDS CAL purchase (Server 2022 Per User, sized to active user count) Outstanding — grace period status unknown High
Yealink phone fleet provisioning (11 pending phones) Outstanding since 2026-04-22 Medium
Cleanup: delete C:\VHD\server3-g.vhd (99 GB) on HYPERV1 + XenServer G: snapshot + F:\Darv\Darv-rar (135 GB) once source compiles Pending Low
UPS assessment for HP ProLiant Outstanding since 2026-04-22 Medium
HP iLO reconfiguration post factory-reset (2026-04-22) [verify — was accessible 2026-05-12 so credentials re-established] Medium
scanner AD account password rotation Outstanding since 2026-04-13 Low
UDM UPnP audit Outstanding since 2026-04-13 Low
DRAC IP documentation for VWP-QBS Not yet recorded Low
Existing Syncro + Datto RMM agent uninstalls GPOs disabled 2026-06-13 (stops new installs); existing agents still on machines — awaiting user direction Low
Old-Net DHCP secondary DNS (8.8.8.8) Consider replacing with second internal DC Low

Security Posture

2026-04-13: RDWeb Brute-Force Incident

RDWeb (https://VWP-QBS/RDWeb/Pages/login.aspx) was publicly exposed via UDM port-forward on port 443. A distributed brute-force botnet (residential proxies, IPs from China, Belarus, UAE) hammered POST /RDWeb/Pages/en-US/login.aspx at ~6 req/min, hitting usernames scanner, Guest, Receptionist, triggering AD lockouts.

Resolution: UDM port-forward removed same day. 30-day audit of Event 4624 confirmed zero successful external logons — no compromise.

Current state: RDWeb accessible from VPN and internal LAN only.

Recommendation: If re-exposed publicly — require IPBan, firewall restriction to known IPs, and 2FA/CA.

2026-04-22: Power Outage / NVRAM Corruption

Power outage caused HP ProLiant NVRAM corruption (BIOS/iLO factory reset). VWP-QBS Dell had a boot retry loop (resolved via DRAC). XenServer was offline. All recovered onsite. Root cause: no UPS on HP server.


History Highlights

Date Event
2026-04-13 RDWeb brute-force incident discovered and contained. SSH key deployed to ADSRVR. 30-day audit — no compromise.
2026-04-13 Domain lockout policy temporarily disabled during diagnosis (threshold=0), restored to 5/16min/16min.
2026-04-16 RDS reconfigured to VPN-only (gateway removed). UDM DNS typo fixed (qwp-qbsvwp-qbs). RDS licensing mode set Per User.
2026-04-22 Emergency onsite: power outage, HP ProLiant NVRAM corruption + iLO factory reset, VWP-QBS boot loop (DRAC), XenServer offline. All resolved ~12:00 MST.
2026-04-22 Yealink SIP-T54W fleet (16 devices) added to YMCS. 5 provisioned, 11 pending.
2026-04-27 App modernization project initiated. VB6 P-Code + Jet 3.x stack confirmed; ~130 tables extracted via binary scan; Crystal Reports 8.5 (791 .rpt) documented. Decompilation planned.
2026-05-12 HP ProLiant found powered-off (ADSRVR unreachable). Powered on remotely via iLO paramiko. Syncro ticket #32269, invoice #67594, 1.5 hr block deduction.
2026-05-16 VB6 source search across 3 backup rotation drives. Production location identified (G:\VWP2\ on 97-Server); 4-year gap resolved (Darv worked on compiled EXE only after 2020-06 — no .vbp evolution past ORDERS_C.vbp 2020-06-09). Orders_10A.exe staged to repo.
2026-06-13 SERVER3 (XenServer "server 2003" VM, upgraded to 2008 in-place) retired. G: file share (100 GB) block-migrated via VDI export→VHDX to new VWP-FILES (Gen2 Server 2019 on VWP-HYPERV1 172.16.9.184). 19 SMB shares recreated; MappedDrives GPO repointed to \\VWP-FILES\G-drive. IP takeover: VWP-FILES holds 192.168.0.20 (VLAN 2) for IP-based stragglers. SERVER3 snapshotted and powered off. VWP-FILES enrolled in GuruRMM (site Main Office) + MSP360 backup green. Billed 3.5 h on #32418 (prepay 24.0→20.5).
2026-06-13 VB6 Orders source fully recovered from F:\Darv\Darv.rar on WINFileSvr (192.168.0.35). 12.2 MB staged to repo (source-code/Orders-VWP_Current-2020/). VB Decompiler Pro no longer needed. See projects/valleywide-orders-modernization.
2026-06-13 Syncro and Datto RMM Agent deployment GPOs disabled (AllSettingsDisabled, flags=3) via LDAP on VWP_ADSRVR. Existing agents not yet uninstalled — awaiting direction.

Compilation Notes

Date range covered: 2026-04-13 through 2026-06-13.

Items flagged [unverified]:

  • M365 MFA and mail flow configuration — never investigated
  • HP iLO credentials post factory-reset — accessible 2026-05-12 so credentials were re-established; confirm vault entry
  • DRAC IP for VWP-QBS — functional but undocumented
  • Yealink provisioning status — 11 phones pending as of 2026-04-22; no follow-up confirmed
  • RDS CAL grace period — may have expired
  • AD replication of GPO flags=3 changes to VWP-DC1 — ADWS not reachable over SSH from ADSRVR; normal replication expected but not spot-checked