--- type: client name: valleywide display_name: Valley Wide Plastering last_compiled: 2026-05-24 compiled_by: DESKTOP-0O8A1RL/claude-main sources: - clients/valleywide/README.md - clients/valleywide/PROJECT_STATE.md - clients/valleywide/session-logs/2026-04-13-rdweb-brute-force-incident.md - clients/valleywide/session-logs/2026-04-22-hp-server-nvram-corruption-emergency.md - clients/valleywide/session-logs/2026-05-12-session.md - clients/valleywide/docs/yealink-phones.md - clients/valleywide/docs/yealink-t54w-recovery-procedure.md - clients/valleywide/app-modernization/CONTEXT.md - clients/valleywide/app-modernization/session-logs/2026-04-27-session.md - clients/valleywide/app-modernization/research/schema-analysis.md - clients/valleywide/app-modernization/source-analysis/D-drive-2026-05-16/SUMMARY.md - clients/valleywide/app-modernization/source-analysis/drive2-2026-05-16/SUMMARY.md - clients/valleywide/app-modernization/source-analysis/drive3-2026-05-16/SUMMARY.md backlinks: [] --- # Valley Wide Plastering Plastering / stucco subcontractor based in Arizona. Active ACG client. Primary work has been incident response (RDWeb brute-force, power outage recovery) and an ongoing app modernization project for their custom VB6/Access construction ERP. --- ## Profile - **Company type:** Construction subcontractor (plastering / stucco) - **Domain / site identifier:** VWP (`vwp.local` internal AD domain, `vwp.us` registered external domain, `valleywideplastering.com` M365 domain) - **Contract type:** Prepaid hour block - **Hours remaining:** 10.0 hrs as of 2026-05-12 (after billing 1.5 hrs for HP server emergency). Always live-check Syncro before billing. - **Billing rate:** $150/hr remote labor (`product 1190473 — Labor - Remote Business`) - **Emergency surcharge pattern:** Bill as two line items — 1.0 hr normal + 0.5 hr surcharge. Use product 1190473 for both (NOT product 26184, which bakes in a 1.5x dollar rate that would double-charge prepaid block customers). Results in 1.5 hr block deduction = 150% charge. - **Key contact:** Shelly Dooley / Valley Wide P (Syncro customer display name) - **Syncro customer ID:** `31694734` - **Syncro ticket (2026-05-12 emergency):** #32269 (ID: `110159277`) — HP server powered off, ADSRVR unreachable. Invoiced; invoice #67594 (ID: `1650271395`). Ticket status: Invoiced. - **M365 tenant ID:** `5c53ae9f-7071-4248-b834-8685b646450f` - **M365 domain:** `valleywideplastering.com` --- ## Infrastructure ### Servers & Services | Host | IP | Role | OS | Notes | |---|---|---|---|---| | HP ProLiant DL360 Gen10 (SN: MXQ80400X4) | (LAN — no static IP documented) | Hypervisor / VM host for ADSRVR | — | iLO at 172.16.9.125 (SSH port 22, legacy ssh-rsa key). Power outage 2026-04-22 caused NVRAM corruption + factory iLO reset. Was found powered-off 2026-05-12; powered on remotely via iLO. | | HP iLO | 172.16.9.125 | Out-of-band management for HP ProLiant | — | SSH port 22. **Requires legacy RSA algorithms** — modern OpenSSH rejects it. Use paramiko with `disabled_algorithms={'pubkeys': ['rsa-sha2-256', 'rsa-sha2-512']}`. Credentials in vault: `clients/valleywide/` | | VWP_ADSRVR | 192.168.0.25 | Domain Controller for `vwp.local` | Windows Server 2019 Standard (build 17763) | VM on HP ProLiant DL360 Gen10. SSH enabled, key auth working for `vwp\guru` (ed25519, added 2026-04-13). Default shell is cmd.exe — use `powershell -NoProfile -Command` wrappers. | | VWP-QBS | 172.16.9.169 | QuickBooks server + RDS/RemoteApp host | Windows Server 2022 Standard | **Physical Dell server** (NOT a VM). Has DRAC. Runs IIS (RD Web Access, RD Gateway). Reach from ADSRVR via `Invoke-Command -ComputerName VWP-QBS -Credential` with `vwp\sysadmin` PSCredential — no direct SSH; Kerberos does not forward over SSH double-hop. WinRM on 5985. | | Dell DRAC (VWP-QBS) | [undocumented] | Out-of-band management for VWP-QBS Dell | — | DRAC functional as of 2026-04-22; used to force manual boot after power outage. IP not yet documented. | | DC1 | 172.16.9.2 | Domain Controller | — | Confirmed up 2026-05-12. Separate from ADSRVR. | | XenServer (older Dell) | 192.168.0.104 | VM hypervisor — hosts BACKUP-SRV, Server 2012 R2, Server 2003 | XenServer | Older Dell hardware. Was offline after 2026-04-22 power outage; status resolved. Credentials: `root` / see vault. | | UDM (UniFi Dream Machine) | 172.16.9.1 | Perimeter firewall, OpenVPN server, DHCP, DNS, site router | UniFi OS | DNS override: `vwp-qbs.vwp.us` → 172.16.9.169 (static record in UDM dnsmasq). VPN pushes DNS=192.168.4.1 (UDM). WireGuard site-to-site peers present (wgsts1001, wgsts1003, wgsts1005 — likely UniFi SiteMagic). | **[WARNING] No UPS on HP ProLiant DL360.** The 2026-04-22 power outage caused NVRAM corruption. A UPS assessment is an outstanding priority item — hardware failure from power event is a proven risk. ### Email & Identity - **M365 tenant:** `valleywideplastering.com` | Tenant ID: `5c53ae9f-7071-4248-b834-8685b646450f` - **On-prem AD domain:** `vwp.local` (internal). External registered domain: `vwp.us` (used for internal FQDNs like `vwp-qbs.vwp.us`). - **MFA status:** [unverified] — No M365 CA or MFA configuration documented. Not investigated. - **MX / mail flow:** [unverified] — M365 tenant confirmed but mail flow not audited. ### Network - **ISP / WAN:** Public WAN IP `98.168.18.21` (observed via Yealink YMCS last-seen registrar) - **Firewall / Router:** UniFi Dream Machine at 172.16.9.1 - **VPN:** OpenVPN on UDM. Client pool: `192.168.4.0/24`. Pushes routes for `172.16.9.0/24`, `192.168.0.0/24`, `192.168.3.0/24`. DNS pushed as `192.168.4.1` (UDM). - **Subnets:** - `172.16.9.0/24` — primary internal network (servers, Dell VWP-QBS, UDM, iLO) - `192.168.0.0/24` — secondary internal (AD server, Yealink phones) [WARNING: conflicts with IMC's LAN — be careful when switching VPN contexts between clients] - `192.168.4.0/24` — OpenVPN client pool - **Static DNS (UDM):** `vwp-qbs.vwp.us` → `172.16.9.169` (fixed typo from `qwp-qbs.vwp.us` on 2026-04-16) ### RDS / RemoteApp - **Session host:** VWP-QBS (Windows Server 2022) - **Mode:** VPN-only (direct connect, no RD Gateway). Gateway was removed from the deployment 2026-04-16 after the RDWeb public exposure was closed. RDP manifests write `gatewayusagemethod:i:0`. - **RDS Licensing:** Per User mode. License server pointed at `vwp-qbs.vwp.us` (the same box — RDS-Licensing role was installed and activated on 2026-04-16 but had no real CALs). - **[WARNING] RDS CALs not purchased.** VWP-QBS license server has only the `Built-in TS Per Device CAL` placeholder. Users will start seeing "no licenses available" errors once grace period expires. Action: purchase Windows Server 2022 RDS Per User CALs, sized to active user count (check distinct interactive logons last 30 days via `licmgr.msc`). - **Application:** QuickBooks RemoteApp. VPN clients resolve `vwp-qbs.vwp.us` via UDM dnsmasq override and connect directly. ### Voice / IP Phones - **Fleet:** 16x Yealink SIP-T54W color IP phones (OUIs `805e0c` and `44dbd2`) - **YMCS portal:** https://us.ymcs.yealink.com/manager/sip-product/sipManage — account: Valleywide Plastering (VWP) - **YMCS admin password:** vault — `clients/valleywide/` (Yealink password documented 2026-04-22) - **Status as of 2026-04-22:** 5 phones previously provisioned (Offline in YMCS), 11 pending first boot - **Named phones:** `214-ValleyWidePlastering` (extension 214), `Reception` (front desk, 192.168.0.17) - **Phone subnet:** `192.168.0.0/24` — phones on DHCP, IPs observed at .17, .54, .130, .140, .222 - **[WARNING] Known-bad firmware:** `96.86.0.20` is a documented T54W brick-maker. Confirm YMCS firmware policy is NOT pushing this version before any mass provisioning. - **Recovery procedure:** TFTP recovery documented in `clients/valleywide/docs/yealink-t54w-recovery-procedure.md`. Use Tftpd64 with laptop at `192.168.81.100`, phone at `192.168.81.10`. Multiple recovery file sets may be needed (NEW RM → OLD RM → SPEAKER variant). --- ## Access - **SSH to VWP_ADSRVR:** `ssh vwp\guru@192.168.0.25` (ed25519 key auth — key added 2026-04-13) - **Double-hop to VWP-QBS:** Via WinRM — `Invoke-Command -ComputerName VWP-QBS -Credential $cred` using `vwp\sysadmin` PSCredential from ADSRVR. SSH won't forward Kerberos for domain double-hop. - **HP iLO power management:** Paramiko required (not system OpenSSH). SSH to `172.16.9.125:22`. Use `disabled_algorithms={'pubkeys': ['rsa-sha2-256', 'rsa-sha2-512']}`. Command: `start system1` to power on. - **VWP-QBS DRAC:** IP undocumented — needs to be recorded. DRAC functional. - **VPN:** Connect to VWP OpenVPN (UDM) first; this provides access to both the 172.16.9.0/24 and 192.168.0.0/24 subnets. - **Vault paths:** `clients/valleywide/` (confirmed entries: `adsrvr`, `dc1`, `udm`, `xenserver`, `quickbooks-server-idrac`). Access via `bash "$VAULT" get-field clients/valleywide/ `. --- ## App Modernization Project VWP's core business application is a custom-built construction ERP. The original developer (known as "Darv") is deceased. The app is hitting the 2GB Jet/Access database file size limit. ACG was engaged to assess modernization feasibility. ### Application Stack (Confirmed) | Layer | Technology | Evidence | |---|---|---| | Frontend / logic | Visual Basic 6.0 | `frmPayroll.frm` source file, `.frx` resource files, `VB5!` header in exe | | Compilation | **P-Code** (not Native Code) | Entry point `PUSH+CALL` to ThunRTMain by ordinal — not native binary | | Database | MS Access Jet 3.x (.mdb) | `VWP.mdb` version byte 0x00, Access 97 format | | Reporting | Crystal Reports 8.5 | 791 `.rpt` files (per 2026-04-27 archive); Crystl32.OCX import; SCR85Dev installer found | | Installer | InstallShield Denali 2021 | `Denali2021v1` folder on server | | OCX controls | TABCTL32, mscomct2, comdlg32, Flp32a30, odg7, todg7 | PE import table | **P-Code is the best possible outcome for decompilation.** VB Decompiler Pro (~$200) can recover 70-80% of source including form layouts, procedure names, string literals, and all SQL queries. Decompilation was approved as the next step. ### Database: VWP.mdb - **Current size:** 938 MB (last written 2026-04-24). Growth: 671 MB (2020) → 761 MB (2022) → 938 MB (2026). **Approaching the 2 GB Jet hard limit.** - **Format:** Jet 3.x / Access 97. Modern ACE/DAO drivers refuse to open it — binary scan was used for schema extraction. - **Scale:** ~130 production tables spanning a full construction ERP. #### Domain Coverage | Domain | Key Tables | |---|---| | Projects & Jobs | tblPROJECT, tblLOTINFO, tblPLANS, tblCHANGE, tblSZONE | | Work Orders & Estimating | tblORDERS, tblTAKE, tblMEASURE, tblPlanBill | | Inventory & Purchasing | tblINVPRICE, tblINVTRY, tblSUPPLIER, tblPOrder, tblYardOrder | | Crew & Payroll | tblCREW, tblHRDAILY, tblPAYHEADER, tblPAYROLL, tblCREWRATE | | **Certified Payroll** | **tblCERTIFIED** — government / prevailing wage work. **HARD requirement.** | | Accounts Receivable | tblARMASTER, tblARINVOICE, tblARTRANS | | Accounts Payable | tblAPMASTER, tblAPTRANS, tblJOBCOST, tblCHECKREC | | **Positive Pay (3 banks)** | **tblPosPayVWP, tblPosPayCRD, tblPosPaySWI** — fraud-prevention bank integration. **HARD dependency.** | | Scaffold | tblScaffold, tblSC_Crew | | Repairs | tblREPAIR, tblRepList | | System / Config | tblSECURITY, tblSYSInfo, tblGLAcct | **Modernization complexity: HIGH.** 791 Crystal Reports files, certified payroll (legal compliance — cannot be dropped), positive pay integration with 3 banks, and full AR/AP/Payroll. ### Source Code Status The production exe (`Orders_10A.exe`, 13.4 MB) has four shortcuts pointing to it. The original source was on Darv's personal development machine — only one form file (`frmPayroll.frm`, 32 KB) was found on the server at `C:\Users\sysadmin\Desktop\Darv\Source\VWP\`. The remainder of `C:\Users\sysadmin\Desktop\Darv\` (13,231 files, 15.6 GB) includes Darv's installer projects, Crystal Reports, and personal files. VB6 source (`.vbp`, `.frm`) was scanned across multiple server drives (D: and two additional drives as of 2026-05-16). Substantial VB6 source exists across the drives (thousands of `.frm` and `.vbp` files); Mike was searching to confirm which are for the VWP application specifically. ### Project Status (as of 2026-04-27) | Task | Status | |---|---| | Stack identification | Complete — VB6 P-Code + Jet 3.x confirmed | | Schema mapping (table names) | Complete (~130 tables via binary scan) | | Full schema with field types | Pending — needs Access 97/2000 environment or Jet 3.x → Jet 4.x conversion | | VB6 source search across server drives | In progress — Mike searching | | VB Decompiler Pro purchase and run | Pending ($200 investment) | | Crystal Reports audit (791 .rpt files) | Pending | | VWP staff workflow interviews | Pending | | Feasibility / modernization report | Pending | --- ## Patterns & Known Issues ### iLO Access (Non-Standard) The HP ProLiant iLO at 172.16.9.125 uses legacy SSH host key algorithms (`ssh-rsa`/`ssh-dss`) that are rejected by modern OpenSSH on Windows by default. **Do not use system OpenSSH to connect.** Use Python paramiko with: ```python transport.disabled_algorithms = {'pubkeys': ['rsa-sha2-256', 'rsa-sha2-512']} ``` Power-on command: `start system1`. ### RDS Double-Hop Pattern SSH to ADSRVR (192.168.0.25) works fine with ed25519 key. But you cannot forward Kerberos over SSH to reach VWP-QBS — the WinRM double-hop must be done inside the SSH session using explicit PSCredential: ```powershell $cred = Get-Credential # vwp\sysadmin Invoke-Command -ComputerName VWP-QBS -Credential $cred -ScriptBlock { ... } ``` ### 192.168.0.0/24 Subnet Conflict VWP's AD/phone subnet (`192.168.0.0/24`) is the same RFC1918 range as IMC (another ACG client). When switching between client VPN contexts, verify which 192.168.0.x addresses are being targeted. This is a silent risk — wrong subnet = wrong client. ### Syncro Billing for Prepaid Block Emergency Do not use product 26184 (Labor - Emergency) for prepaid block customers. That product has the 1.5x rate baked in, which would result in double-charging when combined with the surcharge line item pattern. Always use product 1190473 for both normal and surcharge line items. ### AD Account: `scanner` The `scanner` AD account is used by some device or process (original purpose unknown). Its password was last set 2024-10-17. During the 2026-04-13 brute-force incident, it was being locked out every ~20 minutes by attacker attempts through the public-facing RDWeb. **Password rotation is an outstanding hygiene item.** ### LastLogonDate Anomaly VWP-QBS AD object showed `LastLogonDate: 9/28/2049` — flagged as a time-skew artifact during 2026-04-13 incident. Likely cosmetic. --- ## Active Work (as of 2026-05-12) | Item | Status | Priority | |---|---|---| | App modernization: VB Decompiler Pro run against Orders_10A.exe | Pending — decompiler not yet purchased | High | | App modernization: Full schema extraction with field types | Pending — needs Access 97/2000 environment | High | | App modernization: VB6 source search across server drives | In progress | High | | RDS CAL purchase (Windows Server 2022 Per User, sized to user count) | Outstanding — grace period may expire | High | | HP iLO reconfiguration (post factory-reset 2026-04-22) | [unverified — may have been configured during 2026-04-22 onsite; confirm credentials in vault] | Medium | | UPS assessment for HP ProLiant | Outstanding since 2026-04-22 | Medium | | Yealink phone fleet provisioning (11 pending phones) | Outstanding — 11 of 16 phones never connected to YMCS | Medium | | `scanner` AD account password rotation | Outstanding since 2026-04-13 | Low | | UDM UPnP audit | Outstanding since 2026-04-13 | Low | | DRAC IP documentation for VWP-QBS | Not yet recorded | Low | --- ## Security Posture ### 2026-04-13: RDWeb Brute-Force Incident RDWeb (`https://VWP-QBS/RDWeb/Pages/login.aspx`) was publicly exposed via UDM port-forward on port 443. A distributed brute-force botnet (residential proxy infrastructure, IPs from China, Belarus, UAE, and others) was hammering `POST /RDWeb/Pages/en-US/login.aspx` at ~6 req/min, hitting usernames `scanner`, `Guest`, `Receptionist`. This triggered AD lockouts every ~20 minutes (lockout threshold 5, 16-min window) which initially appeared to be a stale internal credential problem. **Resolution:** UDM port-forward removed (same day), IIS reset to drain in-flight sessions, lockout policy restored. 30-day audit of Event 4624 confirmed **zero successful external logons — no compromise**. **Current state:** RDWeb accessible from VPN and internal LAN only (port 443 on VWP-QBS, 172.16.9.0/24). Not reachable from public internet. **Outstanding recommendation:** If RDWeb must be re-exposed publicly, require: IPBan (https://github.com/DigitalRuby/IPBan), firewall restriction to known source IPs, and 2FA/Conditional Access. ### 2026-04-22: Power Outage / NVRAM Corruption Power outage caused HP ProLiant NVRAM corruption (BIOS/iLO factory reset). VWP-QBS Dell server had a boot retry loop (resolved via DRAC). XenServer (older Dell) was offline. All recovered onsite. **Root cause: no UPS on HP server.** --- ## History Highlights | Date | Event | |---|---| | 2026-04-13 | RDWeb brute-force incident discovered and contained. SSH key deployed to ADSRVR. 30-day audit — no compromise. | | 2026-04-13 | Domain lockout policy temporarily disabled during diagnosis (threshold=0), restored to 5/16min/16min. 15-minute window of reduced lockout protection. | | 2026-04-16 | RDS reconfigured to VPN-only (gateway removed). UDM DNS typo fixed (`qwp-qbs` → `vwp-qbs`). RDS licensing mode set Per User, pointed at local license server. | | 2026-04-22 | Emergency onsite: power outage, HP ProLiant NVRAM corruption + iLO factory reset, VWP-QBS boot loop (DRAC), XenServer offline. All resolved ~12:00 MST. | | 2026-04-22 | Yealink SIP-T54W fleet (16 devices) added to YMCS device management. 5 previously-provisioned, 11 pending. | | 2026-04-27 | App modernization project initiated. VB6 P-Code + Jet 3.x stack confirmed. ~130 table schema extracted via binary scan. Crystal Reports 8.5 (791 .rpt files) documented. | | 2026-05-12 | HP ProLiant found powered-off (ADSRVR unreachable). Powered on remotely via iLO paramiko. Syncro ticket #32269, invoice #67594, 1.5 hr block deduction (10.0 hrs remaining). | --- ## Compilation Notes **Date range covered:** 2026-04-13 through 2026-05-12. **Items flagged [unverified]:** - M365 MFA and mail flow configuration — never investigated - HP iLO credentials post factory-reset — should be confirmed via vault; iLO was accessible 2026-05-12 so credentials were re-established at some point - XenServer resolution detail after 2026-04-22 outage — session log notes it offline/critical, subsequent sessions confirm it was up by 2026-05-12 - DRAC IP for VWP-QBS — functional but undocumented - Yealink provisioning status — 11 phones still pending as of 2026-04-22; no follow-up session - RDS CAL grace period expiry timing — unknown; may have already expired