instrumental-music-center.md — AIMsi POS on SQL Server 2019 (Standard under misleading SQLEXPRESS instance name); phantom DC ServerIMC causing slow logons; GuruRMM enrolled (IMC1 fa99e913); OpenVPN subnet-overlap hazard; $175/hr prepaid, 12.5 hrs remaining; SQL max server memory fix approved but unverified applied. valleywide.md — Valley Wide Plastering; HP DL360 Gen10 VM host + XenServer; VB6/Access 97 app modernization (130 tables, 791 Crystal Reports, certified payroll); RDWeb brute-force incident (contained); 11 Yealink phones pending; iLO requires paramiko (legacy ssh-rsa); $175/hr prepaid, 10 hrs remaining. wiki/index.md — both clients added to Clients table and Cross-Reference. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
272 lines
19 KiB
Markdown
272 lines
19 KiB
Markdown
---
|
|
type: client
|
|
name: valleywide
|
|
display_name: Valley Wide Plastering
|
|
last_compiled: 2026-05-24
|
|
compiled_by: DESKTOP-0O8A1RL/claude-main
|
|
sources:
|
|
- clients/valleywide/README.md
|
|
- clients/valleywide/PROJECT_STATE.md
|
|
- clients/valleywide/session-logs/2026-04-13-rdweb-brute-force-incident.md
|
|
- clients/valleywide/session-logs/2026-04-22-hp-server-nvram-corruption-emergency.md
|
|
- clients/valleywide/session-logs/2026-05-12-session.md
|
|
- clients/valleywide/docs/yealink-phones.md
|
|
- clients/valleywide/docs/yealink-t54w-recovery-procedure.md
|
|
- clients/valleywide/app-modernization/CONTEXT.md
|
|
- clients/valleywide/app-modernization/session-logs/2026-04-27-session.md
|
|
- clients/valleywide/app-modernization/research/schema-analysis.md
|
|
- clients/valleywide/app-modernization/source-analysis/D-drive-2026-05-16/SUMMARY.md
|
|
- clients/valleywide/app-modernization/source-analysis/drive2-2026-05-16/SUMMARY.md
|
|
- clients/valleywide/app-modernization/source-analysis/drive3-2026-05-16/SUMMARY.md
|
|
backlinks: []
|
|
---
|
|
|
|
# Valley Wide Plastering
|
|
|
|
Plastering / stucco subcontractor based in Arizona. Active ACG client. Primary work has been incident response (RDWeb brute-force, power outage recovery) and an ongoing app modernization project for their custom VB6/Access construction ERP.
|
|
|
|
---
|
|
|
|
## Profile
|
|
|
|
- **Company type:** Construction subcontractor (plastering / stucco)
|
|
- **Domain / site identifier:** VWP (`vwp.local` internal AD domain, `vwp.us` registered external domain, `valleywideplastering.com` M365 domain)
|
|
- **Contract type:** Prepaid hour block
|
|
- **Hours remaining:** 10.0 hrs as of 2026-05-12 (after billing 1.5 hrs for HP server emergency). Always live-check Syncro before billing.
|
|
- **Billing rate:** $150/hr remote labor (`product 1190473 — Labor - Remote Business`)
|
|
- **Emergency surcharge pattern:** Bill as two line items — 1.0 hr normal + 0.5 hr surcharge. Use product 1190473 for both (NOT product 26184, which bakes in a 1.5x dollar rate that would double-charge prepaid block customers). Results in 1.5 hr block deduction = 150% charge.
|
|
- **Key contact:** Shelly Dooley / Valley Wide P (Syncro customer display name)
|
|
- **Syncro customer ID:** `31694734`
|
|
- **Syncro ticket (2026-05-12 emergency):** #32269 (ID: `110159277`) — HP server powered off, ADSRVR unreachable. Invoiced; invoice #67594 (ID: `1650271395`). Ticket status: Invoiced.
|
|
- **M365 tenant ID:** `5c53ae9f-7071-4248-b834-8685b646450f`
|
|
- **M365 domain:** `valleywideplastering.com`
|
|
|
|
---
|
|
|
|
## Infrastructure
|
|
|
|
### Servers & Services
|
|
|
|
| Host | IP | Role | OS | Notes |
|
|
|---|---|---|---|---|
|
|
| HP ProLiant DL360 Gen10 (SN: MXQ80400X4) | (LAN — no static IP documented) | Hypervisor / VM host for ADSRVR | — | iLO at 172.16.9.125 (SSH port 22, legacy ssh-rsa key). Power outage 2026-04-22 caused NVRAM corruption + factory iLO reset. Was found powered-off 2026-05-12; powered on remotely via iLO. |
|
|
| HP iLO | 172.16.9.125 | Out-of-band management for HP ProLiant | — | SSH port 22. **Requires legacy RSA algorithms** — modern OpenSSH rejects it. Use paramiko with `disabled_algorithms={'pubkeys': ['rsa-sha2-256', 'rsa-sha2-512']}`. Credentials in vault: `clients/valleywide/` |
|
|
| VWP_ADSRVR | 192.168.0.25 | Domain Controller for `vwp.local` | Windows Server 2019 Standard (build 17763) | VM on HP ProLiant DL360 Gen10. SSH enabled, key auth working for `vwp\guru` (ed25519, added 2026-04-13). Default shell is cmd.exe — use `powershell -NoProfile -Command` wrappers. |
|
|
| VWP-QBS | 172.16.9.169 | QuickBooks server + RDS/RemoteApp host | Windows Server 2022 Standard | **Physical Dell server** (NOT a VM). Has DRAC. Runs IIS (RD Web Access, RD Gateway). Reach from ADSRVR via `Invoke-Command -ComputerName VWP-QBS -Credential` with `vwp\sysadmin` PSCredential — no direct SSH; Kerberos does not forward over SSH double-hop. WinRM on 5985. |
|
|
| Dell DRAC (VWP-QBS) | [undocumented] | Out-of-band management for VWP-QBS Dell | — | DRAC functional as of 2026-04-22; used to force manual boot after power outage. IP not yet documented. |
|
|
| DC1 | 172.16.9.2 | Domain Controller | — | Confirmed up 2026-05-12. Separate from ADSRVR. |
|
|
| XenServer (older Dell) | 192.168.0.104 | VM hypervisor — hosts BACKUP-SRV, Server 2012 R2, Server 2003 | XenServer | Older Dell hardware. Was offline after 2026-04-22 power outage; status resolved. Credentials: `root` / see vault. |
|
|
| UDM (UniFi Dream Machine) | 172.16.9.1 | Perimeter firewall, OpenVPN server, DHCP, DNS, site router | UniFi OS | DNS override: `vwp-qbs.vwp.us` → 172.16.9.169 (static record in UDM dnsmasq). VPN pushes DNS=192.168.4.1 (UDM). WireGuard site-to-site peers present (wgsts1001, wgsts1003, wgsts1005 — likely UniFi SiteMagic). |
|
|
|
|
**[WARNING] No UPS on HP ProLiant DL360.** The 2026-04-22 power outage caused NVRAM corruption. A UPS assessment is an outstanding priority item — hardware failure from power event is a proven risk.
|
|
|
|
### Email & Identity
|
|
|
|
- **M365 tenant:** `valleywideplastering.com` | Tenant ID: `5c53ae9f-7071-4248-b834-8685b646450f`
|
|
- **On-prem AD domain:** `vwp.local` (internal). External registered domain: `vwp.us` (used for internal FQDNs like `vwp-qbs.vwp.us`).
|
|
- **MFA status:** [unverified] — No M365 CA or MFA configuration documented. Not investigated.
|
|
- **MX / mail flow:** [unverified] — M365 tenant confirmed but mail flow not audited.
|
|
|
|
### Network
|
|
|
|
- **ISP / WAN:** Public WAN IP `98.168.18.21` (observed via Yealink YMCS last-seen registrar)
|
|
- **Firewall / Router:** UniFi Dream Machine at 172.16.9.1
|
|
- **VPN:** OpenVPN on UDM. Client pool: `192.168.4.0/24`. Pushes routes for `172.16.9.0/24`, `192.168.0.0/24`, `192.168.3.0/24`. DNS pushed as `192.168.4.1` (UDM).
|
|
- **Subnets:**
|
|
- `172.16.9.0/24` — primary internal network (servers, Dell VWP-QBS, UDM, iLO)
|
|
- `192.168.0.0/24` — secondary internal (AD server, Yealink phones) [WARNING: conflicts with IMC's LAN — be careful when switching VPN contexts between clients]
|
|
- `192.168.4.0/24` — OpenVPN client pool
|
|
- **Static DNS (UDM):** `vwp-qbs.vwp.us` → `172.16.9.169` (fixed typo from `qwp-qbs.vwp.us` on 2026-04-16)
|
|
|
|
### RDS / RemoteApp
|
|
|
|
- **Session host:** VWP-QBS (Windows Server 2022)
|
|
- **Mode:** VPN-only (direct connect, no RD Gateway). Gateway was removed from the deployment 2026-04-16 after the RDWeb public exposure was closed. RDP manifests write `gatewayusagemethod:i:0`.
|
|
- **RDS Licensing:** Per User mode. License server pointed at `vwp-qbs.vwp.us` (the same box — RDS-Licensing role was installed and activated on 2026-04-16 but had no real CALs).
|
|
- **[WARNING] RDS CALs not purchased.** VWP-QBS license server has only the `Built-in TS Per Device CAL` placeholder. Users will start seeing "no licenses available" errors once grace period expires. Action: purchase Windows Server 2022 RDS Per User CALs, sized to active user count (check distinct interactive logons last 30 days via `licmgr.msc`).
|
|
- **Application:** QuickBooks RemoteApp. VPN clients resolve `vwp-qbs.vwp.us` via UDM dnsmasq override and connect directly.
|
|
|
|
### Voice / IP Phones
|
|
|
|
- **Fleet:** 16x Yealink SIP-T54W color IP phones (OUIs `805e0c` and `44dbd2`)
|
|
- **YMCS portal:** https://us.ymcs.yealink.com/manager/sip-product/sipManage — account: Valleywide Plastering (VWP)
|
|
- **YMCS admin password:** vault — `clients/valleywide/` (Yealink password documented 2026-04-22)
|
|
- **Status as of 2026-04-22:** 5 phones previously provisioned (Offline in YMCS), 11 pending first boot
|
|
- **Named phones:** `214-ValleyWidePlastering` (extension 214), `Reception` (front desk, 192.168.0.17)
|
|
- **Phone subnet:** `192.168.0.0/24` — phones on DHCP, IPs observed at .17, .54, .130, .140, .222
|
|
- **[WARNING] Known-bad firmware:** `96.86.0.20` is a documented T54W brick-maker. Confirm YMCS firmware policy is NOT pushing this version before any mass provisioning.
|
|
- **Recovery procedure:** TFTP recovery documented in `clients/valleywide/docs/yealink-t54w-recovery-procedure.md`. Use Tftpd64 with laptop at `192.168.81.100`, phone at `192.168.81.10`. Multiple recovery file sets may be needed (NEW RM → OLD RM → SPEAKER variant).
|
|
|
|
---
|
|
|
|
## Access
|
|
|
|
- **SSH to VWP_ADSRVR:** `ssh vwp\guru@192.168.0.25` (ed25519 key auth — key added 2026-04-13)
|
|
- **Double-hop to VWP-QBS:** Via WinRM — `Invoke-Command -ComputerName VWP-QBS -Credential $cred` using `vwp\sysadmin` PSCredential from ADSRVR. SSH won't forward Kerberos for domain double-hop.
|
|
- **HP iLO power management:** Paramiko required (not system OpenSSH). SSH to `172.16.9.125:22`. Use `disabled_algorithms={'pubkeys': ['rsa-sha2-256', 'rsa-sha2-512']}`. Command: `start system1` to power on.
|
|
- **VWP-QBS DRAC:** IP undocumented — needs to be recorded. DRAC functional.
|
|
- **VPN:** Connect to VWP OpenVPN (UDM) first; this provides access to both the 172.16.9.0/24 and 192.168.0.0/24 subnets.
|
|
- **Vault paths:** `clients/valleywide/` (confirmed entries: `adsrvr`, `dc1`, `udm`, `xenserver`, `quickbooks-server-idrac`). Access via `bash "$VAULT" get-field clients/valleywide/<entry> <field>`.
|
|
|
|
---
|
|
|
|
## App Modernization Project
|
|
|
|
VWP's core business application is a custom-built construction ERP. The original developer (known as "Darv") is deceased. The app is hitting the 2GB Jet/Access database file size limit. ACG was engaged to assess modernization feasibility.
|
|
|
|
### Application Stack (Confirmed)
|
|
|
|
| Layer | Technology | Evidence |
|
|
|---|---|---|
|
|
| Frontend / logic | Visual Basic 6.0 | `frmPayroll.frm` source file, `.frx` resource files, `VB5!` header in exe |
|
|
| Compilation | **P-Code** (not Native Code) | Entry point `PUSH+CALL` to ThunRTMain by ordinal — not native binary |
|
|
| Database | MS Access Jet 3.x (.mdb) | `VWP.mdb` version byte 0x00, Access 97 format |
|
|
| Reporting | Crystal Reports 8.5 | 791 `.rpt` files (per 2026-04-27 archive); Crystl32.OCX import; SCR85Dev installer found |
|
|
| Installer | InstallShield Denali 2021 | `Denali2021v1` folder on server |
|
|
| OCX controls | TABCTL32, mscomct2, comdlg32, Flp32a30, odg7, todg7 | PE import table |
|
|
|
|
**P-Code is the best possible outcome for decompilation.** VB Decompiler Pro (~$200) can recover 70-80% of source including form layouts, procedure names, string literals, and all SQL queries. Decompilation was approved as the next step.
|
|
|
|
### Database: VWP.mdb
|
|
|
|
- **Current size:** 938 MB (last written 2026-04-24). Growth: 671 MB (2020) → 761 MB (2022) → 938 MB (2026). **Approaching the 2 GB Jet hard limit.**
|
|
- **Format:** Jet 3.x / Access 97. Modern ACE/DAO drivers refuse to open it — binary scan was used for schema extraction.
|
|
- **Scale:** ~130 production tables spanning a full construction ERP.
|
|
|
|
#### Domain Coverage
|
|
|
|
| Domain | Key Tables |
|
|
|---|---|
|
|
| Projects & Jobs | tblPROJECT, tblLOTINFO, tblPLANS, tblCHANGE, tblSZONE |
|
|
| Work Orders & Estimating | tblORDERS, tblTAKE, tblMEASURE, tblPlanBill |
|
|
| Inventory & Purchasing | tblINVPRICE, tblINVTRY, tblSUPPLIER, tblPOrder, tblYardOrder |
|
|
| Crew & Payroll | tblCREW, tblHRDAILY, tblPAYHEADER, tblPAYROLL, tblCREWRATE |
|
|
| **Certified Payroll** | **tblCERTIFIED** — government / prevailing wage work. **HARD requirement.** |
|
|
| Accounts Receivable | tblARMASTER, tblARINVOICE, tblARTRANS |
|
|
| Accounts Payable | tblAPMASTER, tblAPTRANS, tblJOBCOST, tblCHECKREC |
|
|
| **Positive Pay (3 banks)** | **tblPosPayVWP, tblPosPayCRD, tblPosPaySWI** — fraud-prevention bank integration. **HARD dependency.** |
|
|
| Scaffold | tblScaffold, tblSC_Crew |
|
|
| Repairs | tblREPAIR, tblRepList |
|
|
| System / Config | tblSECURITY, tblSYSInfo, tblGLAcct |
|
|
|
|
**Modernization complexity: HIGH.** 791 Crystal Reports files, certified payroll (legal compliance — cannot be dropped), positive pay integration with 3 banks, and full AR/AP/Payroll.
|
|
|
|
### Source Code Status
|
|
|
|
The production exe (`Orders_10A.exe`, 13.4 MB) has four shortcuts pointing to it. The original source was on Darv's personal development machine — only one form file (`frmPayroll.frm`, 32 KB) was found on the server at `C:\Users\sysadmin\Desktop\Darv\Source\VWP\`. The remainder of `C:\Users\sysadmin\Desktop\Darv\` (13,231 files, 15.6 GB) includes Darv's installer projects, Crystal Reports, and personal files. VB6 source (`.vbp`, `.frm`) was scanned across multiple server drives (D: and two additional drives as of 2026-05-16). Substantial VB6 source exists across the drives (thousands of `.frm` and `.vbp` files); Mike was searching to confirm which are for the VWP application specifically.
|
|
|
|
### Project Status (as of 2026-04-27)
|
|
|
|
| Task | Status |
|
|
|---|---|
|
|
| Stack identification | Complete — VB6 P-Code + Jet 3.x confirmed |
|
|
| Schema mapping (table names) | Complete (~130 tables via binary scan) |
|
|
| Full schema with field types | Pending — needs Access 97/2000 environment or Jet 3.x → Jet 4.x conversion |
|
|
| VB6 source search across server drives | In progress — Mike searching |
|
|
| VB Decompiler Pro purchase and run | Pending ($200 investment) |
|
|
| Crystal Reports audit (791 .rpt files) | Pending |
|
|
| VWP staff workflow interviews | Pending |
|
|
| Feasibility / modernization report | Pending |
|
|
|
|
---
|
|
|
|
## Patterns & Known Issues
|
|
|
|
### iLO Access (Non-Standard)
|
|
|
|
The HP ProLiant iLO at 172.16.9.125 uses legacy SSH host key algorithms (`ssh-rsa`/`ssh-dss`) that are rejected by modern OpenSSH on Windows by default. **Do not use system OpenSSH to connect.** Use Python paramiko with:
|
|
|
|
```python
|
|
transport.disabled_algorithms = {'pubkeys': ['rsa-sha2-256', 'rsa-sha2-512']}
|
|
```
|
|
|
|
Power-on command: `start system1`.
|
|
|
|
### RDS Double-Hop Pattern
|
|
|
|
SSH to ADSRVR (192.168.0.25) works fine with ed25519 key. But you cannot forward Kerberos over SSH to reach VWP-QBS — the WinRM double-hop must be done inside the SSH session using explicit PSCredential:
|
|
|
|
```powershell
|
|
$cred = Get-Credential # vwp\sysadmin
|
|
Invoke-Command -ComputerName VWP-QBS -Credential $cred -ScriptBlock { ... }
|
|
```
|
|
|
|
### 192.168.0.0/24 Subnet Conflict
|
|
|
|
VWP's AD/phone subnet (`192.168.0.0/24`) is the same RFC1918 range as IMC (another ACG client). When switching between client VPN contexts, verify which 192.168.0.x addresses are being targeted. This is a silent risk — wrong subnet = wrong client.
|
|
|
|
### Syncro Billing for Prepaid Block Emergency
|
|
|
|
Do not use product 26184 (Labor - Emergency) for prepaid block customers. That product has the 1.5x rate baked in, which would result in double-charging when combined with the surcharge line item pattern. Always use product 1190473 for both normal and surcharge line items.
|
|
|
|
### AD Account: `scanner`
|
|
|
|
The `scanner` AD account is used by some device or process (original purpose unknown). Its password was last set 2024-10-17. During the 2026-04-13 brute-force incident, it was being locked out every ~20 minutes by attacker attempts through the public-facing RDWeb. **Password rotation is an outstanding hygiene item.**
|
|
|
|
### LastLogonDate Anomaly
|
|
|
|
VWP-QBS AD object showed `LastLogonDate: 9/28/2049` — flagged as a time-skew artifact during 2026-04-13 incident. Likely cosmetic.
|
|
|
|
---
|
|
|
|
## Active Work (as of 2026-05-12)
|
|
|
|
| Item | Status | Priority |
|
|
|---|---|---|
|
|
| App modernization: VB Decompiler Pro run against Orders_10A.exe | Pending — decompiler not yet purchased | High |
|
|
| App modernization: Full schema extraction with field types | Pending — needs Access 97/2000 environment | High |
|
|
| App modernization: VB6 source search across server drives | In progress | High |
|
|
| RDS CAL purchase (Windows Server 2022 Per User, sized to user count) | Outstanding — grace period may expire | High |
|
|
| HP iLO reconfiguration (post factory-reset 2026-04-22) | [unverified — may have been configured during 2026-04-22 onsite; confirm credentials in vault] | Medium |
|
|
| UPS assessment for HP ProLiant | Outstanding since 2026-04-22 | Medium |
|
|
| Yealink phone fleet provisioning (11 pending phones) | Outstanding — 11 of 16 phones never connected to YMCS | Medium |
|
|
| `scanner` AD account password rotation | Outstanding since 2026-04-13 | Low |
|
|
| UDM UPnP audit | Outstanding since 2026-04-13 | Low |
|
|
| DRAC IP documentation for VWP-QBS | Not yet recorded | Low |
|
|
|
|
---
|
|
|
|
## Security Posture
|
|
|
|
### 2026-04-13: RDWeb Brute-Force Incident
|
|
|
|
RDWeb (`https://VWP-QBS/RDWeb/Pages/login.aspx`) was publicly exposed via UDM port-forward on port 443. A distributed brute-force botnet (residential proxy infrastructure, IPs from China, Belarus, UAE, and others) was hammering `POST /RDWeb/Pages/en-US/login.aspx` at ~6 req/min, hitting usernames `scanner`, `Guest`, `Receptionist`. This triggered AD lockouts every ~20 minutes (lockout threshold 5, 16-min window) which initially appeared to be a stale internal credential problem.
|
|
|
|
**Resolution:** UDM port-forward removed (same day), IIS reset to drain in-flight sessions, lockout policy restored. 30-day audit of Event 4624 confirmed **zero successful external logons — no compromise**.
|
|
|
|
**Current state:** RDWeb accessible from VPN and internal LAN only (port 443 on VWP-QBS, 172.16.9.0/24). Not reachable from public internet.
|
|
|
|
**Outstanding recommendation:** If RDWeb must be re-exposed publicly, require: IPBan (https://github.com/DigitalRuby/IPBan), firewall restriction to known source IPs, and 2FA/Conditional Access.
|
|
|
|
### 2026-04-22: Power Outage / NVRAM Corruption
|
|
|
|
Power outage caused HP ProLiant NVRAM corruption (BIOS/iLO factory reset). VWP-QBS Dell server had a boot retry loop (resolved via DRAC). XenServer (older Dell) was offline. All recovered onsite. **Root cause: no UPS on HP server.**
|
|
|
|
---
|
|
|
|
## History Highlights
|
|
|
|
| Date | Event |
|
|
|---|---|
|
|
| 2026-04-13 | RDWeb brute-force incident discovered and contained. SSH key deployed to ADSRVR. 30-day audit — no compromise. |
|
|
| 2026-04-13 | Domain lockout policy temporarily disabled during diagnosis (threshold=0), restored to 5/16min/16min. 15-minute window of reduced lockout protection. |
|
|
| 2026-04-16 | RDS reconfigured to VPN-only (gateway removed). UDM DNS typo fixed (`qwp-qbs` → `vwp-qbs`). RDS licensing mode set Per User, pointed at local license server. |
|
|
| 2026-04-22 | Emergency onsite: power outage, HP ProLiant NVRAM corruption + iLO factory reset, VWP-QBS boot loop (DRAC), XenServer offline. All resolved ~12:00 MST. |
|
|
| 2026-04-22 | Yealink SIP-T54W fleet (16 devices) added to YMCS device management. 5 previously-provisioned, 11 pending. |
|
|
| 2026-04-27 | App modernization project initiated. VB6 P-Code + Jet 3.x stack confirmed. ~130 table schema extracted via binary scan. Crystal Reports 8.5 (791 .rpt files) documented. |
|
|
| 2026-05-12 | HP ProLiant found powered-off (ADSRVR unreachable). Powered on remotely via iLO paramiko. Syncro ticket #32269, invoice #67594, 1.5 hr block deduction (10.0 hrs remaining). |
|
|
|
|
---
|
|
|
|
## Compilation Notes
|
|
|
|
**Date range covered:** 2026-04-13 through 2026-05-12.
|
|
|
|
**Items flagged [unverified]:**
|
|
- M365 MFA and mail flow configuration — never investigated
|
|
- HP iLO credentials post factory-reset — should be confirmed via vault; iLO was accessible 2026-05-12 so credentials were re-established at some point
|
|
- XenServer resolution detail after 2026-04-22 outage — session log notes it offline/critical, subsequent sessions confirm it was up by 2026-05-12
|
|
- DRAC IP for VWP-QBS — functional but undocumented
|
|
- Yealink provisioning status — 11 phones still pending as of 2026-04-22; no follow-up session
|
|
- RDS CAL grace period expiry timing — unknown; may have already expired
|