Files
claudetools/clients/valleywide/session-logs/2026-06/2026-06-13-mike-vwp-server3-migration-and-orders-source-recovery.md
Mike Swanson 6f9520ff35 sync: auto-sync from GURU-5070 at 2026-06-14 06:29:50
Author: Mike Swanson
Machine: GURU-5070
Timestamp: 2026-06-14 06:29:50
2026-06-14 06:30:07 -07:00

14 KiB
Raw Permalink Blame History

Session 2026-06-13 — VWP SERVER3 retirement, G: migration to VWP-FILES, Orders VB6 source recovery (+ GuruRMM NT 6.0 legacy agent)

User

  • User: Mike Swanson (mike)
  • Machine: GURU-5070
  • Role: admin

Session Summary

Started on the GuruRMM legacy-agent floor: SERVER3 rejected the legacy agent with "not a valid Win32 application." Confirmed the cause is the PE OS/subsystem version stamp (6.1 = Win7/2008 R2) vs the box's NT version. Built a 6.0-stamped legacy agent on Pluto (the office Windows builder) so the agent loads on Server 2008 / NT 6.0 — needed for Glaztech's pre-R2 2008 servers. Two real fixes were required: pin zeroize = =1.8.1 (1.9.0 needs edition2024, which Rust 1.77 can't parse — a latent break in the production legacy pipeline) and add -C link-arg=/SUBSYSTEM:CONSOLE,6.00. Header verified 6.0/6.0, console subsystem preserved.

SERVER3 (the VWP XenServer "server 2003" VM) was upgraded in-place to Server 2008 by Mike. That broke networking (PV NIC gone) and then bricked with 0x7B INACCESSIBLE_BOOT_DEVICE after a XenTools install. Recovered it: safe-mode XenTools uninstall, then a clean reinstall with the xenbus/ xenfilt/xenvbd boot-start fix applied before reboot. Came up healthy (PV 7.1 build 1270, NIC up on 192.168.0.20). Snapshots taken at each safe point.

The main work: migrated the G: file share off SERVER3 onto a new Hyper-V file server, VWP-FILES (pre-staged Gen2 Server 2019 VM on VWP-HYPERV1). Block-migrated the 100 GB G: VDI (XenServer vdi-export streamed straight to HYPERV1 over HTTPS, converted to VHDX), attached, brought online, recreated all 19 SMB shares with exact perms (via WMI ACL extraction — the registry-import route was blocked by SMB admin-share refusal), repointed the MappedDrives GPO G: → \\VWP-FILES\G-drive (and cleaned two stale G: maps), and did the IP takeover so VWP-FILES holds 192.168.0.20 (VLAN 2 "Old Net") for IP-based stragglers while serving the new net on 172.16.9.132. Verified, billed 3.5 h remote on #32418, prepay 24.0 → 20.5.

Finally, hunted for the lost VWP "Orders" VB6 source (#32280). It was not loose on any server — but Darv's full machine backup (F:\Darv\Darv.rar, 51 GB, on WINFileSvr) contained the complete Orders project tree. Staged 12.2 MB of pure source (147 .frm, 4 .bas, 5 .vbp; newest ORDERS_C.vbp 2020-06-09) into the repo. The source is fully recovered — VB Decompiler is no longer needed.

Key Decisions

  • 6.0 PE stamp, not a separate agent SKU. A 6.0-stamped image still loads on Win7→11, so one legacy binary covers 2008 → 11 — just lower the floor. Pin zeroize=1.8.1 to clear the edition2024 wall on Rust 1.77 (mirrors the existing uuid/url pins).
  • Block-migrate G:, not file-copy. ~100 GB of many small files → a whole-volume image move (VDI export → VHDX) is far faster and preserves NTFS/perms/timestamps. Streamed the export directly XenServer→HYPERV1 (dom0 has only ~15 GB scratch; can't stage 100 GB locally).
  • WMI ACL extraction for shares, not registry import. SERVER3's admin share (C$) refused (error 67) and reg export via WMI didn't land, so recreated all 19 shares with New-SmbShare
    • the exact share-level ACLs pulled via Win32_LogicalShareSecuritySetting. Result identical (mostly Everyone Full; Shelly read-only; G-drive +Domain Users).
  • GPO points G: at \\VWP-FILES by name, decoupled from the .20 IP takeover, so it survives regardless. Disabled DNS registration on the .20 NIC so the name resolves only to 172.16.9.132 (avoids new-net clients hitting the asymmetric-route dead end on .20).
  • IP takeover via VLAN tag, no cabling. UDM "Old Net" = VLAN 2; HYPERV1's uplink already trunks it (UniFi default), so a tagged 2nd vNIC + .20 was fully remote — no on-site patch needed.
  • Don't decompile — recover the source. Pursued Darv.rar over the VB-Decompiler path once the full .vbp tree turned up; we now have real, buildable 2020 source.

Problems Encountered

  • "not a valid Win32 application" on SERVER3 — PE stamped NT 6.1; box was below the floor. Fixed by building a 6.0-stamped agent (and SERVER3 was upgraded to 2008 anyway).
  • 0x7B INACCESSIBLE_BOOT_DEVICE after XenTools on the 2008 guest — xenvbd took the boot disk without being boot-critical. Fixed by setting xenbus/xenfilt/xenvbd Start=0 before reboot.
  • zeroize 1.9.0 edition2024 broke the 1.77 legacy build — pinned =1.8.1. (Production legacy pipeline will hit the same; the pin should be committed to agent/Cargo.toml.)
  • RMM JSON transport eats one backslash from UNC literals — bit me 3× (GPO path stored as \VWP-FILES, Test-Path failures). Worked around by building backslashes from [char]92.
  • Asymmetric routing to .20 — dual-homed VWP-FILES replies via its 172.16.9 NIC, so cross-subnet (DC, my VPN) can't reach .20; only same-subnet Old-Net clients can. Acceptable: stragglers are Old-Net, new-net uses the GPO \\VWP-FILES. Used 172.16.9.132 for my own file pulls.
  • SMB admin-share (C$) refusal + reg-export-via-WMI not landing on SERVER3 → pivoted to WMI ACL extraction for the share migration.
  • Per-site MSI installer endpoint 500/"Missing authorization" when enrolling VWP-FILES — Mike pushed the agent via ScreenConnect instead (now online, site Main Office).

Configuration Changes

  • VWP-FILES (Hyper-V VM on VWP-HYPERV1): checkpoint merged (flat VHDX), AutomaticCheckpoints disabled, RAM 4→16 GB, install ISO ejected. Added 2nd vNIC OldNet-VLAN2 (AccessVlanId 2), static 192.168.0.20/24 (no gateway), DNS registration disabled on that NIC. Attached migrated C:\VHD\server3-g.vhdx (SCSI0:2) as G:. 19 SMB shares created. GuruRMM agent installed (via SC).
  • GPO MappedDrives (VWP.US): G: drive map \\192.168.0.20\G-drive\\VWP-FILES\G-drive; version bumped (gpt.ini + AD) to 4980736.
  • GPO Default Domain Policy: removed stale G: map (\\192.168.0.96\g$); version → 1114165.
  • Orphaned SYSVOL policy {FCB17CA3-155F-4BD3-B633-4E1F5CF01679} (no AD object) deleted (backed up to C:\GPO-backup-Gmig-20260613-170451 on VWP-DC1).
  • SERVER3 (XenServer VM 0c73ce00-...): clean-shutdown (retired). Snapshots created: server2008-clean-noTools-20260613 (45d56386), server2008-working-pvtools-20260613 (b2f50da4).
  • Repo: staged clients/valleywide/app-modernization/source-code/Orders-VWP_Current-2020/ (12.2 MB recovered VB6 source); darv-G-finds/ (ORDERS0727-2015.exe, ORDERS-2005.exe, frmPayroll, Package1508); updated app-modernization/CONTEXT.md (G: finds + source recovery).
  • Vault: created clients/vwp/domain-sysadmin (VWP\sysadmin).

Credentials & Secrets

  • VWP\sysadmin (VWP.US domain admin) — r3tr0gradE99# — vaulted at clients/vwp/domain-sysadmin. Used for PowerShell Direct into VWP-FILES + WMI/SMB to VWP servers.
  • VWP XenServer root — r3tr0gradE99! — vault clients/vwp/xenserver (host 192.168.0.104).
  • VWP UDM root — Gptf*77ttb123!@#-vwp — vault clients/vwp/udm (host 172.16.9.1).

Infrastructure & Servers

  • VWP-HYPERV1 172.16.9.184 — Dell R740, Server 2025, 112 vCPU / 255 GB RAM, C: 10.7 TB (10.37 free), Hyper-V, one External vSwitch on Intel 10G NIC (NIC1 linked 1 Gbps; NIC2/3/4 spare). VHDs in C:\VHD. RMM agent bdc3e142-1eb3-4034-8ac2-300f46dbc1d9.
  • VWP-FILES — Hyper-V Gen2, Server 2019, now 16 GB RAM; IPs 172.16.9.132 (primary) + 192.168.0.20 (VLAN 2). Serves G: (NTFS "G Drive", ~88 GB used). RMM site Main Office.
  • VWP XenServer 192.168.0.104 — XenServer 7.6, PowerEdge R720. SERVER3 VM 0c73ce00-c4cc-c549-5f55-98657a66b203 (now off). G: VDI 828ea0ff-... (100 GB), snapshot for export 9ee0e831-....
  • DC 192.168.0.25 (VWP_ADSRVR) — Old Net DC, Server 2019, healthy (LDAP/SMB/DNS up).
  • VWP-DC1 172.16.9.x — Server 2019 DC (RMM 8eefbba6-...).
  • WINFileSvr 192.168.0.35 — Server 2019 file server; holds O:/P: shares AND F:\Darv\Darv.rar (the source backup) + F:\Darv\Darv-rar (135 GB extract). RMM 62db0264-88c2-44db-996d-84996afe8706.
  • UDM 172.16.9.1 — networks: Default 172.16.9.0/24 (untagged), Old Net 192.168.0.0/24 = VLAN 2 (gw .1, DHCP .100-.199, DNS=192.168.0.25 + 8.8.8.8), Management 192.168.3.0/24 VLAN 99, OpenVPN .4.0/24.
  • Pluto 172.16.3.36 — office Windows builder (Rust 1.77 + i686; C:\gururmm). GURU-5070 SSH key added to its administrators_authorized_keys this session.

Commands & Outputs

  • 6.0 legacy build (Pluto): cargo +1.77 build --ignore-rust-version --release --features legacy --target i686-pc-windows-msvc with $env:RUSTFLAGS='-C link-arg=/SUBSYSTEM:CONSOLE,6.00', after cargo +stable add zeroize@=1.8.1 and moving agent\Cargo.lock aside. Artifact: C:\Users\guru\Downloads\gururmm-agent-legacy-pre2008-x86.exe (SHA256 0BB363E2…), header 6.0/6.0.
  • xenvbd boot-start fix (in-guest, pre-reboot): reg add HKLM\SYSTEM\CurrentControlSet\Services\ {xenbus,xenfilt,xenvbd} /v Start /t REG_DWORD /d 0 /f.
  • G: export (HYPERV1 pulls from Xen): curl.exe -sk -o C:\VHD\server3-g.vhd "https://192.168.0.104/export_raw_vdi?session_id=<OpaqueRef>&vdi=9ee0e831-...&format=vhd" (~100 MB/s, final 99.33 GB), then Convert-VHD … -VHDType Dynamic → server3-g.vhdx (99.38 GB).
  • Share ACLs via Get-WmiObject Win32_LogicalShareSecuritySetting … .GetSecurityDescriptor().
  • Source recovery sweep found …\Kingston\Project\VWP_Current\ORDERS_C.vbp (2020-06-09); project references 80 forms + 10 OCX (TABCTL32, Crystl32, True DBGrid, FarPoint Spread).

Pending / Incomplete Tasks

  • Commit zeroize = "=1.8.1" to guru-rmm/agent/Cargo.toml (production legacy build will break on next run without it). Open as its own fix.
  • Validate the 6.0 legacy agent on a real NT 6.0 box (Glaztech 2008 server, or a lab VM). Then correct SPEC-029 §12 / installer/legacy/README.md ("blocked / NSIS+NSSM") and pursue unified x86-MSI + native-SCM.
  • Modernization (#32280): stand up a VB6 build env, confirm ORDERS_C.vbp compiles from the recovered tree, then scope the modernization from source. Optionally stage VWP_Inv / VWP_Update projects too. Bill #32280 for the recovery search.
  • Cleanup (on sign-off): delete C:\VHD\server3-g.vhd (99 GB intermediate) on HYPERV1, the XenServer G: snapshot, and F:\Darv\Darv-rar (135 GB) once source verified; SERVER3 VM stays off-but-intact as deepest rollback.
  • Commit the staged source + darv-G-finds (this /save syncs them).
  • Beast C:\Temp\gururmm-nsvc* leftovers from the prior native-SCM probe still pending cleanup.
  • Old-Net DHCP secondary DNS = 8.8.8.8 → consider changing to a second internal DC.

Reference Information

  • Tickets: #32418 G-Drive Migration (Invoiced, 3.5 h remote, $0 prepaid; id 112613439). #32280 Source Code Data Recovery (New; id 110305931) — recovery write-up posted.
  • Repo paths: clients/valleywide/app-modernization/source-code/Orders-VWP_Current-2020/, …/source-analysis/darv-G-finds/, …/CONTEXT.md.
  • Source of record: WINFileSvr F:\Darv\Darv-rar (extract) + F:\Darv\Darv.rar (51 GB archive).
  • RMM: http://172.16.3.30:3001 — VWP site "Main Office" 7c18fe2c-38fe-4ebf-8a26-b47745db2d19 (code INNER-TIGER-8330).
  • VWP Syncro customer: 31694734.

Update: 05:40 PT — Orders modernization: research + wiki

  • Inventoried the recovered source (ORDERS_C.vbp): ~40 active forms + modADO (ADO.bas/ADO2.bas data layer); confirmed full construction-ERP domain (Projects/Lots, Takeoffs, Crew/Payroll incl. Certified Payroll, Scaffold, Inventory/PO/Suppliers, AR/AP/Billing/Job-Cost, Positive Pay 3 banks, Repairs, Texture/Foam). Stack from .vbp refs: VB6 + ADO 2.6/Jet, Crystal 7 (craxdrt/Crystl32), True DBGrid Pro 7 (todg7), FarPoint Spread (Flp32a30), TABCTL32/MSCOMCTL/EasyP, MSXML4.
  • Researched modernization paths: VBUC (Mobilize.Net / gapvelocity AI Migrator) VB6->C#/.NET8; Jet->SQL Server; OCX successors all in MESCIUS/GrapeCity family (True DBGrid->FlexGrid w/ migration utility; FarPoint Spread->Spread for WinForms, .ss2->XML; Crystal->Crystal for VS auto-convert).
  • Created wiki article: wiki/projects/valleywide-orders-modernization.md (app domain, stack, recovered-source status + locations, modernization assessment + first concrete steps, sources).
  • Linked from wiki/index.md (Projects table) and wiki/clients/valleywide.md (app-mod section pointer + "source RECOVERED" note superseding the old "lost" text).

Update: 06:16 PT — VWP-FILES RMM enrollment, fleet check, MSP360 backup verified

  • VWP-FILES enrolled in GuruRMM (Mike pushed the agent via ScreenConnect after the per-site MSI endpoint 500'd). Agent id 8e02fbbc-0db1-4044-b4c2-b0732d64f029, online, site "Main Office", v0.6.66.
  • VWP server fleet check (RMM): all 11 enrolled Windows servers online/healthy (VWP-HYPERV1 2025, VWP-QBS/VWP-SERVER 2022, VWP-DC1/VWP_ADSRVR/VWP-FIN/WIN-Server97/WINFileSvr/WIN-AD-SRVR-2/ WIN-Backup-SRV 2019, SERVER19 2012 R2). VWP-FILES had been the only unmonitored server until enrolled.
  • MSP360 backup verified on VWP-FILES via GET /api/agents/<id>/backup-status: provider msp360, plan "Files Backup Plan VWP-FILES", status=success, 103,849 files / 0 failed, ~63.7 GB of ~91 GB copied to Cloud (B2), build 8.6.0.338, last 2026-06-13 18:18 PT, next Jun 15 07:00 UTC. (Only message is Info-level "987 system/hidden files skipped" — normal.)
  • Tagged Winter (<@624666486362996755>) in Discord #bot-alerts re: the new VWP-FILES backup, for her backup monitoring/billing.
  • New RMM API note: GET /api/agents/<id>/backup-status returns the MSP360/CloudBerry backup result (provider, plan_name, status, files_copied/failed, data_copied/total_bytes, last/next_backup_at).