From f617c507c442ea1e14dbded912cbec54f3f3e971 Mon Sep 17 00:00:00 2001 From: Howard Enos Date: Mon, 13 Jul 2026 17:32:32 -0700 Subject: [PATCH] sync: auto-sync from HOWARD-HOME at 2026-07-13 17:32:04 Author: Howard Enos Machine: HOWARD-HOME Timestamp: 2026-07-13 17:32:04 --- .claude/memory/MEMORY.md | 4 +- .claude/memory/project_vwp_rose_qb_updater.md | 14 +- ...rd-reliant-phones-dataforth-rdp-signing.md | 171 ++++++++++++++++++ 3 files changed, 178 insertions(+), 11 deletions(-) create mode 100644 session-logs/2026-07/2026-07-13-howard-reliant-phones-dataforth-rdp-signing.md diff --git a/.claude/memory/MEMORY.md b/.claude/memory/MEMORY.md index 34c1cd6b..b5764f0b 100644 --- a/.claude/memory/MEMORY.md +++ b/.claude/memory/MEMORY.md @@ -152,7 +152,7 @@ - [GURU-BEAST-ROG Setup Status](machine_windows_guru_setup_status.md) — Windows workstation fully configured except SSH key deployment to servers. ## Project -- [VWP-ROSE QB updater crash-loop](project_vwp_rose_qb_updater.md) — qbupdate/FCS heap-corruption crash loop on VWP-ROSE (QB Desktop 2024); update skippable, Rose working; resume fix 2026-07-13 4pm; Syncro ticket 113779446; login in vault clients/vwp/vwp-rose-workstation. +- [VWP-ROSE QB updater crash-loop](project_vwp_rose_qb_updater.md) — RESOLVED 2026-07-13 via remote QB repair install; billed 0.5h to VWP prepay block, ticket #32545 Invoiced. Open side findings: NETLOGON 5719 at boot + Syncro agent service crashes on VWP-ROSE. - [GuruRMM software-removal PR #58](project_gururmm_software_removal_pr58.md) — SPEC-030 hardening (driver exclusion in removal UI, lingering-uninstaller sweep, job list + stale reaper) on PR #58, UNMERGED; merge = beta deploy; live testing planned 2026-07-06 on DESKTOP-MS42HNC. - [Cascades network + CS-SERVER SMB instability](project_cascades_network_segments.md) — NAS->CS-SERVER migration. NOT a CSC-ENT-vs-CSCNet issue (corrected): fresh SMB to `\\cs-server\*` fails err 67 from BOTH Wi-Fis; only persistent admin-mapped drives work, intermittently → CS-SERVER-side SMB instability (multichannel advertises .248/.254/IPv6 ULAs; Get-SmbServerConfiguration errors). Blockers: CSCNet=WPA3 (old adapters can't join); workstations store domain-admin cred. Repoint tool [[drive-map]]. - [CyndyOffice physical HP lockups](cyndyoffice-physical-hp-lockups.md) — RMM "Howard-VM" site agent CyndyOffice is a PHYSICAL HP Pavilion TP01 (not a VM); ~20 hard freezes/6wk = Kernel-Power 41 bugcheck-0, no dump/WHEA = hardware (RAM/PSU/BIOS), SSD healthy. UUID re-enrolls. @@ -229,4 +229,4 @@ - [EDR auto-isolates GuruRMM PowerShell](project_edr_rmm_autoisolation_fp.md) — Datto "Exfiltration Over HTTP" rule auto-isolates benign GuruRMM scripts fleet-wide (vwp-qbs 4h outage); watcher + narrow suppression deployed, policy fix pending Mike - [Background tasks — no shell `&`](feedback_background_task_no_ampersand.md) — with run_in_background:true, run the command in the foreground of the shell; adding `&`/`disown` forks + exits 0 instantly, orphaning a blocking wait so it never delivers (hit twice building ask-forum) - [ask-forum — human-in-the-loop via #ct-forum](../skills/ask-forum/SKILL.md) — ask a teammate a question in the private ct-forum Discord forum, get their human answer back in-session; forum-only, any human can answer, run the --wait as a proper background task -- [Claude Code Bun crash -> pin 2.1.110](reference_claude_code_bun_crash_pin.md) - v2.1.114+ segfaults on Win11 at new-session start; npm pin 2.1.110 + DISABLE_AUTOUPDATER=1; unpin on #55219 fix +- [Claude Code Bun crash -> pin 2.1.110](reference_claude_code_bun_crash_pin.md) - v2.1.114+ segfaults on Win11 at new-session start; npm pin 2.1.110 + DISABLE_AUTOUPDATER=1; unpin on #55219 fix diff --git a/.claude/memory/project_vwp_rose_qb_updater.md b/.claude/memory/project_vwp_rose_qb_updater.md index a8f785b8..6bfe2f8b 100644 --- a/.claude/memory/project_vwp_rose_qb_updater.md +++ b/.claude/memory/project_vwp_rose_qb_updater.md @@ -1,16 +1,12 @@ --- name: project-vwp-rose-qb-updater -description: VWP-ROSE QuickBooks updater crash-loop - repair deferred to 2026-07-13 4pm, Syncro ticket 113779446 +description: VWP-ROSE QuickBooks updater crash-loop - RESOLVED 2026-07-13 (repair install); open side findings NETLOGON 5719 + Syncro agent crashes metadata: type: project --- -QuickBooks Desktop 2024 (v34.0.4020) updater on **VWP-ROSE** (Valley Wide Plastering, GuruRMM agent, online) is in a crash loop: `Intuit.QuickBooks.FCS.exe` and `qbupdate.exe` crash with heap corruption (0xc0000374 in ntdll), `qbpatch2.exe` hangs (Event 1002). Pattern survives reboot -> corrupted update cache / damaged FCS on disk. `QBFCService` service is Stopped. +**RESOLVED 2026-07-13:** QuickBooks Desktop 2024 updater crash loop on VWP-ROSE (Valley Wide Plastering) - qbupdate/FCS heap corruption (0xc0000374) surviving reboot. Fixed via remote session: repair on the QuickBooks installation; update now completes. Billed 0.5 hr remote against the VWP prepay block (16.25 hrs remaining), invoice #68041, ticket #32545 Invoiced (https://computerguru.syncromsp.com/tickets/113779446). Rose's login vaulted at `clients/vwp/vwp-rose-workstation`. -Status 2026-07-13 ~11am: Mike ran the basic Tool Hub repair (didn't fix it), found the update can be SKIPPED so Rose can work. **Plan: resume remediation 2026-07-13 at 4:00pm** when Rose is done for the day. - -Remaining fix path (in order): clear `C:\ProgramData\Intuit\QuickBooks 2024\Components\DownloadQB34\` + re-run update with "Reset Update"; if still crashing -> QuickBooks Install Diagnostic Tool + Repair from Programs and Features; fallback -> manually download+install the latest QB 2024 release patch from Intuit. - -- Syncro ticket: https://computerguru.syncromsp.com/tickets/113779446 -- Rose's Windows login: vault `clients/vwp/vwp-rose-workstation` -- Side findings on the box (separate issues): NETLOGON 5719 / DNS registration failures at boot (VWP DC unreachable), Syncro agent services crashed twice morning of 7/13. +**Open side findings on VWP-ROSE** (not yet addressed, from 7/13 event logs): +- NETLOGON 5719 + DNS registration failures at boot - machine can't reach a VWP domain controller at startup; may cause auth/mapped-drive issues. +- Syncro agent services (Syncro.Service.Runner / SyncroLive.Service.Runner) crashed twice on the morning of 2026-07-13 (KERNELBASE .NET exceptions). diff --git a/session-logs/2026-07/2026-07-13-howard-reliant-phones-dataforth-rdp-signing.md b/session-logs/2026-07/2026-07-13-howard-reliant-phones-dataforth-rdp-signing.md new file mode 100644 index 00000000..aaa53fa9 --- /dev/null +++ b/session-logs/2026-07/2026-07-13-howard-reliant-phones-dataforth-rdp-signing.md @@ -0,0 +1,171 @@ +# Session Log — 2026-07-13 — Reliant/Farwest phone outage investigation + Dataforth RemoteApp printer-redirection fix + +## User +- **User:** Howard Enos (howard) +- **Machine:** Howard-Home +- **Role:** tech + +## Session Summary + +Investigated a phone outage reported by Channa (Reliant Well Drilling and Pump, Syncro cid +10736261): Willcox rings once and disconnects, Tucson rings then rolls to the on-call cell; +she suspects numbers are "not being forwarded to the system." Established that Reliant is +billed 15 "Packetdial - Phone Extension" seats monthly (Syncro schedule 144683, + 4x E911 + +~$140.83 pass-through taxes) but is NOT present anywhere on ACG's OIT/NetSapiens reseller +platform: reseller territory 91912.service contains exactly 4 domains / 18 users (0000 DID +holding, arizonacomputerguru, russo, vwp) with none of Reliant's DIDs, and Yealink YMCS has +only VWP/GuruHQ/Ace Pick Up Parks sites. Per Mike's hint, searched "Farwest Drilling" — +found Syncro customer Farwest Well Drilling (id 140212, also Channa) with phone-project +history back to 2017, but no Farwest domain on the PBX or YMCS either. Attempted NetSapiens +password-grant auth with the OIT Prov_Admin credential (vaulted 7/9) — fails with +Invalid_client OA051 (needs an OAuth client_id we don't have; it is a GUI portal login). +Conclusion: Reliant/Farwest's 15 extensions live on OIT's side outside our reseller API +territory (or a legacy platform only Mike knows). Confirming requires the OIT manager +portal GUI (Prov_Admin) or Mike. Syncro ticket preview prepared but NOT yet created +(awaiting confirm); question for Mike (where is the account hosted) not yet posted. + +Second task: dlopez on DESKTOP-U8PHS72 (Dataforth D2) could not enable the Printers checkbox +in the RemoteApp security warning when launching the Sage RemoteApp (!sbtlaunch on +SAGE-SQL.INTRANET.DATAFORTH.COM, gateway apps.dataforth.com). Root-caused to the April 2026 +Windows security update (KB5083769 / CVE-2026-26151): unsigned .rdp files now get all +resource redirection disabled by default, plus a known multi-monitor dialog rendering bug +(Sage.rdp sets use multimon:i:1). Verified server side is fine (RDS collection "Dataforth" +on SAGE-SQL has ClientPrinterRedirected=1, EasyPrint on; drives/PnP deliberately off) and +the .rdp itself grants printers (redirectprinters:i:1). Applied the Microsoft-documented +interim fix via GuruRMM: RedirectionWarningDialogVersion=1 (DWORD) under +HKLM\Software\Policies\Microsoft\Windows NT\Terminal Services\Client on DESKTOP-U8PHS72 +(cmd 47ccb98b, verified value readback; bot alert posted). + +Designed the permanent fix (sign .rdp files + trusted-publisher GPO). Verified Dataforth +domain has NO AD CS (role absent on AD1, no enrollment services in AD). Mike suggested using +"our signing cert" — identified it as Azure Trusted Signing account gururmm-signing (vault +services/azure-trusted-signing) and proved live that it cannot sign .rdp files: rdpsign.exe +supports only /sha256 (no dlib/KSP/cloud provider), Set-RDCertificate +needs a PFX (Trusted Signing keys are non-exportable HSM-held), and the ARM API shows the +profile has issued 89 distinct 3-day certs since 2026-04-16 (current 60ADBE88... expires +7/16) — thumbprint churn breaks publisher-pinning GPO by design. Recommendation stands: +dedicated self-signed code-signing cert on SAGE-SQL + GPO trust (Trusted Root + trusted +.rdp publisher thumbprints), sign RD publishing + saved .rdp files, verify on an untouched +machine, then revert the interim reg value on DESKTOP-U8PHS72 (user correctly noted the reg +fix must be reverted before testing or it masks the result). + +Also vaulted a plaintext credential found in Syncro customer notes (Channa's email +password) and logged a Defender-killed-curl friction event. + +## Key Decisions + +- Reliant phone investigation: exhausted API surface (NetSapiens reseller key sees only + 4 domains; YMCS parent account has 3 client sites) before concluding the account is + outside our territory — did not guess at a platform. +- Applied the temporary RedirectionWarningDialogVersion=1 registry fix on the single + affected machine only (not fleet-wide) because Microsoft documents it as deprecated-to-be; + permanent fix is signing + GPO. +- Rejected Azure Trusted Signing for .rdp signing with live evidence (rdpsign option + surface + ARM cert-profile query) rather than assumption, after Mike suggested it. +- Chose self-signed cert + GPO-pushed trust over buying an OV cert: all Dataforth clients + are domain-joined, so public trust adds nothing; static thumbprint lasts years. +- Test plan ordering (user's catch): verify signing on an untouched machine FIRST, then + revert the interim reg fix on DESKTOP-U8PHS72 and re-test — the reg fix masks results. + +## Problems Encountered + +- NetSapiens password-grant with Prov_Admin failed: Invalid_client [OA051] — needs an OAuth + client_id; the credential is a portal GUI login. No API path to OIT-side accounts. +- Local Windows Defender killed /mingw64/bin/curl mid-session (Permission denied) breaking + an RMM dispatch; Howard disabled Defender to continue. Logged as --friction. REMINDER: + re-enable Defender on Howard-Home. +- First AD1 dispatch included certutil -config - -ping which would have hung interactively; + rewritten without it on retry. +- rdpsign /? exits non-zero under PowerShell NativeCommandError while still printing usage — + cosmetic. + +## Configuration Changes + +- DESKTOP-U8PHS72 (Dataforth D2, agent 25a75932-31cf-4b33-b94b-f4c404830877): + HKLM\Software\Policies\Microsoft\Windows NT\Terminal Services\Client + RedirectionWarningDialogVersion = 1 (DWORD) — TEMPORARY, revert during permanent-fix + verification (RMM cmd 47ccb98b-600e-4026-88a7-68df8a2e1f5b). +- Vault: created clients/reliant/channa-email.sops.yaml (see Credentials). +- errorlog.md: +1 --friction entry (bash/env, Defender killed curl). + +## Credentials & Secrets + +- clients/reliant/channa-email.sops.yaml — Channa Vaught email channa@reliantpump.services, + password 20Password20& (found in plaintext in Syncro customer notes cid 10736261; + recommend scrubbing the Syncro note and rotating). Vault commit/push pending next vault sync. +- Used (read-only): msp-tools/oitvoip (nsr_hSGUB5Wo reseller key), msp-tools/ + oitvoip-provisioning (Prov_Admin — GUI portal login, NOT API-usable without client_id), + services/azure-trusted-signing (SP 516d0bdc-5416-4d02-8521-b70e2bb26d29 token-tested OK + against ARM). +- Syncro customer notes for Reliant also contain a UNMS key (wss://unms.azcomputerguru.com) + — left in place, already infrastructure-known. + +## Infrastructure & Servers + +- PacketDial/NetSapiens reseller 91912.service: domains 0000.91912.service, + arizonacomputerguru, russo.91912.service, vwp.91912.service; 18 users total. Reliant and + Farwest NOT present. DID holding domain has 15203725901-09 (unassigned). +- YMCS parent "Arizona Computer Guru LLC": sites VWP, GuruHQ, Ace Pick Up Parks (live 7/13). +- Syncro: Reliant cid 10736261 (main 5202760913, channa@reliantpump.services); Farwest Well + Drilling cid 140212 (5202939778, channa@farwestwell.com); Reliant schedules 144683 + (PacketDial phones) + 54149 (GPS Pro). +- Dataforth: SAGE-SQL 192.168.0.153 (RDS broker/web/session host; collection "Dataforth"; + TSGateway role installed but service Stopped/Disabled), agent 120ba7bf; AD1 agent bf7bc5ee + (no AD CS); DESKTOP-U8PHS72 agent 25a75932 (D2, users kellynwackerly + dlopez); + apps.dataforth.com = 67.206.163.122 (public WAN). Saved .rdp files: + C:\Users\kellynwackerly\Desktop\{Sage,Stonefield Query}.rdp; dlopez uses shortcuts + Sage-REMOTE/Stonefield Query .lnk. +- Azure Trusted Signing: account gururmm-signing (westus2, sub e507e953-2ce9-4887-ba96- + 9b654f7d3267), profile gururmm-public-trust: 89 certs issued since 2026-04-16, 3-day + lifetime, current thumbprint 60ADBE88EFF0E3C508F5001EA10FCC7574EF1DFB expires 2026-07-16. + +## Commands & Outputs + +- `ns.py domains` / `resellers` → 4 domains, users-total 18 (proves Reliant absent). +- `ns.py domain reliant|farwest|...` → HTTP 404 for all name guesses. +- Password grant POST /ns-api/v2/tokens (Prov_Admin) → {"code":400,"message":"Invalid_client [OA051]"}. +- SAGE-SQL: Get-RDSessionCollectionConfiguration -Client → ClientPrinterRedirected 1, + RDEasyPrintDriverEnabled 1, ClientDeviceRedirectionOptions AudioVideoPlayBack,SmartCard, + Clipboard; WinStations RDP-Tcp fDisableCpm=0, fDisableCdm=1, fDisablePNPRedir=1. +- Sage.rdp contents: redirectprinters:i:1, redirectclipboard:i:1, redirectdrives:i:1, + use multimon:i:1, gatewayusagemethod:i:2, signed=NO (no signscope/signature lines). +- AD1: Get-WindowsFeature AD-Certificate,ADCS-* → all False; Enrollment Services container + → none published. +- Registry fix dispatch cmd 47ccb98b → exit 0, "RedirectionWarningDialogVersion = 1". +- rdpsign /? → options only /sha256 HASH, /q, /v, /l. +- ARM GET certificateProfiles/gururmm-public-trust → 89 certificates, all 3-day validity. + +## Pending / Incomplete Tasks + +- [ ] Reliant phone outage: URGENT Syncro ticket prepared, NOT created (awaiting confirm). +- [ ] Ask Mike where Reliant/Farwest's phone account is actually hosted (OIT manager portal + with Prov_Admin should show it if OIT-side). Then diagnose the one-ring-disconnect / + forward-to-cell symptoms; likely carrier/trunk side. +- [ ] Consider migrating Reliant's phone account under ACG's 91912.service territory for + API manageability. +- [ ] Dataforth RDS signing project: Syncro ticket preview ready, NOT created. Phases: + self-signed code-signing cert on SAGE-SQL (vault PFX password at + clients/dataforth/rds-publishing-cert) → Set-RDCertificate -Role RDPublishing → + GPO on AD1 (Trusted Root + Trusted Publishers import + "Specify SHA1 thumbprints of + certificates representing trusted .rdp publishers") → rdpsign saved desktop .rdp + files → verify on untouched machine → revert RedirectionWarningDialogVersion on + DESKTOP-U8PHS72 → dlopez re-test. +- [ ] Await Mike: does an older exportable code-signing PFX exist? (Azure Trusted Signing + proven unsuitable.) +- [ ] Scrub Channa's plaintext password from Syncro customer notes; consider rotation. +- [ ] RE-ENABLE WINDOWS DEFENDER on Howard-Home (disabled mid-session to unblock curl); + consider exclusion for Git-for-Windows /mingw64/bin. +- [ ] Sync vault repo (new clients/reliant/channa-email entry uncommitted). + +## Reference Information + +- Syncro: https://computerguru.syncromsp.com/customers/10736261 (Reliant), + customers/140212 (Farwest). Old phone tickets: #21327 (2019 switchover), #22455 (2019 + number port), #16163 (2017 Farwest phone project). +- MS doc on the RDP warning change: https://learn.microsoft.com/en-us/windows-server/remote/remote-desktop-services/remotepc/understanding-security-warnings + (CVE-2026-26151, KB5083769). Interim key: HKLM\Software\Policies\Microsoft\Windows NT\ + Terminal Services\Client : RedirectionWarningDialogVersion=1 (may be removed by future update). +- Multi-monitor greyed-checkbox workaround KB: https://support.dom.edu/TDClient/2074/Portal/KB/Article/172888 +- RMM commands: 9b387efc (SAGE-SQL collection read), 855a94cb/(follow-ups) DESKTOP-U8PHS72 + recon, 47ccb98b (reg fix write), af4c5dc9 (AD1 AD CS check). +- Bot alerts: #dev-alerts message 1526359366276616292 (reg fix dispatch).