From 0171107d4141e8373c5205270b2836ac2023ff8e Mon Sep 17 00:00:00 2001 From: Mike Swanson Date: Tue, 23 Jun 2026 12:11:12 -0700 Subject: [PATCH] sync: auto-sync from GURU-5070 at 2026-06-23 12:10:19 Author: Mike Swanson Machine: GURU-5070 Timestamp: 2026-06-23 12:10:19 --- ...026-06-23-mike-pbx-no-inbound-calls-fix.md | 74 ++++++++++++++++++ ...23-mike-vwp-qbs-firewall-ccroom1new-uac.md | 78 +++++++++++++++++++ 2 files changed, 152 insertions(+) create mode 100644 clients/dataforth/session-logs/2026-06/2026-06-23-mike-pbx-no-inbound-calls-fix.md create mode 100644 session-logs/2026-06/2026-06-23-mike-vwp-qbs-firewall-ccroom1new-uac.md diff --git a/clients/dataforth/session-logs/2026-06/2026-06-23-mike-pbx-no-inbound-calls-fix.md b/clients/dataforth/session-logs/2026-06/2026-06-23-mike-pbx-no-inbound-calls-fix.md new file mode 100644 index 00000000..97ca5e17 --- /dev/null +++ b/clients/dataforth/session-logs/2026-06/2026-06-23-mike-pbx-no-inbound-calls-fix.md @@ -0,0 +1,74 @@ +## User +- **User:** Mike Swanson (mike) +- **Machine:** GURU-5070 +- **Role:** admin + +## Session Summary + +Dataforth reported no inbound calls to their main line (emergency). Loaded context: Dataforth runs Sangoma/FreePBX 17 (Asterisk 22.5.2) at 192.168.100.2 on VLAN 100, inbound via a FirstDigital SIP trunk (SBC 66.7.123.215, IP-auth, match 66.7.123.0/24, no registration). Prior work (2026-06-09, #32392) established that inbound depends on a UDM-Pro port-forward (WAN 67.206.163.122 UDP 5060 -> 192.168.100.2:5060, source-locked to FD) persisted only in /data/on_boot.d/30-freepbx-sip-forward.sh and never added as a UI rule. + +Confirmed connectivity over the Dataforth OpenVPN (all targets reachable; VPN MTU correctly 1400). Root SSH key worked directly to the UDM (192.168.0.254). Found the smoking gun: the UDM had NO SIP 5060 DNAT rule at all (NO_SIP_DNAT) — flushed by a UniFi provision/update because the rule was never a UI/provision-surviving rule and on_boot.d only re-applies on boot. Verified the persisted script was intact and the correct SIP-only version (RTP forward removed per the 2026-06-09 outbound-audio fix), then re-ran it; verified the DNAT and forward-accept are present. Confirmed the PBX is up and listening on UDP 5060, restoring the full inbound path. Mike later confirmed inbound calls tested OK. + +Updated Syncro ticket #32450 (Dataforth Corp): posted a customer-visible resolution comment, added a 1.5h emergency-remote line item (product 26184; 1h x 1.5 emergency premium in the quantity because Dataforth is prepaid with 31.5 block hours), created the invoice (nets $0.00, debits 1.5 -> block 30.0), set the block-hours invoice note, marked the ticket Invoiced, and posted to #bot-alerts. + +Cleanup/credential work: investigated the PBX SSH password. The ACG SSH key authenticates as sangoma@192.168.100.2 (operational access fine). I initially mis-diagnosed the vaulted password as "stale" off a sudo test that was mangled by shell-quoting of the special chars, then a de-escape that silently no-op'd; Mike corrected that the password is not stale. Confirmed the correct password is Gptf*77ttb!@#!@# and the vault had a backslash-corrupted copy (Gptf*77ttb\!@#\!@#). Corrected the vault entry via sops set (byte-verified, still encrypted) and pushed. Filed coord to-dos for the durable UDM fix. + +## Key Decisions + +- **Re-applied the existing on_boot.d SIP-forward script rather than hand-crafting iptables** — it's idempotent, source-locked, and already the corrected SIP-only version; avoids re-introducing the RTP-forward bug that breaks outbound audio. +- **Did NOT forward the RTP range** — hard rule from 2026-06-09: a static RTP DNAT collides with the PBX outbound RTP conntrack and kills outbound-call audio. SIP 5060 only. +- **Emergency billing for a prepaid customer: product 26184 at qty = actual hours x 1.5** (premium in the quantity; invoice nets $0 and debits the block by quantity) per the 2026-05-27 rule. +- **Did not overwrite the vault with a guessed/unverified password** when I (wrongly) thought it was stale — waited for Mike's authoritative value, then set exactly Gptf*77ttb!@#!@#. +- **Filed the durable fix as a coord to-do tagged to clients/dataforth** so it's reliably findable (an earlier untagged to-do wasn't surfacing in the 100-cap list). + +## Problems Encountered + +- **No inbound calls (root cause):** UDM SIP 5060 DNAT missing — flushed by a UniFi provision; on_boot.d only re-applies on boot. Fixed by re-running the persist script; verified rules present. +- **plink password auth failed to UDM and PBX:** UDM uses keyboard-interactive and the azcomputerguru SSH password path was denied; PBX password rejected. Resolved by using the ACG root SSH key for the UDM and the sangoma SSH key for the PBX. +- **SSH to IX (earlier in day) and rapid retries:** not this incident, but fail2ban/cPHulk sensitivity noted; used pinned host keys / key auth. +- **Mis-diagnosed PBX password as stale:** sudo -S test piped a password full of !@#* through two shell layers (mangled) -> false negative; a base64 "clean" retry also failed, likely because sangoma is in the `sudo` group but not authorized in sudoers (authorization failure, not auth). Mike corrected; the password is correct. Lesson: don't infer credential validity from a sudo test that can fail on authorization or quoting. +- **Vault password backslash-corruption:** stored as Gptf*77ttb\!@#\!@# (shell-escaping leaked into storage). bash `${VAR//\\/}` de-escape silently no-op'd in one attempt; resolved by building the JSON value with jq and `sops set` from inside the vault dir. +- **sops set from wrong dir earlier:** "no creation rules"; must cd into vault root (or use --config). Already logged as friction in a prior session. + +## Configuration Changes + +- **Dataforth UDM 192.168.0.254:** re-applied `/data/on_boot.d/30-freepbx-sip-forward.sh` — restored `UBIOS_PREROUTING_USER_HOOK` SIP 5060 DNAT (WAN 67.206.163.122, src 66.7.123.0/24 -> 192.168.100.2:5060) + `UBIOS_FORWARD_IN_USER` accept for udp/5060 to the PBX. No file edits; rule re-instantiation only. +- **Vault `clients/dataforth/pbx.sops.yaml`:** corrected `credentials.password` from `Gptf*77ttb\!@#\!@#` to `Gptf*77ttb!@#!@#` (sops set; committed vault 0f503d4). +- **Syncro ticket #32450:** comment 420355206, line item 42980800 (26184, qty 1.5), invoice 1650772809 ($0.00), status Invoiced, invoice note "Block hours remaining: 30." +- **Coord to-dos:** created 5cfa4263 (cryoweave reCAPTCHA, prior task), 1a362d8a (closed — carried wrong "stale credential" framing), 45572ee1 (durable UDM SIP-forward fix, clients/dataforth). (8536155f was created but not findable in the 100-cap list; 45572ee1 supersedes it.) +- **Session log:** this file. + +## Credentials & Secrets + +- **Dataforth PBX (FreePBX):** `sangoma@192.168.100.2`, password **Gptf*77ttb!@#!@#** (corrected in vault `clients/dataforth/pbx.sops.yaml`). ACG SSH key (`~/.ssh/id_ed25519`) also authenticates as sangoma. NOTE: sangoma is in the `sudo` group but sudo authorization appears not granted on this distro — privileged ops need verification. +- **Dataforth UDM-Pro 192.168.0.254:** root via ACG SSH key (`~/.ssh/id_ed25519`) works directly over the OpenVPN. Vaulted `azcomputerguru` SSH password was rejected by plink (keyboard-interactive) — key is the reliable path. Vault: `clients/dataforth/udm.sops.yaml`. +- No new secrets created. + +## Infrastructure & Servers + +- **FreePBX:** 192.168.100.2 (pbx.intranet.dataforth.com), Sangoma FreePBX 17 / Asterisk 22.5.2, Debian 12, VLAN 100. Listening UDP 5060. Extensions 201-343. +- **SIP trunk:** FirstDigital, SBC 66.7.123.215:5060 (Sonus), match 66.7.123.0/24, IP-auth (no registration), ignores OPTIONS but answers INVITEs. +- **UDM-Pro:** 192.168.0.254 / 192.168.0.1, UniFi OS. WAN 67.206.163.122. SIP forward persisted at `/data/on_boot.d/30-freepbx-sip-forward.sh` (boot-only re-apply). OpenVPN server, client subnet 192.168.6.0/24 (GURU-5070 = 192.168.6.2). +- **Access path:** GURU-5070 -> Dataforth OpenVPN (adapter MTU pinned 1400) -> 192.168.0.0/24 + 192.168.100.0/24 via 192.168.6.1. +- **D2TESTNAS jump:** 192.168.0.9 (root / Paper123!@#) — not needed this session (direct VPN worked). + +## Commands & Outputs + +- Confirm missing DNAT: `ssh -i ~/.ssh/id_ed25519 root@192.168.0.254 "iptables -t nat -S | grep -iE '5060|192.168.100.2'"` -> NO_SIP_DNAT. +- Re-apply: `ssh ... root@192.168.0.254 "sh /data/on_boot.d/30-freepbx-sip-forward.sh"`; verify `iptables -t nat -S UBIOS_PREROUTING_USER_HOOK | grep 5060` and `iptables -S UBIOS_FORWARD_IN_USER | grep 192.168.100.2`. +- PBX listening check: `ssh -i ~/.ssh/id_ed25519 sangoma@192.168.100.2 "ss -lun | grep 5060"` -> `0.0.0.0:5060`. +- Syncro emergency prepaid: product 26184, qty = hours x 1.5; invoice nets $0, block debits by qty. +- Vault fix: `cd /d/vault; VAL=$(jq -nc --arg p 'Gptf*77ttb!@#!@#' '$p'); sops set clients/dataforth/pbx.sops.yaml '["credentials"]["password"]' "$VAL"` (SOPS_AGE_KEY_FILE=/c/Users/guru/.config/sops/age/keys.txt). + +## Pending / Incomplete Tasks + +- **Durable UDM fix (coord 45572ee1):** make the SIP 5060 forward provision-surviving (UI port-forward or a cron/watcher re-running the idempotent script). Needs a maintenance window. SIP 5060 only — never RTP. +- **PBX sudo authorization:** sangoma is in `sudo` group but sudo appears unauthorized — verify how privileged FreePBX ops are intended to run (root console, asterisk group, etc.). +- **cryoweave reCAPTCHA (coord 5cfa4263):** still parked, awaiting Google reCAPTCHA v3 keys (separate task). + +## Reference Information + +- Syncro ticket: #32450 id 112979216, customer Dataforth Corp 578095. Invoice 1650772809. https://computerguru.syncromsp.com/tickets/112979216 +- Coord to-dos: 45572ee1 (durable UDM fix), 5cfa4263 (cryoweave reCAPTCHA). +- Vault: clients/dataforth/pbx.sops.yaml, clients/dataforth/udm.sops.yaml. Vault commit 0f503d4. +- Prior context: clients/dataforth wiki; session-logs/2026-06/2026-06-09-mike-dataforth-freepbx-safesite-forensics.md; ticket #32392. diff --git a/session-logs/2026-06/2026-06-23-mike-vwp-qbs-firewall-ccroom1new-uac.md b/session-logs/2026-06/2026-06-23-mike-vwp-qbs-firewall-ccroom1new-uac.md new file mode 100644 index 00000000..f5a9b585 --- /dev/null +++ b/session-logs/2026-06/2026-06-23-mike-vwp-qbs-firewall-ccroom1new-uac.md @@ -0,0 +1,78 @@ +# Quick ops — VWP-QBS firewall disabled + Country Club CCroom1New UAC re-enabled + +## User +- **User:** Mike Swanson (mike) +- **Machine:** GURU-5070 +- **Role:** admin + +## Session Summary + +Two quick remediation ops following the VWP SMB1/Orders work (logged separately in +`clients/valleywide/session-logs/2026-06/2026-06-23-mike-vwp-smb1-orders-xp-g-drive.md`), +both executed via the GuruRMM agent fleet. + +1. **VWP-QBS — Windows Firewall disabled.** Per Mike's direction (troubleshooting), disabled all + three firewall profiles (Domain/Private/Public) on VWP-QBS (172.16.9.169, the QuickBooks + + RD Web Access host). Captured the pre-state first (all three enabled) for clean restore. Flagged + the security context (this is the box brute-forced in April, now internal/VPN-only) and that it + should be re-enabled after the test. **The firewall is still OFF — re-enable is pending.** + +2. **Country Club CCroom1New — UAC re-enabled.** Mike reported the machine "acting weird." The + GuruRMM agent had just been installed (it didn't appear at first; Mike confirmed install, then it + showed as `CCroom1New`, Win11 26200, site Country Club). Read the UAC registry state and found UAC + **fully disabled** (`EnableLUA=0`, `ConsentPromptBehaviorAdmin=0`, `PromptOnSecureDesktop=0`) — the + classic cause of Win11 "acting weird" (Store/UWP apps won't launch, Start/search misbehave). + Restored standard Win11 UAC (`EnableLUA=1`, `ConsentPromptBehaviorAdmin=5`, + `PromptOnSecureDesktop=1`), rebooted (required for EnableLUA to take effect), and verified + post-reboot (uptime 1 min, EnableLUA=1 active). Recommended an on-site smoke test. + +## Key Decisions +- **Captured firewall pre-state before disabling** VWP-QBS (all profiles enabled) so it restores to + the exact prior posture; documented the re-enable command. +- **Restored full UAC defaults, not just EnableLUA** on CCroom1New — set ConsentPromptBehaviorAdmin + and PromptOnSecureDesktop back to Win11 defaults too, since all three had been zeroed. +- **Rebooted CCroom1New to apply** — EnableLUA only takes effect after a restart; verified active + after the box came back. + +## Problems Encountered +- **CCROOM1NEW not in RMM initially** — the GuruRMM agent wasn't installed yet (0 hostname matches; + only Country Club's CC1-NEW22 / CC2-NEW22 existed). Mike installed the agent; it then enrolled as + `CCroom1New` and the work proceeded. +- **First post-reboot verify was a false read** — it completed in ~10s showing uptime 108.5 min, + i.e. it ran in the Restart-Computer grace window before the box actually went down. Re-verified + after a short wait: uptime 1 min + EnableLUA=1 confirmed the reboot completed and UAC is active. + +## Configuration Changes +- **VWP-QBS (172.16.9.169):** `Set-NetFirewallProfile -All -Enabled False` (Domain/Private/Public + all OFF). Pre-state: all enabled. **Reversal pending:** `Set-NetFirewallProfile -All -Enabled True`. +- **CCroom1New (Country Club, Win11):** `HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System` + — `EnableLUA` 0->1, `ConsentPromptBehaviorAdmin` 0->5, `PromptOnSecureDesktop` 0->1; rebooted. + +## Credentials & Secrets +- None created or discovered. All actions via GuruRMM agents (SYSTEM context). No interactive creds. + +## Infrastructure & Servers +- **VWP-QBS** 172.16.9.169 — Windows Server 2022, QuickBooks + RD Web Access host (physical Dell), + GuruRMM agent `f3386f0a-b3ee-417e-ace9-995cc1d0662b`. Firewall currently DISABLED. +- **CCroom1New** — Windows 11 (build 26200), site Country Club, GuruRMM agent + `aaa4b694-a464-4961-a8ff-096a5698afee`. UAC re-enabled + active post-reboot. +- Country Club site also has CC1-NEW22, CC2-NEW22 (online). + +## Commands & Outputs +- Firewall: `Get-NetFirewallProfile | Select Name,Enabled` (before: all True) -> `Set-NetFirewallProfile + -All -Enabled False` -> verify all False. +- UAC: `Set-ItemProperty 'HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System' -Name + EnableLUA -Value 1 -Type DWord` (+ ConsentPromptBehaviorAdmin=5, PromptOnSecureDesktop=1) -> + `Restart-Computer -Force` -> post-reboot verify uptime_min 1, EnableLUA 1. +- Dispatched via `/rmm` (RMM API http://172.16.3.30:3001), all read+write attributed to mike. + +## Pending / Incomplete Tasks +- **RE-ENABLE VWP-QBS firewall** when troubleshooting is done — currently OFF on the QuickBooks/RDWeb + host (defense-in-depth gap). `Set-NetFirewallProfile -All -Enabled True` via RMM (agent f3386f0a). +- **CCroom1New on-site smoke test** — confirm Start menu / Store apps / search behave now that UAC + is active. + +## Reference Information +- #dev-alerts posts: VWP-QBS firewall (msg 1519048232435716218), CCroom1New UAC (msg 1519055241763356683). +- RMM agents: VWP-QBS f3386f0a-b3ee-417e-ace9-995cc1d0662b ; CCroom1New aaa4b694-a464-4961-a8ff-096a5698afee. +- Related same-day work: VWP SMB1/Orders (#32448) — see clients/valleywide/session-logs/2026-06/.