From 95b89c56a8d0da7d2feae9b78f18fc1c1b3769b3 Mon Sep 17 00:00:00 2001 From: Mike Swanson Date: Tue, 9 Jun 2026 10:14:16 -0700 Subject: [PATCH] sync: auto-sync from GURU-5070 at 2026-06-09 10:13:37 Author: Mike Swanson Machine: GURU-5070 Timestamp: 2026-06-09 10:13:37 --- ...26-06-08-safesite-nexsite-pdf-forensics.md | 73 +++++++++++++++++++ ...ke-dataforth-freepbx-safesite-forensics.md | 73 +++++++++++++++++++ 2 files changed, 146 insertions(+) create mode 100644 clients/safesite/session-logs/2026-06-08-safesite-nexsite-pdf-forensics.md create mode 100644 session-logs/2026-06/2026-06-09-mike-dataforth-freepbx-safesite-forensics.md diff --git a/clients/safesite/session-logs/2026-06-08-safesite-nexsite-pdf-forensics.md b/clients/safesite/session-logs/2026-06-08-safesite-nexsite-pdf-forensics.md new file mode 100644 index 0000000..6e6979d --- /dev/null +++ b/clients/safesite/session-logs/2026-06-08-safesite-nexsite-pdf-forensics.md @@ -0,0 +1,73 @@ +# Safe Site — NexSite recalled-PDF forensic investigation + +## User +- **User:** Mike Swanson (mike) +- **Machine:** GURU-5070 +- **Role:** admin + +## Date +- Investigation opened 2026-06-08; forensic sweep + reconstruction 2026-06-09. +- Reconstructed 2026-06-09 from work done on GURU-5070 (live session `eebb22f9-…`, which was never `/save`d — hence this log). + +## Client +- **Safe Site Utility Services LLC** — M365 tenant `safesitellc.com`, tenant ID `71b4e637-c802-4137-a812-ae50dbc839e3`. +- GuruRMM client **Safesite** `fe17552f-736b-42ec-86a2-0e6f107f2397` (sites Bell / Glendale / Unknown). + +## The request (from Jonathan Byrd, j.byrd@nexsitepartners.com) +External sender `m.paris@nexsitepartners.com` sent **"Re: NWWells - SafeSite - Vendor Forms"** with attachment **`SSUS 06122026.PDF`** to 9 Safe Site recipients on 2026-06-08 ~18:54 UTC. The email was recalled. Question: **was the PDF accessed/downloaded on any managed machine?** + +Recipient → machine (via Datto "Last User"): +| Recipient | Machine | GuruRMM enrolled? | +|---|---|---| +| beeanna | 0225-DELL3550 | yes | +| david | 0622-DAVID-HP | yes | +| jon | 0525-ASUSFX707Z | yes | +| justinb | 0525-DELL3550-1 | yes | +| lennyg | DESKTOP-3USU20B | yes | +| suzannep | 1122-SUZANNE-DELL | yes | +| travisf | MSI | yes | +| thomasc | 0724-DELL3550 | yes | +| jeremiahw | **DESKTOP-LOPKB4G** | **NOT enrolled** | + +## Mail-side findings (COMPLETE) +1. **Mailbox content search** (Graph `$search` for "SSUS 06122026" across all 9 mailboxes) → **all `[CLEAN]`**. The recall succeeded — no message carrying the PDF remains in any of the 9 mailboxes. +2. **EXO recall-proof** (`_recall_proof_poller.sh` → `~/Downloads/safesite-recall-proof.json`, pulled 2026-06-09 03:39 UTC, after the Exchange Operator SP's Exchange-Admin role finally propagated): + - `Search-UnifiedAuditLog` FreeText "SSUS 06122026" → **0 rows**. + - Delete/purge ops (HardDelete/SoftDelete/MoveToDeletedItems) by the 9 recipients → **0 rows**. + - `Get-MessageTraceV2` (sender m.paris@nexsitepartners.com) → 74 rows; the message shows **Delivered** to all 9 recipients before recall (distribution list `potholing@` = Expanded). + - **Caveat:** the UAL does not log a PDF opened directly from an Outlook attachment, so "0 audit hits" is **not** proof it was never opened — only that there's no mail/SharePoint audit trace. + +## Endpoint forensic sweep (GuruRMM) — the definitive "on disk / downloaded?" check +Forensic PowerShell (runs as SYSTEM) searches each user profile's Downloads / Desktop / Documents / Outlook `Content.Outlook` cache / Temp / Recent / OneDrive for `*06122026*`, reads the **Zone.Identifier (Mark-of-the-Web)** on any hit, and scans Chrome/Edge **download-history** DBs for the pattern. Emits JSON `{host, hitCount, hits[]}`. + +**First dispatch (2026-06-09 03:44–15:05 UTC):** 7 commands; only **2 completed**, both **CLEAN (hitCount 0)** — **MSI** (travisf) and **0525-DELL3550-1** (justinb). The other 5 **failed: "Command timeout"** — the Safesite agents are WS-disconnected (alive, `last_seen` updates ~every minute, but no persistent socket), so short-timeout commands expire before pickup. + +**Re-dispatch (2026-06-09 ~15:4x UTC, this session):** same script, `timeout_seconds=1800` so it survives the agents' frequent reconnects. Sent to the 6 remaining enrolled targets: +| Machine (recipient) | command_id | +|---|---| +| 0225-Dell3550 (beeanna) | 86340d9b | +| 0622-David-HP (david) | 8d3e6530 | +| 0525-ASUSFX707Z (jon) | 9aa25e67 | +| DESKTOP-3USU20B (lennyg) | 1cf8dfea | +| 1122-Suzanne-Dell (suzannep) | 3322e787 | +| 0724-Dell3550 (thomasc) | 16b2a2b1 | + +Results → `~/Downloads/safesite-forensic-results.txt`. **[STATUS: in progress at time of writing — poller collecting.]** + +## Current status / open items (as of 2026-06-09 ~16:10 UTC) +- **CLEAN — 7 of 9 machines** (hitCount 0; no `06122026` file, no Mark-of-the-Web, no browser-DL trace): + MSI (travisf), 0525-DELL3550-1 (justinb), 0225-DELL3550 (beeanna), 0622-DAVID-HP (david), + 0525-ASUSFX707Z (jon), 1122-SUZANNE-DELL (suzannep), 0724-DELL3550 (thomasc). +- **2 machines deferred — both effectively offline:** + - **DESKTOP-3USU20B (lennyg)** — enrolled but agent last checked in 13:43 UTC; forensic command queued (will run on reconnect, else re-dispatch). + - **DESKTOP-LOPKB4G (jeremiahw)** — NOT enrolled in GuruRMM and offline; sweep via Datto/Intune or after agent install once back online. +- **So far: no evidence the PDF was downloaded or opened on any swept machine.** +- Underlying issue: Safesite GuruRMM agents are WS-disconnected (known fleet issue) — they execute on reconnect but short timeouts fail. Use `timeout_seconds=1800` for this fleet until the WS issue is resolved. + +## Syncro +- Ticket **#32395** (Safesite LLC, contact Jonathan Byrd) — created 2026-06-09; initial customer-facing update emailed (recall verified + 6/9-then-7/9 clean); internal progress note added. + +## Artifacts on GURU-5070 +- `~/Downloads/safesite-recall-proof.json` — EXO recall proof. +- `~/Downloads/safesite-forensic-results.txt` — endpoint sweep results. +- `.claude/scripts/_recall_proof_poller.sh` — the EXO poller (one-shot, completed). diff --git a/session-logs/2026-06/2026-06-09-mike-dataforth-freepbx-safesite-forensics.md b/session-logs/2026-06/2026-06-09-mike-dataforth-freepbx-safesite-forensics.md new file mode 100644 index 0000000..fc6c722 --- /dev/null +++ b/session-logs/2026-06/2026-06-09-mike-dataforth-freepbx-safesite-forensics.md @@ -0,0 +1,73 @@ +# Dataforth FreePBX restore · Birth Biologic admin reset tooling · Safesite PDF forensics + +## User +- **User:** Mike Swanson (mike) +- **Machine:** GURU-5070 +- **Role:** admin + +## Session Summary + +Three threads across two days (2026-06-08 → 06-09). The largest was a **total phone outage at Dataforth** (Sangoma FreePBX 17 / Asterisk 22.5.2 at 192.168.100.2, FirstDigital PJSIP trunk). Outbound was failing with `Could not create dialog to invalid URI 'FirstDigital'` — the trunk contact had gone *Unavailable* because FirstDigital's Sonus SBC stopped answering SIP `OPTIONS` (measured 0/5), and Asterisk 22 refuses to build a call to an Unavailable contact, so no INVITE ever left the box. Fix: set the trunk `qualify_frequency=0` (DB `pjsip` id=1) and re-applied the recurring `PJSip.class.php` line-504 patch (wiped again by the Oct FreePBX update, which had broken `fwconsole reload`). After reload, a test INVITE got `100/183/200` from FirstDigital — outbound restored. Then **inbound** was reported dead too; packet captures proved FD's INVITEs weren't reaching the PBX at all. SSH'd into the Dataforth UDM-Pro (via the D2TESTNAS jump + a root key Mike authorized) and found the root cause: **there was never an inbound SIP port-forward** — inbound had only ever survived on NAT pinholes punched by the qualify-OPTIONS keepalive, which the `qualify=0` fix removed. Added a source-locked (66.7.123.0/24) WAN UDP 5060 + RTP 10000-20000 → 192.168.100.2 DNAT + forward-accept, persisted in `/data/on_boot.d/30-freepbx-sip-forward.sh`. Inbound test calls answered. Ticket #32392 resolved, 1.0 hr emergency remote billed (prepaid ×1.5). + +Second thread: **Birth Biologic** `operations@` M365 password reset. The plain Graph `passwordProfile` PATCH 403'd because operations@ holds SharePoint+Teams Admin roles (Microsoft protects admin accounts — needs Global/Privileged Authentication Administrator). Mike reset it via the portal. To make admin-account resets programmatic going forward, built `scripts/reset-password.sh` in the remediation-tool skill: JIT-assigns the Tenant Admin SP the Privileged Authentication Administrator role (the app holds `RoleManagement.ReadWrite.Directory`), resets, then de-elevates. Committed + the vaulted UDM creds correction synced to the fleet. Also confirmed operations@ already has all-SharePoint access via its SharePoint Admin role. + +Third thread: **Safesite (Safe Site Utility Services)** forensic review of a recalled phishing email (`SSUS 06122026.PDF` from m.paris@nexsitepartners.com to 9 recipients). Mail side: all 9 mailboxes clean (recall succeeded), UAL 0 hits, message-trace confirmed delivery-then-recall. Endpoint side: a GuruRMM forensic sweep (per-user Downloads/Outlook-cache/Recent/OneDrive search for the file + Zone.Identifier MotW + browser DL history). First dispatch mostly timed out (Safesite agents are WS-disconnected); re-dispatched with `timeout_seconds=1800`. **7 of 9 machines swept CLEAN; no evidence the PDF was downloaded/opened anywhere.** 2 machines (lennyg DESKTOP-3USU20B, jeremiahw DESKTOP-LOPKB4G) stayed offline → deferred. Opened Syncro #32395 (contact Jonathan Byrd), emailed an initial update, billed 1.5 hr remote (prepaid → 0), and set up a coord auto-followup (todo + fleet broadcast) so any free session completes the last 2 machines and closes out with Jonathan. + +## Key Decisions + +- **Dataforth: do NOT revert `qualify=0`.** Reverting would re-break outbound (FD ignores OPTIONS). The inbound problem was the missing UDM port-forward, not our change — proven by timeline (inbound worked 2.5 h after the AM change, then died with the pinhole). +- **Source-locked the UDM SIP forward to 66.7.123.0/24** (FD's subnet) to keep internet SIP scanners off the PBX. +- **Persisted UDM rules via `/data/on_boot.d/`** (matching the existing Neptune SNAT pattern) rather than the UI, for reboot survival; recommended Mike add a UI rule afterhours for provision-safe persistence. +- **Built JIT-elevation password reset** rather than granting the Tenant Admin app a standing Privileged Auth Admin role — minimizes blast radius; the app could already self-elevate via RoleManagement.ReadWrite.Directory, so no new exposure. +- **Safesite: long timeouts (1800s) for this fleet** — the agents are alive but WS-disconnected (recent last_seen, is_connected=false), so commands must survive until the next reconnect. +- **Safesite followup via coord todo + fleet broadcast, not a cloud routine** — the work needs internal-network access (GuruRMM 172.16.3.30, vault) that a cloud-scheduled agent can't reach, so a fleet workstation session must run it. + +## Problems Encountered + +- **paramiko quoting through nested `sudo bash -c "..."`** truncated Asterisk CLI commands (`asterisk -rx core` → "No such command"). Fixed by uploading scripts via SFTP and running `sudo bash `, or single-quoting inner commands. +- **UDM SSH auth failures** — vaulted password `Paper123!@#-unifi` was stale; the device SSH wanted `azcomputerguru`/`r3tr0gradE99#` and 2FA. Resolved by tunneling through D2TESTNAS and having Mike add a root SSH key. His first add-key command lost the `>>` redirect (echoed the key instead of writing it); re-issued with `tee -a`. +- **GuruRMM forensic timeouts** — first sweep failed with "Command timeout" on WS-disconnected agents; fixed with `timeout_seconds=1800` + re-dispatch. +- **Syncro ticket POST returned empty once** (both #32392 and the recall work) — per skill rule, GET-verified no duplicate before retrying. +- **Coord todo via raw API returned null** — switched to the `coord` skill's `coord.py` which created it cleanly. + +## Configuration Changes + +- `.claude/skills/remediation-tool/scripts/reset-password.sh` — NEW (JIT admin password reset). Mirrored to repo `.claude/skills/...`. +- `.claude/commands/remediation-tool.md` — documented the JIT password-reset pattern + admin-target caveat. +- Dataforth PBX `192.168.100.2`: `pjsip` DB id=1 `qualify_frequency` 60→0; `PJSip.class.php` line 504 re-patched (backup `.bak.20260608083954`). +- Dataforth UDM `192.168.0.254`: `/data/on_boot.d/30-freepbx-sip-forward.sh` — NEW (SIP/RTP DNAT + forward-accept); root key added to `/root/.ssh/authorized_keys`. +- Vault `clients/dataforth/udm.sops.yaml` — corrected creds (azcomputerguru/r3tr0gradE99#), added console_ssh_user + notes (committed 880761d). +- `clients/safesite/session-logs/2026-06-08-safesite-nexsite-pdf-forensics.md` — NEW (Safesite forensic log). + +## Credentials & Secrets + +- **Dataforth UDM** `192.168.0.254`: SSH `azcomputerguru` / `r3tr0gradE99#`; console user `root` (ACG via root SSH key over D2TESTNAS jump); web `azcomputerguru` / `r3tr0gradE99#`. 2FA push. Vault: `clients/dataforth/udm.sops.yaml`. +- **Dataforth PBX** `192.168.100.2`: `sangoma` / `Gptf*77ttb!@#!@#`. Vault: `clients/dataforth/pbx.sops.yaml`. +- **D2TESTNAS** `192.168.0.9`: `root` / `Paper123!@#` (jump host). Vault: `clients/dataforth/d2testnas.sops.yaml`. +- **Birth Biologic** tenant `birthbiologic.com` (19a568e8-9e88-413b-9341-cbc224b39145 via openid; tenant-admin app 709e6eed). operations@ id `d9a0a1af-d216-4cc0-929a-3170573f7dd5`, new password set by Mike in portal (C@lmOp$26). + +## Infrastructure & Servers + +- **Dataforth FreePBX:** 192.168.100.2 (Sangoma FreePBX 17 / Asterisk 22.5.2). Trunk FirstDigital, SBC 66.7.123.215:5060 (Sonus), match 66.7.123.0/24, IP-auth (no registration). Public IP 67.206.163.122 (eth8 on UDM). FD ignores OPTIONS but answers INVITEs. +- **Dataforth UDM-Pro:** 192.168.0.254 / 192.168.0.1, UniFi OS 5.1.15. WAN eth8 67.206.163.122/29 + 67.206.163.124/32 (Neptune). Port-forwards in mongo `ace.portforward` (Exchange→172.16.3.11; new SIP via on_boot.d). +- **Safesite:** M365 `safesitellc.com` (71b4e637-c802-4137-a812-ae50dbc839e3). GuruRMM client `fe17552f-736b-42ec-86a2-0e6f107f2397` (sites Bell/Glendale/Unknown), ~28 agents, all WS-disconnected. GuruRMM API `http://172.16.3.30:3001`. + +## Commands & Outputs + +- UDM jump+key SSH: paramiko Transport over D2TESTNAS `direct-tcpip` channel to 192.168.0.254:22, `auth_publickey('root', ~/.ssh/id_ed25519)`. +- UDM SIP forward: `iptables -t nat -A UBIOS_PREROUTING_USER_HOOK -d 67.206.163.122/32 -s 66.7.123.0/24 -p udp --dport 5060 -j DNAT --to-destination 192.168.100.2:5060` (+ RTP 10000:20000, + `UBIOS_FORWARD_IN_USER` ACCEPTs). +- Safesite forensic sweep: PowerShell searching `C:\Users\*\{Downloads,Desktop,Documents,AppData\Local\Microsoft\Windows\INetCache\Content.Outlook,Temp,Recent,OneDrive*}` for `*06122026*`, reads `-Stream Zone.Identifier`, scans browser `History` DBs; emits `{host,hitCount,hits}`. Result: 7/9 hitCount=0. + +## Pending / Incomplete Tasks + +- **Safesite #32395:** sweep DESKTOP-3USU20B (lennyg, enrolled, cmd 1cf8dfea queued) and DESKTOP-LOPKB4G (jeremiahw, NOT enrolled) when online; then email Jonathan final findings + mark coord todo `5766a59f` done. (Auto-followup coordinated: todo 5766a59f + broadcast faaec0ce.) +- **Dataforth #32392:** Mike to add the UI port-forward afterhours (on_boot.d covers reboots meanwhile). Re-apply `PJSip.class.php` patch after any future `fwconsole ma updateall`. +- **Birth Biologic:** validate `reset-password.sh` end-to-end on next real admin reset. + +## Reference Information + +- **Syncro tickets:** #32392 (Dataforth FreePBX, Resolved, 1.0h emergency remote, prepaid 34.5→33.0). #32395 (Safesite forensics, In Progress, 1.5h remote, prepaid 1.5→0; contact Jonathan Byrd 3458770; invoice 1650620815). +- **Commits:** reset-password.sh + doc (31e5cbd3); vault UDM creds (880761d). +- **Coord:** todo `5766a59f-0ddf-43d8-b16b-1c60024a3c04`; broadcast `faaec0ce-ed5f-4e0f-8693-904a3d000c38`. +- **Artifacts on GURU-5070:** `~/Downloads/safesite-recall-proof.json`, `~/Downloads/safesite-forensic-results.txt`. +- **Forensic cmd ids (Safesite re-dispatch):** 86340d9b, 8d3e6530, 9aa25e67, 1cf8dfea, 3322e787, 16b2a2b1.