--- name: project_edr_rmm_autoisolation_fp description: Datto EDR "Exfiltration Over HTTP" rule auto-isolates benign GuruRMM PowerShell fleet-wide; watcher + narrow suppression deployed, policy fix pending Mike metadata: type: project --- Incident 2026-07-07: **vwp-qbs** (Valley Wide Plastering QuickBooks server) was auto-isolated by Datto EDR — Mike lost ~4h before realizing EDR did it. No notification fired (EDR alert delivery was never wired to any channel). **Root cause:** Datto EDR rule **"Exfiltration Over HTTP Protocol" (MITRE T1041, HIGH)** carries an automated response of **`kill-process` + `isolate-host`**. It fires on *signed* PowerShell doing an outbound HTTPS POST — which our **GuruRMM automation** does routinely. Process tree on the alerts: `gururmm-agent.exe -> cmd.exe -> powershell.exe`. Confirmed benign both times (vwp-qbs = a vSphere VM-power-on script; tps-tina = a network/connectivity diagnostic). **This is SYSTEMIC, not a one-off.** The same rule fired + attempted auto-isolate on **tps-tina** (The Prairie Schooner) the same day — different GuruRMM script, different client. Per-command-line suppression is whack-a-mole. Note: the isolate-host response only actually cuts a host off where its endpoint policy has isolation enabled (vwp-qbs got isolated; tps-tina fired the same alert but `extensionSuccess=None`, stayed online). **FLEET-WIDE FIX LIVE (2026-07-07):** Datto EDR suppression rule **`3365e79a`** "GuruRMM origin - Exfil-over-HTTP (fleet-wide FP)" — matches Rule Name = Exfiltration Over HTTP Protocol + Grand Parent = `gururmm-agent.exe`, org/loc empty = ALL clients. Suppresses this one rule for any GuruRMM-launched process fleet-wide (present + future scripts); every other rule still fires on RMM-origin. Created in the CONSOLE (by Howard) after the API create path proved unsafe. The two earlier narrow per-command-line rules (`e4dd55bf`, `7ea2a577`) are now redundant subsets — harmless, left in place. **API FOOTGUN (do NOT create suppressions via `raw POST` on this tenant):** `POST /SuppressionRules` forces `active:true` and auto-creates an EMPTY match-everything version → a live "suppress all" window (I hit this with `7c36b89f`, deleted within ~2 min). `GET`/`PATCH /SuppressionRules/{id}` 400 ("undefined is not valid JSON"); `DELETE /{id}` works. Create suppressions in the CONSOLE until an atomic API create is verified. Details in `.claude/skills/datto-edr/references/api-reference.md`. **What's deployed (done):** - Narrow suppression rule (Datto EDR, id `e4dd55bf`) for VWP's exact script: matches Process Command Line + Grand Parent = `gururmm-agent.exe`. Does NOT blind other PowerShell. API schema for suppression is now in `.claude/skills/datto-edr/references/api-reference.md`. - **EDR isolation watcher**: `.claude/scripts/edr-isolation-watch.sh` + `register-edr-watcher.ps1`, registered as Windows Scheduled Task **"ClaudeTools - EDR Isolation Watcher"** on Howard-Home (runs every 10 min, posts new auto-isolations to #dev-alerts — see [[feedback_fire_dev_alerts_for_client_work]]). Plain script, zero tokens. State file `.edr-watch-state.json` (gitignored) dedupes; seeded with the vwp-qbs + tps-tina event ids so it stays silent for the already-triaged ones. Retire when GuruRMM EDR webhook (Feature 6) lands. **Pending Mike decisions (do NOT act without him):** 1. Auto-isolate vs **alert-only** on behavioral rules — the real fix (the isolation, not the alert, caused the outage). 2. Whether to allow a small curated "GuruRMM-origin + this-rule OK" exception set — but NOT a blanket `gururmm-agent.exe` whitelist (RMM = top MSP attack path; blinding EDR there is dangerous). 3. **Fix the RMM scripts** to stop looking like exfil (drop `-EncodedCommand`, least-priv accounts). 4. Hygiene: VWP's script had **ESXi root creds hardcoded** (base64, cert-validation disabled) — already in vault `clients/vwp/esxi.sops.yaml`; move to a least-priv service account + vault injection. See [[reference_datto_edr_detection_behavior]] (behavioral rules DO fire on signed PowerShell, unlike reputation rules).