claudetools/.claude/memory/project_edr_rmm_autoisolation_fp.md

name, description, metadata

name	description	metadata
project_edr_rmm_autoisolation_fp	Datto EDR "Exfiltration Over HTTP" rule auto-isolates benign GuruRMM PowerShell fleet-wide; watcher + narrow suppression deployed, policy fix pending Mike	type project

Incident 2026-07-07: vwp-qbs (Valley Wide Plastering QuickBooks server) was auto-isolated by Datto EDR — Mike lost ~4h before realizing EDR did it. No notification fired (EDR alert delivery was never wired to any channel).

Root cause: Datto EDR rule "Exfiltration Over HTTP Protocol" (MITRE T1041, HIGH) carries an automated response of kill-process + isolate-host. It fires on signed PowerShell doing an outbound HTTPS POST — which our GuruRMM automation does routinely. Process tree on the alerts: gururmm-agent.exe -> cmd.exe -> powershell.exe. Confirmed benign both times (vwp-qbs = a vSphere VM-power-on script; tps-tina = a network/connectivity diagnostic).

This is SYSTEMIC, not a one-off. The same rule fired + attempted auto-isolate on tps-tina (The Prairie Schooner) the same day — different GuruRMM script, different client. Per-command-line suppression is whack-a-mole. Note: the isolate-host response only actually cuts a host off where its endpoint policy has isolation enabled (vwp-qbs got isolated; tps-tina fired the same alert but extensionSuccess=None, stayed online).

FLEET-WIDE FIX LIVE (2026-07-07): Datto EDR suppression rule 3365e79a "GuruRMM origin - Exfil-over-HTTP (fleet-wide FP)" — matches Rule Name = Exfiltration Over HTTP Protocol + Grand Parent = gururmm-agent.exe, org/loc empty = ALL clients. Suppresses this one rule for any GuruRMM-launched process fleet-wide (present + future scripts); every other rule still fires on RMM-origin. Created in the CONSOLE (by Howard) after the API create path proved unsafe. The two earlier narrow per-command-line rules (e4dd55bf, 7ea2a577) are now redundant subsets — harmless, left in place.

API FOOTGUN (do NOT create suppressions via raw POST on this tenant): POST /SuppressionRules forces active:true and auto-creates an EMPTY match-everything version → a live "suppress all" window (I hit this with 7c36b89f, deleted within ~2 min). GET/PATCH /SuppressionRules/{id} 400 ("undefined is not valid JSON"); DELETE /{id} works. Create suppressions in the CONSOLE until an atomic API create is verified. Details in .claude/skills/datto-edr/references/api-reference.md.

What's deployed (done):

• Narrow suppression rule (Datto EDR, id e4dd55bf) for VWP's exact script: matches Process Command Line + Grand Parent = gururmm-agent.exe. Does NOT blind other PowerShell. API schema for suppression is now in .claude/skills/datto-edr/references/api-reference.md.
• EDR isolation watcher: .claude/scripts/edr-isolation-watch.sh + register-edr-watcher.ps1, registered as Windows Scheduled Task "ClaudeTools - EDR Isolation Watcher" on Howard-Home (runs every 10 min, posts new auto-isolations to #dev-alerts — see feedback_fire_dev_alerts_for_client_work). Plain script, zero tokens. State file .edr-watch-state.json (gitignored) dedupes; seeded with the vwp-qbs + tps-tina event ids so it stays silent for the already-triaged ones. Retire when GuruRMM EDR webhook (Feature 6) lands.

Pending Mike decisions (do NOT act without him):

1. Auto-isolate vs alert-only on behavioral rules — the real fix (the isolation, not the alert, caused the outage).
2. Whether to allow a small curated "GuruRMM-origin + this-rule OK" exception set — but NOT a blanket gururmm-agent.exe whitelist (RMM = top MSP attack path; blinding EDR there is dangerous).
3. Fix the RMM scripts to stop looking like exfil (drop -EncodedCommand, least-priv accounts).
4. Hygiene: VWP's script had ESXi root creds hardcoded (base64, cert-validation disabled) — already in vault clients/vwp/esxi.sops.yaml; move to a least-priv service account + vault injection.

See reference_datto_edr_detection_behavior (behavioral rules DO fire on signed PowerShell, unlike reputation rules).