sync: auto-sync from HOWARD-HOME at 2026-07-07 11:15:40
Author: Howard Enos Machine: HOWARD-HOME Timestamp: 2026-07-07 11:15:40
This commit is contained in:
@@ -223,3 +223,4 @@
|
||||
- [Backup targets never guessed](feedback_backup_targets_never_guessed.md) — billed-vs-running mismatches are billing questions for Mike/Winter; never infer/deploy backup targets
|
||||
- [Fire #dev-alerts for client/RMM work](feedback_fire_dev_alerts_for_client_work.md) — Howard+Mike watch #dev-alerts Discord for live visibility into what techs/sessions touch. Fire post-bot-alert.sh ([DEV]/[RMM]/[DEPLOY]/[GURURMM] prefix -> #dev-alerts) at the START of any RMM/Dev or active-client-machine action. No CLAUDE.md rule yet — this memory is the rule.
|
||||
- [M365 app: SharePoint needs CERT](reference_m365_app_sharepoint_rest_vs_graph.md) — SP app-only works via the CERT (get-token.sh <tenant> sharepoint); the secret gives "Unsupported app only token". Graph app-only uses the secret.
|
||||
- [EDR auto-isolates GuruRMM PowerShell](project_edr_rmm_autoisolation_fp.md) — Datto "Exfiltration Over HTTP" rule auto-isolates benign GuruRMM scripts fleet-wide (vwp-qbs 4h outage); watcher + narrow suppression deployed, policy fix pending Mike
|
||||
|
||||
24
.claude/memory/project_edr_rmm_autoisolation_fp.md
Normal file
24
.claude/memory/project_edr_rmm_autoisolation_fp.md
Normal file
@@ -0,0 +1,24 @@
|
||||
---
|
||||
name: project_edr_rmm_autoisolation_fp
|
||||
description: Datto EDR "Exfiltration Over HTTP" rule auto-isolates benign GuruRMM PowerShell fleet-wide; watcher + narrow suppression deployed, policy fix pending Mike
|
||||
metadata:
|
||||
type: project
|
||||
---
|
||||
|
||||
Incident 2026-07-07: **vwp-qbs** (Valley Wide Plastering QuickBooks server) was auto-isolated by Datto EDR — Mike lost ~4h before realizing EDR did it. No notification fired (EDR alert delivery was never wired to any channel).
|
||||
|
||||
**Root cause:** Datto EDR rule **"Exfiltration Over HTTP Protocol" (MITRE T1041, HIGH)** carries an automated response of **`kill-process` + `isolate-host`**. It fires on *signed* PowerShell doing an outbound HTTPS POST — which our **GuruRMM automation** does routinely. Process tree on the alerts: `gururmm-agent.exe -> cmd.exe -> powershell.exe`. Confirmed benign both times (vwp-qbs = a vSphere VM-power-on script; tps-tina = a network/connectivity diagnostic).
|
||||
|
||||
**This is SYSTEMIC, not a one-off.** The same rule fired + attempted auto-isolate on **tps-tina** (The Prairie Schooner) the same day — different GuruRMM script, different client. Per-command-line suppression is whack-a-mole. Note: the isolate-host response only actually cuts a host off where its endpoint policy has isolation enabled (vwp-qbs got isolated; tps-tina fired the same alert but `extensionSuccess=None`, stayed online).
|
||||
|
||||
**What's deployed (done):**
|
||||
- Narrow suppression rule (Datto EDR, id `e4dd55bf`) for VWP's exact script: matches Process Command Line + Grand Parent = `gururmm-agent.exe`. Does NOT blind other PowerShell. API schema for suppression is now in `.claude/skills/datto-edr/references/api-reference.md`.
|
||||
- **EDR isolation watcher**: `.claude/scripts/edr-isolation-watch.sh` + `register-edr-watcher.ps1`, registered as Windows Scheduled Task **"ClaudeTools - EDR Isolation Watcher"** on Howard-Home (runs every 10 min, posts new auto-isolations to #dev-alerts — see [[feedback_fire_dev_alerts_for_client_work]]). Plain script, zero tokens. State file `.edr-watch-state.json` (gitignored) dedupes; seeded with the vwp-qbs + tps-tina event ids so it stays silent for the already-triaged ones. Retire when GuruRMM EDR webhook (Feature 6) lands.
|
||||
|
||||
**Pending Mike decisions (do NOT act without him):**
|
||||
1. Auto-isolate vs **alert-only** on behavioral rules — the real fix (the isolation, not the alert, caused the outage).
|
||||
2. Whether to allow a small curated "GuruRMM-origin + this-rule OK" exception set — but NOT a blanket `gururmm-agent.exe` whitelist (RMM = top MSP attack path; blinding EDR there is dangerous).
|
||||
3. **Fix the RMM scripts** to stop looking like exfil (drop `-EncodedCommand`, least-priv accounts).
|
||||
4. Hygiene: VWP's script had **ESXi root creds hardcoded** (base64, cert-validation disabled) — already in vault `clients/vwp/esxi.sops.yaml`; move to a least-priv service account + vault injection.
|
||||
|
||||
See [[reference_datto_edr_detection_behavior]] (behavioral rules DO fire on signed PowerShell, unlike reputation rules).
|
||||
@@ -14,6 +14,8 @@ Verified live 2026-06-25/26 on RMM-TEST-MACHINE (EDR agent `b98b3ba0-...`, group
|
||||
|
||||
**Datto EDR is reputation-based, NOT structural.** A synthetic "looks suspicious" artifact (Run-key/scheduled task launching hidden `-EncodedCommand` powershell) is collected by the forensic scan but scored BENIGN → no alert (powershell.exe is signed/clean). To get an EDR detection you need a real reputation hit: wire a known-bad file as the **executable of an autostart** (Run-key/scheduled task) so the survey collects + hashes it. EICAR-as-autostart works → high-sev `rule` alert. A loose file on disk is NOT scanned by the EDR forensic survey (it only walks execution/persistence artifacts).
|
||||
|
||||
**BUT behavioral rules DO fire on signed PowerShell** (unlike the reputation scoring above). Rules like **"Exfiltration Over HTTP Protocol" (T1041)** key on *runtime behavior* (outbound HTTP from powershell), not file reputation, so they alert HIGH on clean signed `powershell.exe` — and can carry an automated `isolate-host`/`kill-process` response. This routinely false-positives on GuruRMM automation (`gururmm-agent.exe -> cmd.exe -> powershell`). See [[project_edr_rmm_autoisolation_fp]].
|
||||
|
||||
**AV-suppression gotchas (to isolate EDR on an endpoint):**
|
||||
- Datto AV is **tamper-protected**: `Stop-Service EndpointProtectionService2 -Force` is refused ("cannot be stopped"); can't disable from the endpoint. Supported path = console policy (AV disabled / path-exclusion) — console-only, like policy assignment.
|
||||
- Disabling Datto AV in the policy **uninstalls** the AV component on the box (services `EndpointProtectionService`/`...2` go absent; `HUNTAgent` EDR stays). Platform `dattoAvEnabled` flips to False at the console first; the on-box apply lags a few minutes.
|
||||
|
||||
Reference in New Issue
Block a user