# Datto EDR Skill Build + Full Lifecycle Test on RMM-TEST-MACHINE

## User
- **User:** Howard Enos (howard)
- **Machine:** Howard-Home
- **Role:** tech

## Session Summary

Built a new `datto-edr` skill from scratch and ran a full create-group -> install -> scan
lifecycle test against the live ACG Datto EDR tenant (`azcomp4587.infocyte.com`). Started by
answering skill-inventory questions (no EDR/Autotask/Kaseya skills existed) and verifying that
**Syncro's own RMM** (policies, asset/group moves) is GUI-only via API — saved as memory
`reference_syncro_rmm_api_gui_only`. Then scoped Datto EDR control: research established Datto
EDR == rebranded **Infocyte HUNT**, a per-tenant LoopBack REST API, and that **no Datto RMM
skill is needed** (EDR API is standalone).

Howard provided the EDR API token; it was vaulted at `msp-tools/datto-edr.sops.yaml`,
live-verified (215 agents, 96 boxes, 13 client orgs), and the full skill was authored and
committed (`.claude/skills/datto-edr/`, commit `bd1e84d` on main). The skill drives the whole
MSP fleet from one token: orgs/sites/agents/detections/sweep (all live-verified) plus gated
scan/isolate/deploy.

The lifecycle test on **RMM-TEST-MACHINE** (ACG internal Howard-VM) created an EDR target group,
minted a registration key, pushed the agent install via `/rmm`, and confirmed the agent
registered into the group (active, default EDR real-time policy applied). The scan step exposed
that the documented Infocyte scan endpoints are **dead** on this tenant; a research agent reading
the live console's own JS bundle found the **definitive** working scan call. Session paused here
to save + clear context before applying the code fix and running a detection->reporting test.

## Key Decisions

- **No Datto RMM skill** — Datto EDR has its own standalone API (Infocyte HUNT); RMM is a separate
  product/API (already vaulted at `msp-tools/datto-rmm.sops.yaml`, unrelated).
- **Skill modeled on `bitdefender`** — same structure (SKILL.md + `<cli>.py` + `<cli>_client.py` +
  selftest + references), reads free, mutations `--confirm`, vault-keyed, live-verified. This skill
  is the prototype for GuruRMM security-connector #2 (RMM_THOUGHTS Feature 6).
- **Policy assignment is console-only** — verified exhaustively (relation endpoints 404, policies
  are tenant-global typed `av`/`edr` templates, no policyId on org/target/agent, module ships no
  policy cmdlets). Default `av`+`edr` policies auto-apply; chose "proceed with defaults" for the test.
- **Scan one agent via `where` filter** — the scan param is a LoopBack `where`, NOT `ids`; absent
  `where` = tenant-wide. Will rewrite the skill's scan command to this.
- **Cancelled the accidental tenant-wide scan** immediately (was at 0%, contained).

## Problems Encountered

- **Install passed empty `--url`** — `Install-EDR -InstanceName azcomp4587` failed because the
  install script's loose `.com` regex matches "zcom" inside "azcomp4587", so it thought the cname
  was already a full URL and built an empty `$hunturl`. Fix: pass the **full URL**
  `-URL "https://azcomp4587.infocyte.com"`. Re-dispatch succeeded (exit 0).
- **`agentKeys` POST 500 on `{targetId}`** — the key `id` is **caller-supplied** (a 10-char
  string), not auto-generated. `POST /agentKeys {"id":"tstrmm7053","targetId":"<tg>"}` works.
- **All Infocyte scan routes 404** (`targets/{id}/scan`, `targets/scan`, `scans`) — superseded.
- **`POST /Agents/scan` with `{ids:[...]}` or empty body = tenant-wide scan** ("Scanning 156
  hosts"). Root cause: endpoint takes a `where` filter; `ids` is silently ignored, no `where` =
  scan all. Logged to errorlog as friction.
- **`is_connected` is null fleet-wide in GuruRMM** — first install dispatch went to the stale
  (offline) RMM-TEST-MACHINE agent row and queued `pending`. Resolve by **most-recent `last_seen`**,
  not `is_connected`. Cancelled + redispatched to the live agent.
- **`eval "$(rmm-auth.sh)" | tail` lost env vars** — piping puts eval in a subshell; `$TOKEN`/`$RMM`
  never set in the parent. Run `eval` without a pipe.

## Configuration Changes

- **Created skill** `.claude/skills/datto-edr/` — `SKILL.md`, `scripts/edr.py`, `scripts/edr_client.py`,
  `scripts/selftest.py`, `references/api-reference.md`, `.gitignore`. Committed `bd1e84d` (main).
  **NOTE: the committed scan code still uses the DEAD `targets/{id}/scan` endpoint — must be fixed
  next session (see Pending).**
- **Memory** `.claude/memory/reference_syncro_rmm_api_gui_only.md` + MEMORY.md index line. Committed.
- **RMM_THOUGHTS Feature 6** (`projects/msp-tools/guru-rmm/docs/RMM_THOUGHTS.md`) — appended Datto EDR
  connector API research. Committed in guru-rmm submodule `3b3f069`, pointer bumped in main `bd1e84d`.
- **errorlog.md** — one `--friction` entry (scan endpoints dead + tenant-wide footgun).

## Credentials & Secrets

- **Datto EDR API token** — vaulted `msp-tools/datto-edr.sops.yaml` field `credentials.api_token`.
  Value: `FpRvE6IENdctE5Mrf8CS8FpyawbY6MTQXwc9Vw9GmdqQq02TfGlvpfv5skzKhjO7`. Pushed to vault repo.
  **Auth = raw token in `Authorization` header (NO `Bearer`).** Created 2026-06-25, **expires
  ~2027-06-25** (1yr). Generated in console: username menu -> Admin -> Users & Tokens -> API Tokens.
- **EDR group registration key** `tstrmm7053` (minted this session, tied to test group `c3ba0672`).
  Not vaulted (disposable test key).

## Infrastructure & Servers

- **Datto EDR tenant:** `https://azcomp4587.infocyte.com` (API base `/api`). LoopBack REST.
  Explorer/swagger (`/explorer/*`) hangs/times out — unusable; `/api/*` is instant.
- **Data model:** Organization (client) -> Location (site, carries `organizationId`) -> Agent
  (carries `locationId`). `Targets` = scan groups (often alias a Location id). `deviceGroups` =
  global categories ("Servers"/"Workstations"). Policies = tenant-global typed `av`/`edr`, `isDefault`.
- **Test artifacts LIVE on the tenant (pending cleanup decision):**
  - EDR target group `[TEST] RMM-TEST-MACHINE` — targetId `c3ba0672-e6bb-4784-9a37-2f434fc6f08c`, org
    ACG `ac78844a-2d44-4c10-acc8-c9bcb6106346`.
  - Reg key `tstrmm7053`.
  - EDR agent `rmm-test-machine` — id `b98b3ba0-5f82-466f-911a-5a6b24cdbae7`, active, locationId
    `c3ba0672`, dattoAvEnabled=false, version 3.17.1.5409, Win11 22H2. deviceId/deviceShortId null.
- **RMM-TEST-MACHINE in GuruRMM** (`http://172.16.3.30:3001`): ACG / Howard-VM / Windows. **Live
  agent id `99d6d692-99e0-4359-9f9c-f43be89f49e5`** (use most-recent last_seen; stale row is
  `7d3456f5...`).

## Commands & Outputs

- **VERIFIED single-agent scan (apply to skill next session):**
  ```
  POST https://azcomp4587.infocyte.com/api/Agents/scan
  Authorization: <raw token>
  {"where":{"id":{"inq":["<agentId>"]}}, "options":{}, "taskName":"Scan - EDR"}
  ```
  Source: live console JS bundle `index.DhsZtGr7.js` (`post("agents/scan",{where,options,taskName})`).
  Absent `where` => scans ALL active agents (the footgun). Also `POST organizations/scan`,
  `locations/scan`, `locations/{id}/scan` take `{where, options}`. `scanType` is client-side only.
  **AV scans are policy-driven, not callable.**
- **Cancel a scan task:** `POST /userTasks/{id}/cancel` -> 204 (or `PATCH /userTasks/{id}` `{status:"Cancelled"}`).
- **Create group:** `POST /Targets {"name":"...","organizationId":"..."}` -> `{id,...}`.
- **Mint key:** `POST /agentKeys {"id":"<10char>","targetId":"<tg>"}` (id is caller-supplied).
- **Install one-liner (push via /rmm, FULL url):**
  ```
  [System.Net.ServicePointManager]::SecurityProtocol=[Enum]::ToObject([System.Net.SecurityProtocolType],3072); (new-object Net.WebClient).DownloadString("https://raw.githubusercontent.com/Infocyte/PowershellTools/master/AgentDeployment/install_huntagent.ps1") | iex; Install-EDR -URL "https://azcomp4587.infocyte.com" -RegKey tstrmm7053
  ```
  Result: `Installed RTS agent to C:\Program Files\infocyte\agent\agent.exe`, exit 0.
- **Skill CLI (working):** `bash .claude/scripts/py.sh .claude/skills/datto-edr/scripts/edr.py status|orgs|sites --org|agents --org|detections --org --days N|sweep|deploy-cmd|extensions`.

## Pending / Incomplete Tasks

**RESUME PLAN (next session, after context clear):**

1. **Fix the skill scan code** (currently committed with the DEAD `targets/{id}/scan`):
   - `edr_client.py`: replace `scan_target_group`/`scan_single_target` with `scan_agents(agent_ids)`
     -> `POST Agents/scan {"where":{"id":{"inq":[ids]}}, "options":{}, "taskName":"Scan - EDR"}`.
     Add a hard guard: refuse to POST without a non-empty `where`/agent list (prevents tenant-wide).
     Add `cancel_task(id)` (`POST userTasks/{id}/cancel`). Optionally add `create_group`, `mint_key`.
   - `edr.py`: change `scan` to `--agent <id>` (and/or `--agents`), keep `--confirm`; add `cancel`,
     and first-class `create-group` + `mint-key` + `deploy` subcommands. Update `_t_*` as needed.
   - Update `references/api-reference.md` + `SKILL.md`: verified scan endpoint, tenant-wide footgun,
     install full-URL gotcha, agentKeys caller-supplied id, policy console-only. Commit + push.
2. **Detection -> reporting test:** push a **known-detectable file** to RMM-TEST-MACHINE (RMM agent
   `99d6d692`), then scan ONLY that agent (`where id inq [b98b3ba0]`) and verify a detection appears
   in `detections`/Alerts -> proves reporting. **CAVEAT:** the agent is **EDR-only (no Datto AV)**, so
   an EICAR/AV test file may NOT trigger — Datto EDR is behavioral/forensic (reputation/artifact
   scoring). Pick an EDR-detectable artifact (known-bad-hash test binary, or a tool flagged by
   reputation), or assign/enable Datto AV first. Decide the artifact at the start of next session.
3. **Cleanup decision** on the test artifacts (group `c3ba0672`, key `tstrmm7053`, installed agent
   `b98b3ba0` on RMM-TEST-MACHINE) — keep as a live test endpoint, or tear down (agent.exe
   --uninstall via /rmm + delete group/key). Howard leaned toward keeping a test endpoint.

## Reference Information

- Skill: `.claude/skills/datto-edr/` (commit `bd1e84d`, main). Vault: `msp-tools/datto-edr.sops.yaml`.
- Tenant: `azcomp4587.infocyte.com`. Org map e.g. Cascades `2d5ea96e...`, Dataforth `4a2664bf...`,
  ACG `ac78844a-2d44-4c10-acc8-c9bcb6106346`.
- KaseyaDEDR/Infocyte GitHub `PowershellTools` (Apache-2.0) — install script + old API patterns;
  scan routes there are DEAD. Datto EDR help: edr.datto.com/help. RMM_THOUGHTS Feature 6 for the
  GuruRMM "EDR add-on" (webhooks Admin->Webhooks; needs Mike's go to build).
- Research subagents (resumable): scan-endpoint finder `af59ee58a2ba28282`; EDR API research `ab14b157f92f91d49`.

---

## Update: 20:23 PT — code-review fixes applied + AV/EDR detection->reporting proven

### Session Summary

Resumed after context clear. Two threads completed: (1) applied the remaining workflow
code-review findings to the `datto-edr` skill and committed; (2) ran a full
detection->reporting test on RMM-TEST-MACHINE that ended up proving BOTH the Datto AV and
the Datto EDR engines detect and report correctly through the skill.

Applied the 6 still-pending review fixes (the first 4 were done pre-clear): deploy-cmd now
requires explicit `--regkey` or `--group` (no more auto-picking an arbitrary cross-client
key); `raw` blocks a POST to any `*/scan` endpoint with no non-empty `where` (same
tenant-wide footgun the `scan` cmd guards); `main()` got a catch-all that surfaces + logs
unexpected exceptions (and a clean KeyboardInterrupt=130); `isolate` extension match is now
forgiving (exact->substring, excludes the paired "Restore", errors on ambiguity);
`detections --site` renamed `--target-group` (Alert.targetGroupId is a scan-target id, not a
Location id); `_t_status` relabel "Target groups (sites)"->"Scan target groups". SKILL.md +
docstrings updated. py_compile clean, selftest green (216 agents), guards fire on
no-key/empty-where/no-agent, deploy-cmd --group picks the group's own key. Committed `79bda6f`.

Detection test: discovered the test EDR agent is NOT EDR-only (the prior resume note was
wrong) -- `dattoAvEnabled=true`, Datto AV engine ready, Defender disabled. Pushed EICAR to
the box via /rmm; Datto AV auto-quarantined it within ~1-2 min and raised a high-sev alert
(`sourceType: av`, `Eicar-Test-Signature`, responseData quarantine-file), which surfaced
correctly through `edr.py detection`/`detections`. That proved AV detect->respond->report
end-to-end. The skill's `scan`/`task`/`cancel` were all exercised.

To isolate the EDR engine specifically, first tried a behavioral artifact (Run-key +
scheduled task launching hidden encoded PowerShell, benign payload, no AV disable). The EDR
forensic scan collected it but scored it BENIGN -> no alert: Datto EDR alerts on
reputation/threat-intel, not structure. Per Howard's "if that doesn't work then disable AV",
moved to AV suppression. The AV is tamper-protected (can't stop EndpointProtectionService2
from the endpoint); Howard disabled Datto AV in the attached console policy (uninstalls the
AV component) and then disabled Defender (which had auto-reactivated). With both AVs out and
EICAR wired as the executable of a Run-key + scheduled task, the Datto EDR forensic scan
detected it: high-sev alert `9e9b6e7d`, `eicar.exe`, `sourceType: rule`, "Generic Malware
(Reputation - High Severity)". Both AV and EDR detections render correctly via the skill.
Cleaned up all on-box artifacts and restored Defender RTP.

### Key Decisions
- Kept the test fixtures (EDR agent `b98b3ba0`, group `c3ba0672`, key `tstrmm7053`) as a
  standing test endpoint, per Howard's earlier lean.
- Used a behavioral artifact (no AV disable) as the first EDR-isolation attempt; only
  escalated to AV suppression when it produced no EDR alert.
- Refused to brute-force tamper-protected AV from the endpoint (kill protected process /
  disable minifilter / reboot) -- too likely to break the managed AV. Used the supported
  console-policy path (Howard-driven) instead.
- Build EICAR from char codes (not a literal) once Defender was in play -- Defender AMSI
  blocks any PowerShell script containing the literal EICAR string.

### Problems Encountered
- `/tmp` read-back mismatch again (Git-Bash vs Python) -- switched to repo-relative scratch.
  Logged friction.
- `edr.py agent <8charid>` returned HTTP 500 -- the API needs the full UUID; resolved by id
  prefix match client-side over the 216-agent list.
- Disabling Datto AV in policy made Windows Defender auto-reactivate (Security Center) ->
  Defender then quarantined EICAR and AMSI-blocked the dropper script. Fixed by Howard
  disabling Defender + building EICAR from char codes.
- Service-state sync lag: platform `dattoAvEnabled` flipped False at the console first; the
  on-box AV uninstall completed a few minutes later.

### Configuration Changes
- `.claude/skills/datto-edr/scripts/edr.py`, `edr_client.py`, `SKILL.md` -- review fixes (commit `79bda6f`).
- `.claude/memory/reference_datto_edr_detection_behavior.md` (new) + `MEMORY.md` index line.

### Verified Facts (this update)
- Alert `sourceType`: `av` = Datto AV signature; `rule` = Datto EDR reputation detection. Both
  land in `Alerts`, both surface via `edr.py detections`.
- Datto EDR forensic scan surveys execution/persistence artifacts (autostarts, processes,
  modules, etc.) + reputation-checks their hashes; it does NOT scan loose files on disk. Wire a
  known-bad file as an autostart's executable to trip it.
- Canonical EICAR SHA256 (as written): `275A021BBFB6489E54D471899F7DB9D1663FC695EC2FE2A2C4538AABF651FD0F`.
- AV is tamper-protected: `Stop-Service EndpointProtectionService2 -Force` is refused; supported
  disable is console policy only (like Syncro RMM policy).

### Test artifacts / alerts (left in place)
- EDR alert `9e9b6e7d` (eicar.exe, rule) + AV alert `269ab552` (eicar_edrtest.com, av) remain in
  the console as test records -- Howard to archive if desired.
- On-box: all removed (EICAR file, Run-key `EDRTestEicar`, scheduled task `EDRTestEicar`).

### Pending / Action
- **Howard: re-enable Datto AV in the console policy** for RMM-TEST-MACHINE (currently on
  Defender RTP as interim protection; AV component was uninstalled by the policy disable).

### Reference Information
- Test EDR agent `b98b3ba0-5f82-466f-911a-5a6b24cdbae7` (hostname rmm-test-machine), RMM agent
  `99d6d692-99e0-4359-9f9c-f43be89f49e5`, group `[TEST] RMM-TEST-MACHINE` `c3ba0672-...`, org ACG
  `ac78844a-2d44-4c10-acc8-c9bcb6106346`. AV install path `C:\Program Files\infocyte\agent\dattoav\`.
- Commit `79bda6f` datto-edr code-review fixes.