wiki: compile gps-rmm-audit (full) — Phase 4 backup verification, master-status list, 20 client wikis, onboardings, onsite installer kit

This commit is contained in:
2026-07-07 10:15:38 -07:00
parent 33b6140236
commit ff6546bee8
2 changed files with 399 additions and 153 deletions

View File

@@ -72,7 +72,7 @@ Run `/wiki-lint` to check for stale entries and broken backlinks.
|---|---|---| |---|---|---|
| [GuruRMM](projects/gururmm.md) | RMM platform, Rust/Axum server + React dashboard + cross-platform agent; **agent v0.6.78 fleet-wide stable / 0.6.79 beta / server v0.3.103 / dashboard = Fleet-Policy Console**; ~360 enrolled; migrations through 068_visible_agents_folder_id. Merge to main = build+deploy; channel default FAIL-CLOSED beta (BUG-023), explicit promote to stable. Recent (2026-07-06): **Fleet-Policy Management Console** shipped to PROD (policy-ui — folder tree table / place-devices / effective-policy viewer / version-history / backfill; migration 068, folder_id in API); **WS-looper comms-durability** hardening (bounded WS write-timeout + heartbeat-count blackhole-vs-stall diagnostic + AIMD adaptive keepalive; agent 0.6.79 beta / server 0.3.102). Prior (2026-07-04/05): four bug-fix batches v0.3.95->v0.3.99 + policy-core (migration 067). **BUG-024 OPEN/CRITICAL** — Linux glibc incident, Linux binaries QUARANTINED (re-quarantined every build) pending glibc-floor fix; 4 chronic one-way-WS agents now have a server-visible blackhole diagnostic. Active development | 2026-07-06 | | [GuruRMM](projects/gururmm.md) | RMM platform, Rust/Axum server + React dashboard + cross-platform agent; **agent v0.6.78 fleet-wide stable / 0.6.79 beta / server v0.3.103 / dashboard = Fleet-Policy Console**; ~360 enrolled; migrations through 068_visible_agents_folder_id. Merge to main = build+deploy; channel default FAIL-CLOSED beta (BUG-023), explicit promote to stable. Recent (2026-07-06): **Fleet-Policy Management Console** shipped to PROD (policy-ui — folder tree table / place-devices / effective-policy viewer / version-history / backfill; migration 068, folder_id in API); **WS-looper comms-durability** hardening (bounded WS write-timeout + heartbeat-count blackhole-vs-stall diagnostic + AIMD adaptive keepalive; agent 0.6.79 beta / server 0.3.102). Prior (2026-07-04/05): four bug-fix batches v0.3.95->v0.3.99 + policy-core (migration 067). **BUG-024 OPEN/CRITICAL** — Linux glibc incident, Linux binaries QUARANTINED (re-quarantined every build) pending glibc-floor fix; 4 chronic one-way-WS agents now have a server-visible blackhole diagnostic. Active development | 2026-07-06 |
| [GuruConnect](projects/guruconnect.md) | ACG's proprietary Rust remote-access/remote-support tool (ScreenConnect-class) — Windows agent + web dashboard; **v0.3.0 production** at connect.azcomputerguru.com; versioned integration contract with GuruRMM; co-located on the physical .30 box (shares the PG cluster); next: SPEC-018 session broker / capture worker as SYSTEM | 2026-06-12 | | [GuruConnect](projects/guruconnect.md) | ACG's proprietary Rust remote-access/remote-support tool (ScreenConnect-class) — Windows agent + web dashboard; **v0.3.0 production** at connect.azcomputerguru.com; versioned integration contract with GuruRMM; co-located on the physical .30 box (shares the PG cluster); next: SPEC-018 session broker / capture worker as SYSTEM | 2026-06-12 |
| [GPS -> GuruRMM Coverage Audit](projects/gps-rmm-audit.md) | Verify every GPS-billed client is fully enrolled in GuruRMM with services real (backup/AV/email); enrollment 111/189 (baseline 46); staging auto-enroll pipeline + 30-min harvest loop; found+fixed GuruRMM cross-site dup bug (v6.77); Phase 4 AV matrix (8 clients/20 devices NO-AV); billing findings HELD for Winter/Mike; AV migration BD->EDR (exception: Glaztech only) | 2026-07-04 | | [GPS -> GuruRMM Coverage Audit](projects/gps-rmm-audit.md) | Verify every GPS-billed client is fully enrolled in GuruRMM with billed services real (backup/AV/email). Single current list = `gps-master-status.md`. Phase 4 backup verification: authoritative source = MSP360 API (not B2 buckets); only 16/40 bill backup (7 OK / 2 review / 7 NOT backing up, endpoint-verified). Built msp360/seafile/owncloud/datto-workplace skills; 20 client wikis (56->76); onsite installer kit (Desktop/CLIENTS). Onboarded Janet Altschuler/Marty Ryan/BST + 3 shells. Datto Workplace key pending Mike; billing findings HELD for Winter/Mike; AutoEnroll task disabled | 2026-07-07 |
| [Dataforth DOS — Test Datasheet Pipeline](projects/dataforth-dos.md) | DOS update system + TestDataDB pipeline (Node.js, PostgreSQL, Hoffman API); 469K records, 458.5K live on website; 2025 crypto attack recovery; security incident 2026-03-27; SCMVAS/SCMHVAS extension; email notifications via Graph API | 2026-05-24 | | [Dataforth DOS — Test Datasheet Pipeline](projects/dataforth-dos.md) | DOS update system + TestDataDB pipeline (Node.js, PostgreSQL, Hoffman API); 469K records, 458.5K live on website; 2025 crypto attack recovery; security incident 2026-03-27; SCMVAS/SCMHVAS extension; email notifications via Graph API | 2026-05-24 |
| [ClaudeTools Discord Bot](projects/discord-bot.md) | Claude Agent SDK bot in Discord; one persistent session per thread; Phase 1.5 complete (native tools, no hand-written tools); Phases 2-4 (API integration, remediation, UX) pending; runs as NSSM service on BEAST | 2026-05-24 | | [ClaudeTools Discord Bot](projects/discord-bot.md) | Claude Agent SDK bot in Discord; one persistent session per thread; Phase 1.5 complete (native tools, no hand-written tools); Phases 2-4 (API integration, remediation, UX) pending; runs as NSSM service on BEAST | 2026-05-24 |
| [The Computer Guru Show](projects/radio-show.md) | Radio show archive processing pipeline (Whisper + pyannote + SQLite FTS5) + post-show content workflow; 572 episodes indexed; FastAPI UI redesigned; Jupiter audio-file gap open | 2026-05-24 | | [The Computer Guru Show](projects/radio-show.md) | Radio show archive processing pipeline (Whisper + pyannote + SQLite FTS5) + post-show content workflow; 572 episodes indexed; FastAPI UI redesigned; Jupiter audio-file gap open | 2026-05-24 |

View File

@@ -2,17 +2,28 @@
type: project type: project
name: gps-rmm-audit name: gps-rmm-audit
display_name: "GPS -> GuruRMM Coverage Audit" display_name: "GPS -> GuruRMM Coverage Audit"
last_compiled: 2026-07-04 last_compiled: 2026-07-07
compiled_by: HOWARD-HOME/claude-main compiled_by: HOWARD-HOME/claude-main
sources: sources:
- wiki/_templates/project.md - wiki/_templates/project.md
- projects/gps-rmm-audit/gps-master-status.md
- projects/gps-rmm-audit/backup-map.md
- projects/gps-rmm-audit/tracker.md - projects/gps-rmm-audit/tracker.md
- projects/gps-rmm-audit/session-logs/2026-07/2026-07-03-howard-gps-rmm-coverage-audit.md
- projects/gps-rmm-audit/dedup-plan.md - projects/gps-rmm-audit/dedup-plan.md
- projects/gps-rmm-audit/session-logs/2026-07/2026-07-03-howard-gps-rmm-coverage-audit.md
- projects/gps-rmm-audit/session-logs/2026-07/2026-07-05-howard-backup-verification-skills.md
- projects/gps-rmm-audit/session-logs/2026-07/2026-07-05-howard-edr-mass-deploy-imc-printserver.md
- projects/gps-rmm-audit/session-logs/2026-07/2026-07-05-howard-cpqb-fixed-imcprintserver-tls-blocked.md
- projects/gps-rmm-audit/session-logs/2026-07/2026-07-05-howard-gps-backup-map-msp360.md
- projects/gps-rmm-audit/session-logs/2026-07/2026-07-07-howard-gps-onboard-endpoint-backup-verify.md
- projects/gps-rmm-audit/session-logs/2026-07/2026-07-07-howard-gps-master-status-wikis-onsite-kit.md
- .claude/memory/project_av_migration_bitdefender_to_edr.md - .claude/memory/project_av_migration_bitdefender_to_edr.md
- .claude/memory/reference_rmm_deploy_via_screenconnect.md - .claude/memory/reference_rmm_deploy_via_screenconnect.md
- .claude/memory/reference_screenconnect_custom_property_slots.md - .claude/memory/reference_screenconnect_custom_property_slots.md
- .claude/memory/feedback_screenconnect_cleanup_wiki_source.md - .claude/memory/feedback_screenconnect_cleanup_wiki_source.md
- .claude/memory/reference_msp360_backup_monitoring.md
- .claude/memory/feedback_backup_targets_never_guessed.md
- .claude/memory/feedback_backup_check_only_billed.md
backlinks: [] backlinks: []
--- ---
@@ -24,39 +35,98 @@ Verifies every business paying ACG for GPS (Guru Protection Service) has GuruRMM
correctly enrolled: org/site exists, all billed machines present and reporting, billed correctly enrolled: org/site exists, all billed machines present and reporting, billed
services (backup/AV/email/VoIP) actually configured. "Should have" = Syncro active services (backup/AV/email/VoIP) actually configured. "Should have" = Syncro active
recurring GPS schedules (device counts + service lines); "reality" = live GuruRMM recurring GPS schedules (device counts + service lines); "reality" = live GuruRMM
`/api/agents`, cross-checked against Datto EDR, Bitdefender, and Syncro assets. `/api/agents`, cross-checked against Datto EDR, Bitdefender, MSP360, and Syncro assets.
Scope: 40 active GPS clients (4 paused excluded: Marcia Ashton, Tucson Mountain Motors, Scope: 40 active GPS clients (4 paused excluded: Marcia Ashton, Tucson Mountain Motors,
Richard Pittman, Brenda Lopez). GPS device count = workstation + server SKUs only (AV Richard Pittman, Brenda Lopez). GPS device count = workstation + server SKUs only (AV
add-on/discount/setup lines excluded). Started 2026-07-03 (Howard). add-on/discount/setup lines excluded). Started 2026-07-03 (Howard); still open as of
2026-07-07, now mostly in a backup-verification and per-client cleanup tail.
## Current State (2026-07-04 night close) **Current single source of truth is `projects/gps-rmm-audit/gps-master-status.md`**
(regenerable from live GuruRMM `/api/agents` + Syncro backup seats + `backup-map.md`
verdicts) — the at-a-glance per-client list: GPS device count, RMM enrollment, backup
billing + status, wiki presence. `tracker.md` holds the detailed bucket-by-bucket history
and remains the reference for how each finding was reached; `backup-map.md` holds the
full Phase-4 backup verification detail.
- **Enrollment 111/189 devices** (true count, post-dedup). Baseline 2026-07-03: 46/189, ## Current State (2026-07-07)
32 clients short.
- **Bucket A (7, count matched)** — verified end-to-end: machines/backup/AV/vault/wiki. - **Enrollment**: closed steadily via ScreenConnect push + the Staging auto-enroll
1 wiki article created (Arizona Medical Transit). Findings held for Winter/Mike. pipeline since the 2026-07-03 baseline (46/189) through 112/189 at the 2026-07-05
- **Bucket B (8, present but short)** — RMM-vs-Datto-vs-Bitdefender matrix split it into close (that denominator excludes Glaz-Tech's 159-device billing anomaly). Per-client
real deploy gaps (IMC, Safesite, Horseshoe, Grabb & Durando, Quantum Wealth Mgmt — all counts are no longer tracked as one fleet-wide fraction — `gps-master-status.md` is
pushed via SC) and billing flags (Jimmy Company, Stamback Septic, Glaz-Tech — held). now the live source, listing RMM agent count against GPS billed count for each of the
- **Bucket C (25, no RMM org)** — 16 onboarded + ~44 agents deployed 2026-07-03; 3 of the 40 clients individually. **7 clients still have zero RMM footprint**: Andy's Mobile
"no footprint" clients were later found via the staging pipeline and onboarded Fuel, Business Services of Tucson LLC, Heieck (Sheila), Little Hearts Little Hands,
(Ridgetop Group, Gary A Hartman LLC, Robyn Pittman). True no-footprint remaining: MVAN Enterprises Inc, Residential and Renovation Engineering, The Marc Group — all
Little Hearts Little Hands, Residential and Renovation Engineering, Janet Altschuler, now have a GuruRMM client+site provisioned (site codes minted for the onsite kit,
Business Services of Tucson LLC, Marty Ryan (~5 clients, ~16 devices) — need an online see below) but no machine has enrolled yet; onboarding proceeds onsite or as machines
window, discovery, or onsite visit. come online.
- **Phase 4 AV matrix (2026-07-04)** — cid-matched Bitdefender + EDR vs GPS billing, - **Backup verification (Phase 4) — the major addition this week.** Authoritative
fleet-wide. NO-AV: 8 clients / 20 paid devices (down from 9/22). 4 of 5 reachable NO-AV backup-status source is now the **MSP360 Managed Backup Service API**
machines now protected (round 1). (`/api/Monitoring`, via the `msp360` skill), not B2 bucket contents — a bucket with
- Everything reachable via ScreenConnect is deployed; remaining gaps are offline files does not prove a recent backup ran. Only **16 of the 40 GPS clients actually
machines tracked by the daily/30-min automation until they surface. bill a backup line** (Syncro-verified 2026-07-06; the tracker's older `Svc=B` flags
were wrong for Horseshoe, Quantum Wealth Management, T&C Sorensen, and Business
Services of Tucson — none of those four bill backup). Verdict on the 16:
**7 OK** (Dataforth, Valley Wide Plastering, Curtis Plumbing, Ridgetop Group, Brett
Interiors, Gary A Hartman LLC, Mineralogical Record via ownCloud), **2 REVIEW**
(Glaz-Tech — 2nd server 251d stale; Jimmy Company — 12 seats billed, 1 machine),
**7 NOT BACKING UP** — 3 found in the MSP360/B2 pass (Grabb & Durando cloud plan
failing/0 bytes, Bill Tedards ~359d stale, Arizona Medical Transit 0 plans) plus
**4 confirmed by direct endpoint scan 2026-07-06/07** (Reliant Well Drilling, The
Prairie Schooner, Janet Altschuler, Design and Brand Envoys — no backup software,
service, scheduled task, Windows Server Backup, or Datto backup client on any
machine; Prairie's `TPS-SERVER` has a dead Windows Server Backup to USB, last ran
3/7/2018).
- **Revenue leaks identified** (backing up in MSP360/B2/Seafile, but Syncro has no Data
Backup line — billing question, held for Mike/Winter, nothing sent): Cascades of
Tucson, Len's Auto Brokerage, Russo Law Firm, Horseshoe Management, Business Services
of Tucson. **$0-priced backup lines** (billed a seat at $0 — bundled or under-billed,
same hold): Ridgetop Group, Janet Altschuler, Design and Brand Envoys, Arizona
Medical Transit.
- **Orphan B2 buckets resolved via MSP360 CompanyName**: `ACG-PST` = Peaceful Spirit
(not Prairie Schooner), `ACG-TCA` = Tucson Coin and Autograph (not T&C Sorensen —
corrects an earlier tentative guess; T&C Sorensen does not bill for backup).
- **Datto Workplace enumeration still BLOCKED** on a Mike-provisioned Super
Administrator API key (current key = 0 projects visible; the public API is
file-sharing-only with no admin/team/device endpoints — confirmed via Datto's
published docs, not assumption). The 4 backup-map "held" clients were resolved by
direct endpoint verification instead of waiting on that key.
- **20 client wiki articles created** (wiki count 56 -> 76) closing the wiki gaps
surfaced by `gps-master-status.md`.
- **Onboarded to GuruRMM this week**: Janet Altschuler, Marty Ryan, Business Services
of Tucson LLC (all real clients with a reachable machine); Andy's Mobile Fuel,
Little Hearts Little Hands, Residential and Renovation Engineering (onboarded with
no machine yet, specifically to mint site codes for the onsite installer kit).
MVAN Enterprises and The Marc Group turned out to already exist from the 2026-07-03
Bucket C onboarding (0 agents, not "no client") — the earlier "no footprint" label
meant zero agents, not zero RMM presence.
- **GuruRMM confirmed to support Windows, Linux, and macOS agents**
(`/install/<code>/{windows,linux,macos}`), resolving the two Mac-only clients
(Heieck, Residential and Renovation Engineering) as onsite-installable rather than
blocked.
- **Onsite installer kit built** at `C:\Users\Howard\Desktop\CLIENTS\` — one folder per
site (20) with Windows/Linux/macOS installer scripts + README, per-location
subfolders for the two multi-site clients (Grabb & Durando: Main/Bob's/Jeff's;
Glaz-Tech: Involta/SLC/Tucson) so the correct site code reaches the correct location.
Howard is taking this onsite to close the remaining no-footprint gaps in person.
- **A wrong weekend findings DM to Winter was deleted** (2026-07-06) — superseded by
the MSP360 + Syncro-seat data that arrived after it was sent. The corrected findings
were PATCHed into the existing Mike/Winter/Howard message thread in place rather than
sent as new messages, to keep one authoritative thread.
- **GPS-RMM-AutoEnroll scheduled task is now DISABLED** per Howard — manual sweeps only
(`bash .claude/scripts/gps-rmm-autoenroll.sh`) rather than automatic every 30 minutes.
**GPS-RMM-Progress** (daily 8:07am gap-check DM) remains active.
## Tooling ## Tooling
`projects/gps-rmm-audit/tools/`: `projects/gps-rmm-audit/tools/`:
- **bucketc-onboard-deploy.sh** `<"Client"> <bd_company_id> <slug>` — onboards a GuruRMM - **bucketc-onboard-deploy.sh** `<"Client"> <bd_company_id> <slug>` — onboards a GuruRMM
client+site (idempotent), vaults the enrollment key, deploys via SC to every machine in client+site (idempotent), vaults the enrollment key, deploys via SC to every machine in
the client's Bitdefender company with a live session. Used for Bucket C (16 clients). the client's Bitdefender company with a live session. Used for the original Bucket C
(16 clients) and later ad hoc onboardings (Janet Altschuler, Marty Ryan, BST, Andy's,
Little Hearts, Residential and Renovation Engineering).
- **needs-sc.py** — reconciles Syncro assets (authoritative inventory) vs live RMM agents - **needs-sc.py** — reconciles Syncro assets (authoritative inventory) vs live RMM agents
per client; writes `needs-screenconnect.md`; `MAXDAYS` env filters by Syncro per client; writes `needs-screenconnect.md`; `MAXDAYS` env filters by Syncro
`last_synced_at` recency. Online-detection superseded by `rebuild-and-push.py`. `last_synced_at` recency. Online-detection superseded by `rebuild-and-push.py`.
@@ -67,20 +137,27 @@ add-on/discount/setup lines excluded). Started 2026-07-03 (Howard).
guard added after the contamination bug (see Findings). guard added after the contamination bug (see Findings).
- **rebuild-and-push.py** — rebuilds the "not in RMM" list using the correct SC online - **rebuild-and-push.py** — rebuilds the "not in RMM" list using the correct SC online
signal (`ActiveConnections` `ProcessType==2`, not stale Syncro `last_synced_at`), pushes signal (`ActiveConnections` `ProcessType==2`, not stale Syncro `last_synced_at`), pushes
the generic **Staging** installer (`DARK-STORM-3150`) to every machine online now. the generic **Staging** installer (`DARK-STORM-3150`) to every machine online now. Now
run manually via `gps-rmm-autoenroll.sh` (loop task disabled).
- **reassign-staging.py** — moves agents out of `Staging - Auto Enroll` into their real - **reassign-staging.py** — moves agents out of `Staging - Auto Enroll` into their real
client by hostname -> Syncro `business_name` -> GuruRMM client (LLC/Inc/Corp-normalized). client by hostname -> Syncro `business_name` -> GuruRMM client (LLC/Inc/Corp-normalized).
Idempotent; `--dry` supported. Idempotent; `--dry` supported.
- **edr-deploy.py** — throttled Bitdefender -> Datto EDR deployer: recomputes coverage
from a live Datto EDR agent dump each run, targets ONLINE Windows RMM agents of GPS
clients with a vaulted registration key not already in EDR, pushes the official
Install-EDR one-liner through RMM one machine at a time (20s gap, retry-on-5xx,
poll-to-completion). Glaz-Tech excluded by design. Re-run to sweep newly-online
machines; idempotent.
`.claude/scripts/`: `.claude/scripts/`:
- **gps-rmm-autoenroll.sh** — closed loop: `rebuild-and-push.py` -> sleep 150s -> - **gps-rmm-autoenroll.sh** — closed loop: `rebuild-and-push.py` -> sleep 150s ->
`reassign-staging.py` -> post to `#dev-alerts`. Logs to `reassign-staging.py` -> post to `#dev-alerts`. Logs to
`projects/gps-rmm-audit/autoenroll.log`. Windows task **GPS-RMM-AutoEnroll** `projects/gps-rmm-audit/autoenroll.log`. **Windows task GPS-RMM-AutoEnroll is
(HOWARD-HOME): starts Mon 2026-07-06 06:00, every 30 min. Safe to re-run (server DISABLED** (Howard, 2026-07-07) — run this script manually instead of on the 30-min
v6.77+ dedups by device_id). Retire: `schtasks /Delete /TN GPS-RMM-AutoEnroll`. schedule. Safe to re-run (server v6.77+ dedups by device_id).
- **gps-rmm-progress-check.sh** — daily read-only: `targets.json` vs live `/api/agents`, - **gps-rmm-progress-check.sh** — daily read-only: `targets.json` vs live `/api/agents`,
DMs Howard the per-client gaps, reports COMPLETE when met. Windows task DMs Howard the per-client gaps, reports COMPLETE when met. Windows task
**GPS-RMM-Progress**, daily 8:07am, Howard-Home. **GPS-RMM-Progress**, daily 8:07am, Howard-Home — still active.
**Staging site pattern (`DARK-STORM-3150`)**: Bucket-C machines are often offline or **Staging site pattern (`DARK-STORM-3150`)**: Bucket-C machines are often offline or
briefly online, so per-client SC pushes miss the window. A Syncro policy pushes one briefly online, so per-client SC pushes miss the window. A Syncro policy pushes one
@@ -88,182 +165,351 @@ generic installer to all managed machines; anything that comes online enrolls in
catch-all client `Staging - Auto Enroll` / site `Staging` (key vaulted at catch-all client `Staging - Auto Enroll` / site `Staging` (key vaulted at
`infrastructure/gururmm-staging-site.sops.yaml`); `reassign-staging.py` auto-sorts it to `infrastructure/gururmm-staging-site.sops.yaml`); `reassign-staging.py` auto-sorts it to
the real client. Proven end-to-end (15 machines enrolled + correctly reassigned in one the real client. Proven end-to-end (15 machines enrolled + correctly reassigned in one
run, 2026-07-04); this is what `gps-rmm-autoenroll.sh` runs every 30 minutes. run, 2026-07-04). The 30-minute scheduled loop was disabled 2026-07-07; the mechanism is
run on demand instead.
**Reassign / move flow**: `POST /api/agents/:id/move {site_id}` is a clean UPDATE **Reassign / move flow**: `POST /api/agents/:id/move {site_id}` is a clean UPDATE
(confirmed not the dup root cause). Throttled 2s + 5x retry to ride out intermittent (confirmed not the dup root cause). Throttled 2s + 5x retry to ride out intermittent
500s. Used for both Staging reassignment and one-time site corrections (e.g. Dataforth 500s. Used for Staging reassignment, one-time site corrections (Dataforth D1/D2 split,
D1/D2 split, 39 moves). 39 moves), and per-client reassignment (Janet Altschuler, Marty Ryan, Reliant).
### Backup verification tooling (new, Phase 4)
- **`msp360` skill** (`.claude/skills/msp360/`) — full CRUD against the MSP360 MBS API
(`https://api.mspbackups.com`, Bearer login via `msp-tools/msp360-api.sops.yaml`).
Reads: `status`, `monitoring` (`--failed`/`--company`/`--stale`/`--json`), `companies`,
`users`, `licenses`, `billing`, raw GET. Writes (all `--confirm`-gated): create/delete
company, create/delete user, grant/release/revoke license, raw passthrough. This is
the **authoritative "is this client actually backing up" source** for the whole audit —
`/api/Monitoring` returns per-computer LastStart/Status/DataCopied across 66 plans /
37 companies. `/api/Computers` returns HTTP 400 (Remote Management API not enabled on
the account) — use Monitoring/Companies/Users instead.
- **`seafile` skill** — Seafile Pro Web API (admin creds vaulted `services/seafile-pro.sops.yaml`).
Reference implementation for the pattern; `usage` / `audit [--client] [--rmm] [--json]`.
- **`owncloud` skill** — SSH + `occ` (no OCS web-admin cred exists in the vault, only root
SSH to the VM); per-user size read from ownCloud's own MySQL filecache, `du` fallback
for dirty-cache users. Same `usage`/`audit` interface.
- **`datto-workplace` skill** — HTTP Basic (api-key:api-secret,
`msp-tools/datto-workplace.sops.yaml`); file-access API only, currently blind (0
projects visible) until Mike enables Super Administrator scope on the integration.
GuruRMM endpoint-detection (`Datto Workplace`/`Workplace2` patterns — never bare
`"Datto"`, which false-matches Datto EDR/RMM/AV) is the practical fallback signal.
- **`.claude/scripts/backup_common.py`** — shared plumbing for all four skills: repo-root/
vault resolution, minimal JSON HTTP, and a GuruRMM `RmmClient` (login, list agents, run
a read-only detection PowerShell to confirm a sync client is installed and signed in).
- **`b2` skill** (pre-existing) — Backblaze B2 buckets; `bucket-size` per-bucket (the
full `usage` scan across all buckets times out at 400s listing every object version).
- Skill routing consolidated (`.claude/SKILL_ROUTING.md`): a backup request is a two-step
route — (1) identify which backend the client actually uses via the client wiki +
`backup-map.md` (never assume B2/MSP360), then (2) route to that backend's skill.
Provisioning writes go through the target backend's own skill (all gated).
## Key Infrastructure & Integrations ## Key Infrastructure & Integrations
- **GuruRMM API** — `http://172.16.3.30:3001`, vault `infrastructure/gururmm-server.sops.yaml`. - **GuruRMM API** — `http://172.16.3.30:3001`, vault `infrastructure/gururmm-server.sops.yaml`.
Server at close: v6.77 (dedup fix deployed). Dashboard `rmm.azcomputerguru.com`; agent WS `wss://rmm-api.azcomputerguru.com/ws`
(resolves direct IP, not Cloudflare). Server at last check: v6.77+ (dedup fix deployed).
Install endpoints: `https://rmm.azcomputerguru.com/install/<site_code>/<platform>`,
platform = `windows | windows-x86 | windows-legacy | windows-legacy-x86 | linux | macos`
— confirmed Windows + Linux + macOS agents all exist. Offline-install path: raw binaries
at `/install/<code>/download/<variant>`.
- **ScreenConnect** — `https://computerguru.screenconnect.com`, RESTful API Manager ext - **ScreenConnect** — `https://computerguru.screenconnect.com`, RESTful API Manager ext
(Service.ashx). Primary deploy + metadata channel for the audit. CP slots: CP1=Company, (Service.ashx). Primary deploy + metadata channel for the audit. CP slots: CP1=Company,
CP2=Site, CP3=Department, CP4=Device Type, CP5-7 unused, CP8=Tag. API hides property CP2=Site, CP3=Department, CP4=Device Type, CP5-7 unused, CP8=Tag. API hides property
labels (values only); `UpdateSessionCustomProperties` replaces the whole 8-element labels (values only); `UpdateSessionCustomProperties` replaces the whole 8-element
array — always read current values before writing. array — always read current values before writing. Extension exposes only
`GetSessionsByName` (no full-fleet listing) — need a hostname to target a machine.
- **Syncro assets** — authoritative managed-device inventory (more complete than - **Syncro assets** — authoritative managed-device inventory (more complete than
BD/Datto for discovery). Public API has no policy endpoint (`GET /policies` = 404); SC BD/Datto for discovery). Public API has no policy endpoint (`GET /policies` = 404); SC
install policy is console-only. Assets carry `policy_folder_id` to target it. install policy is console-only. Assets carry `policy_folder_id` to target it. Recurring
schedules are the authoritative "billed backup seat" source — must match on line NAME,
not description (base GPS Workstation/Server SKUs carry "Backups" in boilerplate
description text, producing false positives if matched loosely).
- **MSP360 Managed Backup Service (MBS)** — `https://api.mspbackups.com`, vault
`msp-tools/msp360-api.sops.yaml`. Authoritative "is it backing up" source for the whole
fleet (37 companies / 59 users / 66 plans at discovery). `msp360` skill.
- **Backblaze B2** — 16 buckets, destination-only (not a freshness signal). `b2` skill.
- **Seafile Pro** — Docker on Jupiter, `http://172.16.3.20:8082` / `https://sync.azcomputerguru.com`.
5 active accounts / ~5 TB at discovery (Russo Law 2.2 TB is the only GPS client on it).
- **ownCloud** — VM `172.16.3.22` (cloud.acghosting.com), Rocky Linux 9.6, ownCloud 10.16
Community. 10 users / 19.2 TB at discovery (`pavon` 14.2 TB is a video archive, not a
backup; GPS clients on it: Mineralogical Record `minrec`, Business Services of Tucson
`bst`; The Marc Group `themarcgroup` is non-GPS/email-only).
- **Datto Workplace** — SaaS `acg.workplace.datto.com` (team 6) + on-prem Datto Workplace
Server VM `ACG-DWP-X-BB` on Jupiter (confirmed on Birth Biologic via endpoint
detection, non-GPS). File-access-only API; full enumeration blocked pending Mike's
Super Administrator key.
- **Datto EDR** — target AV end-state for nearly all clients. Managed via the - **Datto EDR** — target AV end-state for nearly all clients. Managed via the
`datto-edr` skill (`create-group` -> `mint-key` -> deploy). `avInstalled` reads null `datto-edr` skill (`create-group` -> `mint-key` -> deploy). `avInstalled` reads null
right after registration — confirm via org policy, not the raw field. right after registration — confirm via org policy, not the raw field. Authorized
agent count rose 228 -> 302 during the 2026-07-05 mass-deploy pass.
- **Bitdefender GravityZone** — retired as a coverage target, kept only as a discovery - **Bitdefender GravityZone** — retired as a coverage target, kept only as a discovery
source. Company names carry the Syncro cid suffix (`_<cid>`) — exact join key vs source. Company names carry the Syncro cid suffix (`_<cid>`) — exact join key vs
`targets.json`, used for the Phase 4 matrix (fuzzy name matching gave false negatives `targets.json` / `gps-clients.json`.
on reversed names, e.g. Sheila Heieck).
## Findings & Decisions ## Findings & Decisions
- **Billing findings HELD for Winter/Mike** (an early DM to Winter was retracted before ### Machine / AV findings (2026-07-03/05, preserved)
this rule was set): Dataforth +8 machines over billed; Cascades duplicate agent - **Billing findings VERIFIED + SENT 2026-07-05** (an early DM to Winter was retracted
(`RECEPTIONIST-PC`) + unbilled backup bucket (`ACG-Cascades` exists, no Data Backup before the send-when-verified rule was set): Dataforth +9 machines over billed (52 RMM
line); Len's Auto `LAB-SVR` offline since 2026-06-18; AMT billed backup with no B2 vs 43 billed); Cascades `RECEPTIONIST-PC` — re-verified as **two live machines sharing
destination found; Jimmy Company (12 billed/1 real) and Stamback Septic (8 billed/~2 one hostname** (different device_ids, both online simultaneously), not a stale
real) — billing-vs-reality mismatch, nothing to enroll. duplicate — needs a rename, not a delete; Len's Auto `LAB-SVR` offline since
2026-06-18; Jimmy Company (12 billed/1 real) and Stamback Septic (8 billed/4 real, up
from 2) — billing-vs-reality mismatch. Sent via Discord DM to Mike, Winter, and Howard
2026-07-05; hold released.
- **Glaz-Tech anomaly**: 159 GPS billed vs 5 real machines in RMM/Datto EDR, vs 242 - **Glaz-Tech anomaly**: 159 GPS billed vs 5 real machines in RMM/Datto EDR, vs 242
Bitdefender records (stale, years of ghosts). RMM+Datto agree on the real footprint; Bitdefender records (stale, years of ghosts). RMM+Datto agree on the real footprint;
flagged for Mike's billing review, not treated as 154 missing agents. flagged for Mike's billing review, not treated as 154 missing agents. Its 2nd backup
- **AV strategy**: Bitdefender -> Datto EDR for all clients. Exception narrowed during seat (`GTI-INV-VMHOST`) is also 251 days stale — folded into the backup REVIEW verdict.
the audit (Dataforth originally excepted, then included 2026-07-04 — already 51 EDR - **AV strategy**: Bitdefender -> Datto EDR for all clients. **Final exception:
agents, only 5 BD endpoints left to convert: D1-ENGI-006, DESKTOP-L2LE31M, Glaztech only** (Dataforth was originally excepted too, then fully migrated
DATAFORTH-PC, SURFACEOPS, MING-HP). **Final exception: Glaztech only.** Target state 2026-07-04 — its final 5 legacy BD endpoints converted during the 2026-07-05 mass
per machine (non-exempt) = GuruRMM + Datto EDR + Bitdefender removed. Detail: memory deploy, except two genuine legacy-OS blocks below). Target state per machine
`project_av_migration_bitdefender_to_edr`. (non-exempt) = GuruRMM + Datto EDR + Bitdefender removed. Bitdefender removal is
- **NO-AV list (revised 2026-07-04 night)**: 8 clients / 20 paid devices — Little Hearts **not yet started** anywhere — deferred per-client, only after that client's EDR is
Little Hands (8), Ridgetop Group (3), Residential and Renovation Engineering (2), Janet confirmed ACTIVE. Detail: memory `project_av_migration_bitdefender_to_edr`.
Altschuler (2 — confirmed SEPARATE from JANC Excavation, Syncro 457710 vs 2248945), Business - **NO-AV mass deploy (2026-07-05)**: `edr-deploy.py` pushed EDR to ~78 machines across
Services of Tucson LLC (2), Gary A Hartman LLC (1), Robyn Pittman (1), Marty Ryan (1). ~25 clients in one throttled run (74/79 clean on the first pass). Two genuine
- **NO-AV round 1**: Datto EDR deployed to the 5 reachable machines. 4/5 confirmed legacy-OS blocks remain: **AMT-PC** (Windows 7 Home SP1, PowerShell 2.0 — modern
ACTIVE; Robyn Pittman's `DESKTOP-PL2RCGL` failed mid-delivery (went offline) — retry Infocyte agent crashes on execution) and **SRV02** (Curtis Plumbing, Windows Server
Monday, reg key in tracker.md. Rest ride autoenroll -> RMM -> EDR-push as they surface. 2008 R2 — agent launches but no-ops silently). Both need OS prerequisites or an
onsite/legacy-agent path. Five apparent "failures" that session were mostly false
negatives (Install-EDR is silent in non-interactive mode) — see Patterns below.
- **GuruRMM cross-site duplication bug — found, fixed, deployed (v6.77)**: both agent - **GuruRMM cross-site duplication bug — found, fixed, deployed (v6.77)**: both agent
enroll paths in `server/src/ws/mod.rs` deduped only by `(site_id, device_id)`. A enroll paths in `server/src/ws/mod.rs` deduped only by `(site_id, device_id)`. A
machine already enrolled at one site that ran a different site's installer (new code, machine already enrolled at one site that ran a different site's installer minted a
or Staging) wasn't found in the target site, so the server minted a duplicate row with duplicate row with the same device_id. Fix: `db::get_agent_by_device_id()` (global
the same device_id. Fix: `db::get_agent_by_device_id()` (global lookup) re-homes the lookup) re-homes the existing record instead of creating a duplicate. Deployed via
existing record instead of creating a duplicate; the enrollment-key path only uses the cherry-pick (`hotfix-agent-dedup` -> main). Cleanup: 40 stale orphans deleted, 39
global match when a real device_id exists, falling back to site-scoped hostname one-time moves, 0 failures. Full plan: `dedup-plan.md`.
matching otherwise (so distinct same-named machines, e.g. 4x `SERVER`, are never
merged). No blanket `UNIQUE(device_id)` constraint added (would collide on legacy
hostname-fallback rows). Deployed via cherry-pick (`hotfix-agent-dedup` -> main, not a
merge); webhook auto-built 6.66 -> 6.77. Verified live: re-running the staging
installer reused the same record; move+restart kept placement. Cleanup: 40 stale
orphans deleted (0 failed), 39 one-time moves (0 failed — 24 Dataforth D1->D2, 13
Staging->real client, 2 diagnostic corrections). 13 same-hostname/different-device_id
pairs deliberately left for manual review. Full plan: `dedup-plan.md`.
- **ScreenConnect contamination incident**: fleet `sc-cleanup.py` used - **ScreenConnect contamination incident**: fleet `sc-cleanup.py` used
`GetSessionsByName`, which matches by name across ALL clients, not client-scoped. `GetSessionsByName`, which matches by name across ALL clients, not client-scoped.
Writing Company to shared names (`SERVER`, `Accounting`) cross-contaminated other Writing Company to shared names (`SERVER`, `Accounting`) cross-contaminated other
clients' sessions with the last-processed value (11 `SERVER` sessions across ~8 clients' sessions with the last-processed value. Fixed (`if len(ss) > 1: continue`);
clients stamped "Zeus Nestora"; 3 `Accounting` sessions mis-stamped). Fixed (skip any remediated via `GuestNetworkAddress` (WAN IP) -> client. **5 sessions unrecoverable**
lookup returning >1 session); remediated via `GuestNetworkAddress` (WAN IP) -> client, (3 `SERVER` + 2 `Accounting`) — left blank for manual SC-console tagging. Logged as
using a WAN map from unique-named machines only. **5 sessions unrecoverable** (original both correction and friction.
overwritten, unidentifiable) — 3 `SERVER` + 2 `Accounting`, left blank for manual - **CP-QB (Curtis Plumbing) root cause found and fixed (2026-07-04/05)**: corrupt WMI
SC-console tagging. Logged as both correction and friction. class store (`Win32_Processor` = Invalid class, broken since ~2022) killed the
universal installer's architecture check and froze its SC inventory since 2022;
`winmgmt /verifyrepository` reported CONSISTENT (doesn't catch a missing class) —
`mofcomp cimwin32.mof` restored it. Installer then ran clean; enrolled + reassigned.
- **IMC-PRINTSERVER installed but TLS-blocked**: agent fully installed and running
(manual no-PowerShell install over LAN, since PowerShell is dead on the box and SMB
hangs), but the box silently drops ALL outbound TLS (even google.com — ClientHello
then silence) while ScreenConnect works (non-TLS relay). Escalated to Mike (by-design
lockdown vs. fix); **no reinstall needed** — the agent self-enrolls the instant TLS
egress is unblocked.
### Backup findings (2026-07-05/07, new this week)
- **Only 16 of 40 GPS clients actually bill a backup line** — a Syncro sub-agent pull
of recurring-schedule Data Backup line-item NAME (not description, to avoid the
boilerplate false-positive) corrected the tracker's old `Svc=B` flags for Horseshoe
Management, Quantum Wealth Management, T&C Sorensen, and Business Services of
Tucson — none of the four actually bill backup.
- **7 of the 16 billed clients are NOT backing up**: Grabb & Durando (`GND-SERVER`
cloud plan failing, 0 bytes — flapping), Bill Tedards (`DESKTOP-SUFJR0J` last good
359 days ago), Arizona Medical Transit (MSP360 company exists, 0 plans), and 4
**endpoint-verified 2026-07-06** with a full on-machine scan (no backup software,
service, scheduled task, WSB, or Datto backup client on any machine, and nothing in
MSP360/B2/Seafile/ownCloud/Datto Workplace): Reliant Well Drilling, The Prairie
Schooner (a Windows Server Backup to USB exists on `TPS-SERVER` but last ran
3/7/2018, ~8 years dead), Janet Altschuler (only a stopped Dropbox client), Design
and Brand Envoys (`OSGOOD-PAUL` was offline/unscanned).
- **2 REVIEW verdicts** — seats billed exceed machines actually backing up: Jimmy
Company (12 seats / 1 machine, `Blaster2`, known 12/1 anomaly) and Glaz-Tech
Industries (2 seats / 1 truly current — `GTI-INV-VMHOST` 251 days stale).
- **Revenue leaks** (backing up, not billed — held for Mike/Winter, nothing sent):
Cascades of Tucson (`CS-SERVER` daily, B2 `ACG-Cascades`), Len's Auto Brokerage
(`LAB-SVR`, now stale, B2 `ACG-Lens`), Russo Law Firm (`RUSSO-SRV` 2.7 TB cloud +
Seafile 2.2 TB), Horseshoe Management (`HSM-NewServer` daily, B2 `Horseshoe`),
Business Services of Tucson (`KAT1` daily, B2 `ACG-BST` + ownCloud `bst`).
- **$0-priced backup lines** (billed a seat at $0 — bundled or under-billed, flag for
Mike/Winter): Ridgetop Group, Janet Altschuler, Design and Brand Envoys, Arizona
Medical Transit.
- **Orphan B2 buckets resolved via MSP360 CompanyName**: `ACG-PST` = Peaceful Spirit
(22,054 files, non-GPS), `ACG-TCA` = Tucson Coin and Autograph (non-GPS) — corrects an
earlier tentative guess that `ACG-TCA` was T&C Sorensen. Other identified orphans:
`ACG-REDNOUR` = Rednour Law, `ACG-Rohrbach` = Rohrbach, `ACG-IX`/`ACG-Internal`/
`ACG-Tech03L` = ACG internal, `MSPBackups20200311` = stale ex-client FSG (deletion
candidate). ownCloud `mara`/`jburger` still unidentified (verify).
- **Datto Workplace research**: confirmed there is no admin/team/device REST API at
all — the public API is strictly file-sharing; user/storage administration is
Workplace-Manager-portal-only. A Super Administrator toggle on the existing API
integration (or team-sharing projects to it) is the only way to make `/file/projects`
enumerate the whole team. That toggle is a portal action Claude cannot perform — Mike
is provisioning a Super-Administrator-scoped integration; parked until it lands.
- **Rule (Howard, standing)**: backup targets are never guessed. A billed seat count
higher than machines-currently-backing-up is a Mike/Winter billing question, not a
deploy instruction. Memory: `feedback_backup_targets_never_guessed`. Corollary added
2026-07-07: only check backup status for clients that actually bill a backup line
(e.g. stop checking Marty Ryan, which has none) — memory
`feedback_backup_check_only_billed`.
## Patterns & Known Issues ## Patterns & Known Issues
- **`GetSessionsByName` matches across ALL SC clients** — never write properties to a - **`GetSessionsByName` matches across ALL SC clients** — never write properties to a
name returning >1 session (`if len(ss) > 1: continue`); root cause of the contamination name returning >1 session (`if len(ss) > 1: continue`); root cause of the
incident above. contamination incident above. The extension also exposes no full-fleet listing —
finding a specific client's machines requires already knowing hostnames.
- **`/api/agents/:id/move` intermittently 500s under rapid calls** (transient DB/load; - **`/api/agents/:id/move` intermittently 500s under rapid calls** (transient DB/load;
handler is a clean UPDATE) — throttle 2s + retry 5x reliably rides over it. handler is a clean UPDATE) — throttle 2s + retry 5x reliably rides over it.
- **Syncro `last_synced_at` goes stale** when agent check-in breaks, making an online - **Syncro `last_synced_at` goes stale** when agent check-in breaks, making an online
machine look offline 45+ days. Use SC `ActiveConnections` `ProcessType==2` + machine look offline 45+ days. Use SC `ActiveConnections` `ProcessType==2` +
`GuestInfoUpdateTime` instead (originally read `GuestConnectedCount`, always null). `GuestInfoUpdateTime` instead.
- **Datto EDR org agent/site counts are stale rollups** after re-parenting Locations - **Datto EDR org agent/site counts are stale rollups** after re-parenting Locations
(`PATCH /Locations/{id} {organizationId}`) — trust a fresh `GET /Locations` grouped by trust a fresh `GET /Locations` grouped by org, not the org-list summary.
org, not the org-list summary. One raw `GET` also silently ignored a `--filter` param — - **`Install-EDR` is silent in non-interactive mode** — every status/error line is
verify with an unfiltered fetch + manual group-by when in doubt. gated behind `if ($Interactive)`; a "completed / blank output" RMM result means
nothing. The visible "Installed RTS agent" text comes from `agent.windows.exe`
itself, so its absence just means the exe never ran (early-return on an existing
service, or a crash). Verify with `-Interactive *> file` + read-back, or check
`Get-Service huntAgent` + the binary path directly. `-Force` reinstalls over a stale
huntAgent service left by a broken partial install.
- **RMM command output is in `.stdout`, not `.output`**, on `GET /api/commands/:id`. - **RMM command output is in `.stdout`, not `.output`**, on `GET /api/commands/:id`.
- **Syncro recurring-schedule line matching must use the line NAME, not description** —
base GPS Workstation/Server SKUs carry the word "Backups" in boilerplate description
text, producing false-positive "bills for backup" matches.
- **MSP360 bucket presence != a current backup** — B2/Seafile/ownCloud show a
destination exists; MSP360 `/api/Monitoring` LastStart+Status is the only real
freshness signal. `/api/Computers` is gated off (HTTP 400, feature not enabled on
this account) — use Monitoring/Companies/Users instead.
- **Datto backup-detection regex must be precise** (`"Datto Workplace"`/`"Workplace2"`)
— a bare `"Datto"` pattern false-matches Datto EDR/RMM/AV agents that have nothing to
do with backup.
- AV coverage splits by client size (Datto for large, Bitdefender historically for - AV coverage splits by client size (Datto for large, Bitdefender historically for
small) — check **both** before declaring a gap; formalized by the Phase 4 cid join. small) — check **both** before declaring a gap.
- SC coverage varies per client — universal on some (IMC/Horseshoe/QWM/Safesite), sparse - SC coverage varies per client — verify a live session exists before assuming the
on others (Grabb & Durando) — verify a live session exists before assuming the channel channel works; where absent, discover via Syncro assets instead.
works; where absent, discover via Syncro assets instead.
- The client wiki is the primary source of truth for machine -> department/person/ - The client wiki is the primary source of truth for machine -> department/person/
location during SC cleanup; where missing, learn it and write it back. location during SC cleanup, and now also for **which backup backend a client uses**
(per the consolidated wiki-first skill routing) — where missing, learn it and write
it back.
- **GuruRMM install scripts are bootstrappers, not offline installers** — the
`/install/<code>/<platform>` endpoints return a script that downloads the
arch-matched binary at run time (needs onsite internet); raw binaries for true
offline install live at `/install/<code>/download/<variant>`.
## Operations Playbook ## Operations Playbook
- **Daily 8:07am** — `GPS-RMM-Progress` runs `gps-rmm-progress-check.sh`, DMs Howard the - **Daily 8:07am** — `GPS-RMM-Progress` runs `gps-rmm-progress-check.sh`, DMs Howard the
gaps vs `targets.json`; reports COMPLETE and retires when met. gaps vs `targets.json`; reports COMPLETE and retires when met.
- **Every 30 min from Mon 2026-07-06 06:00** — `GPS-RMM-AutoEnroll` runs - **Auto-enroll (manual, on demand)** — `GPS-RMM-AutoEnroll`'s 30-minute schedule is
`gps-rmm-autoenroll.sh` (push Staging to online-but-missing machines, wait 150s, **disabled** as of 2026-07-07 (Howard: manual sweeps only). Run
reassign, post to `#dev-alerts`). Retire once fully enrolled. `bash .claude/scripts/gps-rmm-autoenroll.sh` by hand to push Staging to online-but-
- **Add a client (Bucket C pattern)**: `bucketc-onboard-deploy.sh "<Client>" <bd_company_id> missing machines, wait, reassign, and post to `#dev-alerts`.
<slug>` — creates client+site, vaults the key, deploys to reachable Bitdefender - **Add a client (no-footprint pattern)**: `bucketc-onboard-deploy.sh "<Client>"
machines. Domain-joined pattern: DC already in RMM -> AD is the authoritative list -> <bd_company_id> <slug>` — creates client+site, vaults the key, deploys to reachable
push the site installer to domain members via SC (IMC template — DC remote-exec, Bitdefender machines. Domain-joined pattern: DC already in RMM -> AD is the
WMI/DCOM, `schtasks /S`, WinRM all failed on default Win10/11 clients; SC authoritative list -> push the site installer to domain members via SC. If no
`send-command` as SYSTEM was the only clean channel). Steps + encoding: BD/Datto footprint exists at all, onboard anyway (mints the site code) and deploy
`reference_rmm_deploy_via_screenconnect`. onsite or via the flash-drive kit.
- **Deploy Datto EDR**: create-group -> mint-key (via `datto-edr` skill) -> push - **Deploy Datto EDR**: create-group -> mint-key (via `datto-edr` skill) -> push
`Install-EDR` through GuruRMM (`POST /api/agents/:id/command`, visible stdout) rather `Install-EDR` through GuruRMM (`POST /api/agents/:id/command`, visible stdout only
than blind SC push (failed on offline-prone boxes, e.g. Dataforth `DATAFORTH-PC`, same with `-Interactive`) rather than blind SC push. Re-run `edr-deploy.py` to sweep newly
as `CP-QB`). Reg keys tracked in `tracker.md`, not reproduced here. online machines; it recomputes coverage from a live EDR dump each run.
- **SC metadata cleanup, per client**: wiki first, then hostname role tokens, then UniFi - **SC metadata cleanup, per client**: wiki first, then hostname role tokens, then UniFi
switch/AP naming; run `sc-cleanup.py` for the safe no-guess pass; never touch Site switch/AP naming; run `sc-cleanup.py` for the safe no-guess pass; never touch Site
without a topology source. without a topology source.
- **Backup verification, per client**: check the client wiki + `backup-map.md` first
for which backend applies (never assume B2/MSP360); confirm with the matching skill
(`msp360`/`b2`/`seafile`/`owncloud`/`datto-workplace`); only check clients that
actually bill a backup line (`gps-master-status.md` "Billed for backup" table).
Endpoint-scan directly (RMM PowerShell) when the destination-side tools are
inconclusive or blocked (as done for the 4 Datto-pending clients).
- **Onsite visit**: use the flash-drive kit at `Desktop\CLIENTS\<slug>\` — per-site
installer scripts (Windows/Linux/macOS) + README; multi-site clients have
per-location subfolders with the correct site code for that location.
## History Highlights ## History Highlights
- **2026-07-03** — Audit started; 40 active GPS clients identified; `tracker.md` built - **2026-07-03** — Audit started; 40 active GPS clients identified; `tracker.md` built
(Bucket A 7 / B 8 / C 25); `GPS-RMM-Progress` registered (baseline 46/189). (Bucket A 7 / B 8 / C 25); `GPS-RMM-Progress` registered (baseline 46/189). IMC deploy
- **2026-07-03** — Bucket A verified end-to-end; renamed mislabeled client "Russo, established the ScreenConnect send-command channel as the working deploy pattern
Steve" -> Russo Law Firm; set the AV strategy (Bitdefender -> Datto EDR). after 4 remote-exec methods failed. Bucket B/C pushed and onboarded (~60 agents).
- **2026-07-03** — IMC deploy as the template: DC remote-exec failed 4 ways; pivoted to - **2026-07-04** — Fleet-wide SC cleanup pass (~45 clients); found + remediated the
SC send-command (1 -> 12 agents); saved as `reference_rmm_deploy_via_screenconnect`. shared-hostname cross-contamination bug. Corrected online-detection logic (SC
- **2026-07-03** — Bucket B push (IMC/Horseshoe/QWM/Safesite/Grabb) via SC; Bucket C `ActiveConnections` vs stale Syncro `last_synced_at`); Staging auto-enroll pipeline
onboarded (16 clients, ~44 agents) via `bucketc-onboard-deploy.sh`. proven end-to-end. GuruRMM dup-agent bug found/fixed/deployed (v6.77); 40 orphans
- **2026-07-03** — Discovery pivoted to Syncro assets (authoritative); `needs-sc.py` deleted + 39 moves, 0 failures. Phase 4 AV matrix built (9/22 NO-AV); EDR "Default
built; found Syncro's stored SC session GUID on assets was stale. RMM Org" dismantled (18 orgs created, 21 locations re-parented), NO-AV revised to
- **2026-07-03 evening** — SC session-hygiene cleanup began (Dataforth, then Cascades): 8 clients/20 devices; round-1 EDR remediation (4/5 active). Session closed at
reverse-engineered CP1-CP8 slots; normalized Company, set Site/Dept/Device Type via 111/189 enrollment.
UniFi + client wiki. - **2026-07-05** — CP-QB fixed (corrupt WMI class store) and enrolled. IMC-PRINTSERVER
- **2026-07-04** — Fleet-wide SC easy-win pass (~45 clients); found + remediated the installed but found TLS-egress-blocked (escalated to Mike). Full Bitdefender ->
cross-contamination bug from shared hostnames. Datto EDR mass deploy: `edr-deploy.py` protected ~78 machines across ~25 clients in
- **2026-07-04** — Corrected online-detection logic (SC `ActiveConnections` vs stale one throttled run (authorized EDR agents 228 -> 302); 2 genuine legacy-OS blocks
Syncro `last_synced_at`); Staging auto-enroll pipeline proven end-to-end (15 machines). identified (AMT-PC Win7/PS2, SRV02 Server 2008 R2). Held billing findings verified
- **2026-07-04** — GuruRMM dup-agent bug found/fixed/deployed (v6.77); 40 orphans deleted and sent to Mike/Winter/Howard via Discord DM. Built the `seafile`, `owncloud`, and
+ 39 moves executed, 0 failures. `datto-workplace` skills (+ shared `backup_common.py`) after Howard corrected the
- **2026-07-04** — SC<->RMM consistency pass across all 333 agents; closed the no-SC gap assumption that backup destinations were B2-only. Built the `msp360` skill (full
for 6 machines by pushing the SC installer through RMM itself. CRUD) once its API was found to be the authoritative backup-status source; code
- **2026-07-04** — Phase 4 AV matrix built (Bitdefender cid join vs GPS billing vs review (18 agents) fixed 10 findings including a credential-leak in dry-run output.
EDR): 9/22 NO-AV, 27-client/141-endpoint BD->EDR migration scope identified. Consolidated backup skill routing behind a wiki-first rule. Ran the first full
- **2026-07-04 night** — EDR "Default RMM Org" dismantled: 18 orgs created, 21 locations 40-client backup verification (16 of 40 actually bill backup; 7 OK / 2 REVIEW /
re-parented; NO-AV revised to 8 clients/20 devices. 3 confirmed-not-backing-up + 4 held pending Datto).
- **2026-07-04 night** — NO-AV round 1: EDR deployed to 5 machines, 4/5 active; - **2026-07-06** — Deleted the wrong weekend findings DM to Winter (superseded by the
`GPS-RMM-AutoEnroll` registered; session closed at 111/189, verification sweep green. MSP360 + Syncro-seat data). Endpoint-verified the 4 held clients (Reliant, Prairie
Schooner, Janet Altschuler pending onboard, Design and Brand Envoys) — all confirmed
NOT backing up. `backup-map.md` fully rewritten with MSP360 as source of truth.
- **2026-07-07** — Onboarded Janet Altschuler and Marty Ryan to GuruRMM (machines
located via SC hostnames, agents pushed and enrolled); confirmed Janet's machines
already have Datto EDR (left alone). Updated the Mike/Winter Discord messages in
place (PATCH) with the endpoint-verified results rather than sending new ones. Built
`gps-master-status.md` as the single current per-client list (superseding repeated
reconciliation of 3 stale docs), consolidating GPS device count, RMM enrollment,
backup billing + verdict, and wiki presence for all 40 clients. Created the 20
missing client wiki articles via two parallel sub-agents (wiki count 56 -> 76).
Onboarded Business Services of Tucson (real machine) and, to generate site codes for
the onsite kit, Andy's Mobile Fuel and Little Hearts Little Hands and Residential and
Renovation Engineering (no machine yet). Confirmed GuruRMM has Windows, Linux, and
macOS agents. Built the flash-drive onsite installer kit at `Desktop\CLIENTS\` (20
site folders, per-location subfolders for Grabb & Durando and Glaz-Tech). Disabled
the `GPS-RMM-AutoEnroll` 30-minute schedule (manual sweeps only, per Howard).
## Pending / Next Steps (Monday queue, 2026-07-06) ## Pending / Next Steps
- Run `GPS-RMM-AutoEnroll` (armed 06:00) to harvest the ~25 machines active within 14 - **Howard onsite tour** with the `Desktop\CLIENTS\` flash-drive kit to close the 7
days as offices reopen. zero-RMM-footprint clients in person (Andy's Mobile Fuel, Business Services of
- Dataforth EDR tail: convert the remaining 4 BD endpoints (D1-ENGI-006, DATAFORTH-PC, Tucson — machine `KAT1` etc. queued for reconnect, Heieck/Little Hearts/Residential
SURFACEOPS, MING-HP — DESKTOP-L2LE31M was reinstalled/retired, stale BD record deleted and Renovation Engineering/The Marc Group/MVAN — verify which still need a physical
2026-07-04), then remove BD from Dataforth. visit vs. a reconnect). Report machine details back into each client wiki +
- Retry the failed EDR install on `DESKTOP-PL2RCGL` (Robyn Pittman). `gps-master-status.md` as they land.
- Read the error files left on `CP-QB` and `IMC-PRINTSERVER` - **Mike/Winter still owe replies** (as of 2026-07-07) on: the 4 endpoint-verified
(`C:\Windows\Temp\gururmm-install-err.txt`) from the failed holiday-evening pushes. not-backing-up clients (Reliant, Prairie Schooner, Janet Altschuler, Design and Brand
- Confirm `avInstalled` actually enables via org policy for the round-1 EDR deploys. Envoys — decision needed: bill correctly, deploy real backup, or drop the line), the
- Locate or confirm-unmanaged: Little Hearts Little Hands, Residential and Renovation Datto Workplace Super Administrator API key (current key = 0 projects, blocking full
Engineering, Janet Altschuler (separate from JANC — confirmed 2026-07-04), Business Datto enumeration), and the general backup billing decisions below.
Services of Tucson LLC, Marty Ryan. - **7 FAIL + 2 REVIEW backup verdicts need a billing/remediation decision**: Grabb &
- Send held billing findings to Winter/Mike once fully verified (Dataforth +8, Cascades Durando (failing cloud plan), Bill Tedards (359d stale), Arizona Medical Transit (0
dup+unbilled backup, Len's LAB-SVR offline, AMT backup destination unknown, plans), Reliant/Prairie Schooner/Janet Altschuler/Design and Brand Envoys (endpoint-
Jimmy/Stamback/Glaz-Tech anomalies). confirmed none), Jimmy Company (12/1 seat mismatch), Glaz-Tech (2/1, 2nd server 251d
- Continue Phase 4: backup verification (B2/MSP360 vs billed lines), then email host stale). None of these are deploy instructions per the standing rule — Mike/Winter
verification (several "Exchange Hosted Email" lines still undocumented). decide bill-vs-deploy-vs-drop.
- Manual SC-console cleanup: remove flagged duplicate sessions (Cascades - **Revenue-leak decisions** (bill vs. decommission, held, nothing sent): Cascades,
`RECEPTIONIST-PC`, Dataforth `eng-dev-server`, Grabb `DESKTOP-NFK4F5P`, Safesite Len's, Russo, Horseshoe, Business Services of Tucson; plus the 4 $0-priced lines
`0226-Lenovo`/`MSI`); manually tag the 5 unrecoverable blank sessions. (Ridgetop, Janet Altschuler, Design and Brand Envoys, AMT).
- **Two legacy-OS EDR blocks** remain unresolved: AMT-PC (Win7 SP1/PS2) and SRV02
(Server 2008 R2) — need OS prerequisites, the legacy-agent path, or an onsite visit.
- **Bitdefender removal not started anywhere** — per-client, only after EDR is
confirmed ACTIVE for that client; do after the current tail is closed.
- **IMC-PRINTSERVER** still awaiting Mike's ruling on the TLS-egress lockdown
(by-design vs. fix); no reinstall needed once unblocked.
- **Manual SC-console cleanup**: remove flagged duplicate sessions (Cascades
`RECEPTIONIST-PC` needs a RENAME not a delete — confirmed two live machines sharing
one hostname; Dataforth `eng-dev-server`; Grabb `DESKTOP-NFK4F5P`; Safesite
`0226-Lenovo`/`MSI`); manually tag the 5 unrecoverable blank sessions from the
contamination incident.
- **Identify remaining unknowns**: ownCloud `mara`/`jburger` accounts (verify);
Mineralogical Record's ownCloud vs. `universal-minerals.md` wiki article — confirm
which is current (verify).
- Continue Phase 4 to email host verification (several "Exchange Hosted Email" lines
across Bucket A clients still undocumented) once the backup tail closes.
## Backlinks ## Backlinks
*(none yet — link from client wiki articles touched by this audit: Dataforth, Cascades *(link from client wiki articles touched by this audit Dataforth, Cascades of
of Tucson, Instrumental Music Center, Arizona Medical Transit, and others as their pages Tucson, Instrumental Music Center, Arizona Medical Transit, Janet Altschuler, Marty
come to reference this project)* Ryan, Reliant Well Drilling, and the 20 newly created client articles as they come to
reference this project)*