claudetools/wiki/projects/gps-rmm-audit.md

type, name, display_name, last_compiled, compiled_by, sources, backlinks

type	name	display_name	last_compiled	compiled_by	sources	backlinks
project	gps-rmm-audit	GPS -> GuruRMM Coverage Audit	2026-07-07	HOWARD-HOME/claude-main	wiki/_templates/project.md projects/gps-rmm-audit/gps-master-status.md projects/gps-rmm-audit/backup-map.md projects/gps-rmm-audit/tracker.md projects/gps-rmm-audit/dedup-plan.md projects/gps-rmm-audit/session-logs/2026-07/2026-07-03-howard-gps-rmm-coverage-audit.md projects/gps-rmm-audit/session-logs/2026-07/2026-07-05-howard-backup-verification-skills.md projects/gps-rmm-audit/session-logs/2026-07/2026-07-05-howard-edr-mass-deploy-imc-printserver.md projects/gps-rmm-audit/session-logs/2026-07/2026-07-05-howard-cpqb-fixed-imcprintserver-tls-blocked.md projects/gps-rmm-audit/session-logs/2026-07/2026-07-05-howard-gps-backup-map-msp360.md projects/gps-rmm-audit/session-logs/2026-07/2026-07-07-howard-gps-onboard-endpoint-backup-verify.md projects/gps-rmm-audit/session-logs/2026-07/2026-07-07-howard-gps-master-status-wikis-onsite-kit.md .claude/memory/project_av_migration_bitdefender_to_edr.md .claude/memory/reference_rmm_deploy_via_screenconnect.md .claude/memory/reference_screenconnect_custom_property_slots.md .claude/memory/feedback_screenconnect_cleanup_wiki_source.md .claude/memory/reference_msp360_backup_monitoring.md .claude/memory/feedback_backup_targets_never_guessed.md .claude/memory/feedback_backup_check_only_billed.md

GPS -> GuruRMM Coverage Audit

Overview

Verifies every business paying ACG for GPS (Guru Protection Service) has GuruRMM correctly enrolled: org/site exists, all billed machines present and reporting, billed services (backup/AV/email/VoIP) actually configured. "Should have" = Syncro active recurring GPS schedules (device counts + service lines); "reality" = live GuruRMM /api/agents, cross-checked against Datto EDR, Bitdefender, MSP360, and Syncro assets.

Scope: 40 active GPS clients (4 paused excluded: Marcia Ashton, Tucson Mountain Motors, Richard Pittman, Brenda Lopez). GPS device count = workstation + server SKUs only (AV add-on/discount/setup lines excluded). Started 2026-07-03 (Howard); still open as of 2026-07-07, now mostly in a backup-verification and per-client cleanup tail.

Current single source of truth is projects/gps-rmm-audit/gps-master-status.md (regenerable from live GuruRMM /api/agents + Syncro backup seats + backup-map.md verdicts) — the at-a-glance per-client list: GPS device count, RMM enrollment, backup billing + status, wiki presence. tracker.md holds the detailed bucket-by-bucket history and remains the reference for how each finding was reached; backup-map.md holds the full Phase-4 backup verification detail.

Current State (2026-07-07)

• Enrollment: closed steadily via ScreenConnect push + the Staging auto-enroll pipeline since the 2026-07-03 baseline (46/189) through 112/189 at the 2026-07-05 close (that denominator excludes Glaz-Tech's 159-device billing anomaly). Per-client counts are no longer tracked as one fleet-wide fraction — gps-master-status.md is now the live source, listing RMM agent count against GPS billed count for each of the 40 clients individually. 7 clients still have zero RMM footprint: Andy's Mobile Fuel, Business Services of Tucson LLC, Heieck (Sheila), Little Hearts Little Hands, MVAN Enterprises Inc, Residential and Renovation Engineering, The Marc Group — all now have a GuruRMM client+site provisioned (site codes minted for the onsite kit, see below) but no machine has enrolled yet; onboarding proceeds onsite or as machines come online.
• Backup verification (Phase 4) — the major addition this week. Authoritative backup-status source is now the MSP360 Managed Backup Service API (/api/Monitoring, via the msp360 skill), not B2 bucket contents — a bucket with files does not prove a recent backup ran. Only 16 of the 40 GPS clients actually bill a backup line (Syncro-verified 2026-07-06; the tracker's older Svc=B flags were wrong for Horseshoe, Quantum Wealth Management, T&C Sorensen, and Business Services of Tucson — none of those four bill backup). Verdict on the 16: 7 OK (Dataforth, Valley Wide Plastering, Curtis Plumbing, Ridgetop Group, Brett Interiors, Gary A Hartman LLC, Mineralogical Record via ownCloud), 2 REVIEW (Glaz-Tech — 2nd server 251d stale; Jimmy Company — 12 seats billed, 1 machine), 7 NOT BACKING UP — 3 found in the MSP360/B2 pass (Grabb & Durando cloud plan failing/0 bytes, Bill Tedards ~359d stale, Arizona Medical Transit 0 plans) plus 4 confirmed by direct endpoint scan 2026-07-06/07 (Reliant Well Drilling, The Prairie Schooner, Janet Altschuler, Design and Brand Envoys — no backup software, service, scheduled task, Windows Server Backup, or Datto backup client on any machine; Prairie's TPS-SERVER has a dead Windows Server Backup to USB, last ran 3/7/2018).
• Revenue leaks identified (backing up in MSP360/B2/Seafile, but Syncro has no Data Backup line — billing question, held for Mike/Winter, nothing sent): Cascades of Tucson, Len's Auto Brokerage, Russo Law Firm, Horseshoe Management, Business Services of Tucson. $0-priced backup lines (billed a seat at $0 — bundled or under-billed, same hold): Ridgetop Group, Janet Altschuler, Design and Brand Envoys, Arizona Medical Transit.
• Orphan B2 buckets resolved via MSP360 CompanyName: ACG-PST = Peaceful Spirit (not Prairie Schooner), ACG-TCA = Tucson Coin and Autograph (not T&C Sorensen — corrects an earlier tentative guess; T&C Sorensen does not bill for backup).
• Datto Workplace enumeration still BLOCKED on a Mike-provisioned Super Administrator API key (current key = 0 projects visible; the public API is file-sharing-only with no admin/team/device endpoints — confirmed via Datto's published docs, not assumption). The 4 backup-map "held" clients were resolved by direct endpoint verification instead of waiting on that key.
• 20 client wiki articles created (wiki count 56 -> 76) closing the wiki gaps surfaced by gps-master-status.md.
• Onboarded to GuruRMM this week: Janet Altschuler, Marty Ryan, Business Services of Tucson LLC (all real clients with a reachable machine); Andy's Mobile Fuel, Little Hearts Little Hands, Residential and Renovation Engineering (onboarded with no machine yet, specifically to mint site codes for the onsite installer kit). MVAN Enterprises and The Marc Group turned out to already exist from the 2026-07-03 Bucket C onboarding (0 agents, not "no client") — the earlier "no footprint" label meant zero agents, not zero RMM presence.
• GuruRMM confirmed to support Windows, Linux, and macOS agents (/install/<code>/{windows,linux,macos}), resolving the two Mac-only clients (Heieck, Residential and Renovation Engineering) as onsite-installable rather than blocked.
• Onsite installer kit built at C:\Users\Howard\Desktop\CLIENTS\ — one folder per site (20) with Windows/Linux/macOS installer scripts + README, per-location subfolders for the two multi-site clients (Grabb & Durando: Main/Bob's/Jeff's; Glaz-Tech: Involta/SLC/Tucson) so the correct site code reaches the correct location. Howard is taking this onsite to close the remaining no-footprint gaps in person.
• A wrong weekend findings DM to Winter was deleted (2026-07-06) — superseded by the MSP360 + Syncro-seat data that arrived after it was sent. The corrected findings were PATCHed into the existing Mike/Winter/Howard message thread in place rather than sent as new messages, to keep one authoritative thread.
• GPS-RMM-AutoEnroll scheduled task is now DISABLED per Howard — manual sweeps only (bash .claude/scripts/gps-rmm-autoenroll.sh) rather than automatic every 30 minutes. GPS-RMM-Progress (daily 8:07am gap-check DM) remains active.

Tooling

projects/gps-rmm-audit/tools/:

• bucketc-onboard-deploy.sh <"Client"> <bd_company_id> <slug> — onboards a GuruRMM client+site (idempotent), vaults the enrollment key, deploys via SC to every machine in the client's Bitdefender company with a live session. Used for the original Bucket C (16 clients) and later ad hoc onboardings (Janet Altschuler, Marty Ryan, BST, Andy's, Little Hearts, Residential and Renovation Engineering).
• needs-sc.py — reconciles Syncro assets (authoritative inventory) vs live RMM agents per client; writes needs-screenconnect.md; MAXDAYS env filters by Syncro last_synced_at recency. Online-detection superseded by rebuild-and-push.py.
• sc-cleanup.py — fleet-wide SC metadata cleanup via the direct API (~10x faster than the skill's subprocess wrapper on ~500 sessions). Normalizes Company (CP1) to RMM client name, sets Device Type (CP4) from hostname/OS, sets Department (CP3) only on high-confidence hostname tokens; never touches Site. Carries the len(ss)>1: continue guard added after the contamination bug (see Findings).
• rebuild-and-push.py — rebuilds the "not in RMM" list using the correct SC online signal (ActiveConnections ProcessType==2, not stale Syncro last_synced_at), pushes the generic Staging installer (DARK-STORM-3150) to every machine online now. Now run manually via gps-rmm-autoenroll.sh (loop task disabled).
• reassign-staging.py — moves agents out of Staging - Auto Enroll into their real client by hostname -> Syncro business_name -> GuruRMM client (LLC/Inc/Corp-normalized). Idempotent; --dry supported.
• edr-deploy.py — throttled Bitdefender -> Datto EDR deployer: recomputes coverage from a live Datto EDR agent dump each run, targets ONLINE Windows RMM agents of GPS clients with a vaulted registration key not already in EDR, pushes the official Install-EDR one-liner through RMM one machine at a time (20s gap, retry-on-5xx, poll-to-completion). Glaz-Tech excluded by design. Re-run to sweep newly-online machines; idempotent.

.claude/scripts/:

• gps-rmm-autoenroll.sh — closed loop: rebuild-and-push.py -> sleep 150s -> reassign-staging.py -> post to #dev-alerts. Logs to projects/gps-rmm-audit/autoenroll.log. Windows task GPS-RMM-AutoEnroll is DISABLED (Howard, 2026-07-07) — run this script manually instead of on the 30-min schedule. Safe to re-run (server v6.77+ dedups by device_id).
• gps-rmm-progress-check.sh — daily read-only: targets.json vs live /api/agents, DMs Howard the per-client gaps, reports COMPLETE when met. Windows task GPS-RMM-Progress, daily 8:07am, Howard-Home — still active.

Staging site pattern (DARK-STORM-3150): Bucket-C machines are often offline or briefly online, so per-client SC pushes miss the window. A Syncro policy pushes one generic installer to all managed machines; anything that comes online enrolls into catch-all client Staging - Auto Enroll / site Staging (key vaulted at infrastructure/gururmm-staging-site.sops.yaml); reassign-staging.py auto-sorts it to the real client. Proven end-to-end (15 machines enrolled + correctly reassigned in one run, 2026-07-04). The 30-minute scheduled loop was disabled 2026-07-07; the mechanism is run on demand instead.

Reassign / move flow: POST /api/agents/:id/move {site_id} is a clean UPDATE (confirmed not the dup root cause). Throttled 2s + 5x retry to ride out intermittent 500s. Used for Staging reassignment, one-time site corrections (Dataforth D1/D2 split, 39 moves), and per-client reassignment (Janet Altschuler, Marty Ryan, Reliant).

Backup verification tooling (new, Phase 4)

• msp360 skill (.claude/skills/msp360/) — full CRUD against the MSP360 MBS API (https://api.mspbackups.com, Bearer login via msp-tools/msp360-api.sops.yaml). Reads: status, monitoring (--failed/--company/--stale/--json), companies, users, licenses, billing, raw GET. Writes (all --confirm-gated): create/delete company, create/delete user, grant/release/revoke license, raw passthrough. This is the authoritative "is this client actually backing up" source for the whole audit — /api/Monitoring returns per-computer LastStart/Status/DataCopied across 66 plans / 37 companies. /api/Computers returns HTTP 400 (Remote Management API not enabled on the account) — use Monitoring/Companies/Users instead.
• seafile skill — Seafile Pro Web API (admin creds vaulted services/seafile-pro.sops.yaml). Reference implementation for the pattern; usage / audit [--client] [--rmm] [--json].
• owncloud skill — SSH + occ (no OCS web-admin cred exists in the vault, only root SSH to the VM); per-user size read from ownCloud's own MySQL filecache, du fallback for dirty-cache users. Same usage/audit interface.
• datto-workplace skill — HTTP Basic (api-key:api-secret, msp-tools/datto-workplace.sops.yaml); file-access API only, currently blind (0 projects visible) until Mike enables Super Administrator scope on the integration. GuruRMM endpoint-detection (Datto Workplace/Workplace2 patterns — never bare "Datto", which false-matches Datto EDR/RMM/AV) is the practical fallback signal.
• .claude/scripts/backup_common.py — shared plumbing for all four skills: repo-root/ vault resolution, minimal JSON HTTP, and a GuruRMM RmmClient (login, list agents, run a read-only detection PowerShell to confirm a sync client is installed and signed in).
• b2 skill (pre-existing) — Backblaze B2 buckets; bucket-size per-bucket (the full usage scan across all buckets times out at 400s listing every object version).
• Skill routing consolidated (.claude/SKILL_ROUTING.md): a backup request is a two-step route — (1) identify which backend the client actually uses via the client wiki + backup-map.md (never assume B2/MSP360), then (2) route to that backend's skill. Provisioning writes go through the target backend's own skill (all gated).

Key Infrastructure & Integrations

• GuruRMM API — http://172.16.3.30:3001, vault infrastructure/gururmm-server.sops.yaml. Dashboard rmm.azcomputerguru.com; agent WS wss://rmm-api.azcomputerguru.com/ws (resolves direct IP, not Cloudflare). Server at last check: v6.77+ (dedup fix deployed). Install endpoints: https://rmm.azcomputerguru.com/install/<site_code>/<platform>, platform = windows | windows-x86 | windows-legacy | windows-legacy-x86 | linux | macos — confirmed Windows + Linux + macOS agents all exist. Offline-install path: raw binaries at /install/<code>/download/<variant>.
• ScreenConnect — https://computerguru.screenconnect.com, RESTful API Manager ext (Service.ashx). Primary deploy + metadata channel for the audit. CP slots: CP1=Company, CP2=Site, CP3=Department, CP4=Device Type, CP5-7 unused, CP8=Tag. API hides property labels (values only); UpdateSessionCustomProperties replaces the whole 8-element array — always read current values before writing. Extension exposes only GetSessionsByName (no full-fleet listing) — need a hostname to target a machine.
• Syncro assets — authoritative managed-device inventory (more complete than BD/Datto for discovery). Public API has no policy endpoint (GET /policies = 404); SC install policy is console-only. Assets carry policy_folder_id to target it. Recurring schedules are the authoritative "billed backup seat" source — must match on line NAME, not description (base GPS Workstation/Server SKUs carry "Backups" in boilerplate description text, producing false positives if matched loosely).
• MSP360 Managed Backup Service (MBS) — https://api.mspbackups.com, vault msp-tools/msp360-api.sops.yaml. Authoritative "is it backing up" source for the whole fleet (37 companies / 59 users / 66 plans at discovery). msp360 skill.
• Backblaze B2 — 16 buckets, destination-only (not a freshness signal). b2 skill.
• Seafile Pro — Docker on Jupiter, http://172.16.3.20:8082 / https://sync.azcomputerguru.com. 5 active accounts / ~5 TB at discovery (Russo Law 2.2 TB is the only GPS client on it).
• ownCloud — VM 172.16.3.22 (cloud.acghosting.com), Rocky Linux 9.6, ownCloud 10.16 Community. 10 users / 19.2 TB at discovery (pavon 14.2 TB is a video archive, not a backup; GPS clients on it: Mineralogical Record minrec, Business Services of Tucson bst; The Marc Group themarcgroup is non-GPS/email-only).
• Datto Workplace — SaaS acg.workplace.datto.com (team 6) + on-prem Datto Workplace Server VM ACG-DWP-X-BB on Jupiter (confirmed on Birth Biologic via endpoint detection, non-GPS). File-access-only API; full enumeration blocked pending Mike's Super Administrator key.
• Datto EDR — target AV end-state for nearly all clients. Managed via the datto-edr skill (create-group -> mint-key -> deploy). avInstalled reads null right after registration — confirm via org policy, not the raw field. Authorized agent count rose 228 -> 302 during the 2026-07-05 mass-deploy pass.
• Bitdefender GravityZone — retired as a coverage target, kept only as a discovery source. Company names carry the Syncro cid suffix (_<cid>) — exact join key vs targets.json / gps-clients.json.

Findings & Decisions

Machine / AV findings (2026-07-03/05, preserved)

• Billing findings VERIFIED + SENT 2026-07-05 (an early DM to Winter was retracted before the send-when-verified rule was set): Dataforth +9 machines over billed (52 RMM vs 43 billed); Cascades RECEPTIONIST-PC — re-verified as two live machines sharing one hostname (different device_ids, both online simultaneously), not a stale duplicate — needs a rename, not a delete; Len's Auto LAB-SVR offline since 2026-06-18; Jimmy Company (12 billed/1 real) and Stamback Septic (8 billed/4 real, up from 2) — billing-vs-reality mismatch. Sent via Discord DM to Mike, Winter, and Howard 2026-07-05; hold released.
• Glaz-Tech anomaly: 159 GPS billed vs 5 real machines in RMM/Datto EDR, vs 242 Bitdefender records (stale, years of ghosts). RMM+Datto agree on the real footprint; flagged for Mike's billing review, not treated as 154 missing agents. Its 2nd backup seat (GTI-INV-VMHOST) is also 251 days stale — folded into the backup REVIEW verdict.
• AV strategy: Bitdefender -> Datto EDR for all clients. Final exception: Glaztech only (Dataforth was originally excepted too, then fully migrated 2026-07-04 — its final 5 legacy BD endpoints converted during the 2026-07-05 mass deploy, except two genuine legacy-OS blocks below). Target state per machine (non-exempt) = GuruRMM + Datto EDR + Bitdefender removed. Bitdefender removal is not yet started anywhere — deferred per-client, only after that client's EDR is confirmed ACTIVE. Detail: memory project_av_migration_bitdefender_to_edr.
• NO-AV mass deploy (2026-07-05): edr-deploy.py pushed EDR to ~78 machines across ~25 clients in one throttled run (74/79 clean on the first pass). Two genuine legacy-OS blocks remain: AMT-PC (Windows 7 Home SP1, PowerShell 2.0 — modern Infocyte agent crashes on execution) and SRV02 (Curtis Plumbing, Windows Server 2008 R2 — agent launches but no-ops silently). Both need OS prerequisites or an onsite/legacy-agent path. Five apparent "failures" that session were mostly false negatives (Install-EDR is silent in non-interactive mode) — see Patterns below.
• GuruRMM cross-site duplication bug — found, fixed, deployed (v6.77): both agent enroll paths in server/src/ws/mod.rs deduped only by (site_id, device_id). A machine already enrolled at one site that ran a different site's installer minted a duplicate row with the same device_id. Fix: db::get_agent_by_device_id() (global lookup) re-homes the existing record instead of creating a duplicate. Deployed via cherry-pick (hotfix-agent-dedup -> main). Cleanup: 40 stale orphans deleted, 39 one-time moves, 0 failures. Full plan: dedup-plan.md.
• ScreenConnect contamination incident: fleet sc-cleanup.py used GetSessionsByName, which matches by name across ALL clients, not client-scoped. Writing Company to shared names (SERVER, Accounting) cross-contaminated other clients' sessions with the last-processed value. Fixed (if len(ss) > 1: continue); remediated via GuestNetworkAddress (WAN IP) -> client. 5 sessions unrecoverable (3 SERVER + 2 Accounting) — left blank for manual SC-console tagging. Logged as both correction and friction.
• CP-QB (Curtis Plumbing) root cause found and fixed (2026-07-04/05): corrupt WMI class store (Win32_Processor = Invalid class, broken since ~2022) killed the universal installer's architecture check and froze its SC inventory since 2022; winmgmt /verifyrepository reported CONSISTENT (doesn't catch a missing class) — mofcomp cimwin32.mof restored it. Installer then ran clean; enrolled + reassigned.
• IMC-PRINTSERVER installed but TLS-blocked: agent fully installed and running (manual no-PowerShell install over LAN, since PowerShell is dead on the box and SMB hangs), but the box silently drops ALL outbound TLS (even google.com — ClientHello then silence) while ScreenConnect works (non-TLS relay). Escalated to Mike (by-design lockdown vs. fix); no reinstall needed — the agent self-enrolls the instant TLS egress is unblocked.

Backup findings (2026-07-05/07, new this week)

• Only 16 of 40 GPS clients actually bill a backup line — a Syncro sub-agent pull of recurring-schedule Data Backup line-item NAME (not description, to avoid the boilerplate false-positive) corrected the tracker's old Svc=B flags for Horseshoe Management, Quantum Wealth Management, T&C Sorensen, and Business Services of Tucson — none of the four actually bill backup.
• 7 of the 16 billed clients are NOT backing up: Grabb & Durando (GND-SERVER cloud plan failing, 0 bytes — flapping), Bill Tedards (DESKTOP-SUFJR0J last good 359 days ago), Arizona Medical Transit (MSP360 company exists, 0 plans), and 4 endpoint-verified 2026-07-06 with a full on-machine scan (no backup software, service, scheduled task, WSB, or Datto backup client on any machine, and nothing in MSP360/B2/Seafile/ownCloud/Datto Workplace): Reliant Well Drilling, The Prairie Schooner (a Windows Server Backup to USB exists on TPS-SERVER but last ran 3/7/2018, ~8 years dead), Janet Altschuler (only a stopped Dropbox client), Design and Brand Envoys (OSGOOD-PAUL was offline/unscanned).
• 2 REVIEW verdicts — seats billed exceed machines actually backing up: Jimmy Company (12 seats / 1 machine, Blaster2, known 12/1 anomaly) and Glaz-Tech Industries (2 seats / 1 truly current — GTI-INV-VMHOST 251 days stale).
• Revenue leaks (backing up, not billed — held for Mike/Winter, nothing sent): Cascades of Tucson (CS-SERVER daily, B2 ACG-Cascades), Len's Auto Brokerage (LAB-SVR, now stale, B2 ACG-Lens), Russo Law Firm (RUSSO-SRV 2.7 TB cloud + Seafile 2.2 TB), Horseshoe Management (HSM-NewServer daily, B2 Horseshoe), Business Services of Tucson (KAT1 daily, B2 ACG-BST + ownCloud bst).
• $0-priced backup lines (billed a seat at $0 — bundled or under-billed, flag for Mike/Winter): Ridgetop Group, Janet Altschuler, Design and Brand Envoys, Arizona Medical Transit.
• Orphan B2 buckets resolved via MSP360 CompanyName: ACG-PST = Peaceful Spirit (22,054 files, non-GPS), ACG-TCA = Tucson Coin and Autograph (non-GPS) — corrects an earlier tentative guess that ACG-TCA was T&C Sorensen. Other identified orphans: ACG-REDNOUR = Rednour Law, ACG-Rohrbach = Rohrbach, ACG-IX/ACG-Internal/ ACG-Tech03L = ACG internal, MSPBackups20200311 = stale ex-client FSG (deletion candidate). ownCloud mara/jburger still unidentified (verify).
• Datto Workplace research: confirmed there is no admin/team/device REST API at all — the public API is strictly file-sharing; user/storage administration is Workplace-Manager-portal-only. A Super Administrator toggle on the existing API integration (or team-sharing projects to it) is the only way to make /file/projects enumerate the whole team. That toggle is a portal action Claude cannot perform — Mike is provisioning a Super-Administrator-scoped integration; parked until it lands.
• Rule (Howard, standing): backup targets are never guessed. A billed seat count higher than machines-currently-backing-up is a Mike/Winter billing question, not a deploy instruction. Memory: feedback_backup_targets_never_guessed. Corollary added 2026-07-07: only check backup status for clients that actually bill a backup line (e.g. stop checking Marty Ryan, which has none) — memory feedback_backup_check_only_billed.

Patterns & Known Issues

• GetSessionsByName matches across ALL SC clients — never write properties to a name returning >1 session (if len(ss) > 1: continue); root cause of the contamination incident above. The extension also exposes no full-fleet listing — finding a specific client's machines requires already knowing hostnames.
• /api/agents/:id/move intermittently 500s under rapid calls (transient DB/load; handler is a clean UPDATE) — throttle 2s + retry 5x reliably rides over it.
• Syncro last_synced_at goes stale when agent check-in breaks, making an online machine look offline 45+ days. Use SC ActiveConnections ProcessType==2 + GuestInfoUpdateTime instead.
• Datto EDR org agent/site counts are stale rollups after re-parenting Locations — trust a fresh GET /Locations grouped by org, not the org-list summary.
• Install-EDR is silent in non-interactive mode — every status/error line is gated behind if ($Interactive); a "completed / blank output" RMM result means nothing. The visible "Installed RTS agent" text comes from agent.windows.exe itself, so its absence just means the exe never ran (early-return on an existing service, or a crash). Verify with -Interactive *> file + read-back, or check Get-Service huntAgent + the binary path directly. -Force reinstalls over a stale huntAgent service left by a broken partial install.
• RMM command output is in .stdout, not .output, on GET /api/commands/:id.
• Syncro recurring-schedule line matching must use the line NAME, not description — base GPS Workstation/Server SKUs carry the word "Backups" in boilerplate description text, producing false-positive "bills for backup" matches.
• MSP360 bucket presence != a current backup — B2/Seafile/ownCloud show a destination exists; MSP360 /api/Monitoring LastStart+Status is the only real freshness signal. /api/Computers is gated off (HTTP 400, feature not enabled on this account) — use Monitoring/Companies/Users instead.
• Datto backup-detection regex must be precise ("Datto Workplace"/"Workplace2") — a bare "Datto" pattern false-matches Datto EDR/RMM/AV agents that have nothing to do with backup.
• AV coverage splits by client size (Datto for large, Bitdefender historically for small) — check both before declaring a gap.
• SC coverage varies per client — verify a live session exists before assuming the channel works; where absent, discover via Syncro assets instead.
• The client wiki is the primary source of truth for machine -> department/person/ location during SC cleanup, and now also for which backup backend a client uses (per the consolidated wiki-first skill routing) — where missing, learn it and write it back.
• GuruRMM install scripts are bootstrappers, not offline installers — the /install/<code>/<platform> endpoints return a script that downloads the arch-matched binary at run time (needs onsite internet); raw binaries for true offline install live at /install/<code>/download/<variant>.

Operations Playbook

• Daily 8:07am — GPS-RMM-Progress runs gps-rmm-progress-check.sh, DMs Howard the gaps vs targets.json; reports COMPLETE and retires when met.
• Auto-enroll (manual, on demand) — GPS-RMM-AutoEnroll's 30-minute schedule is disabled as of 2026-07-07 (Howard: manual sweeps only). Run bash .claude/scripts/gps-rmm-autoenroll.sh by hand to push Staging to online-but- missing machines, wait, reassign, and post to #dev-alerts.
• Add a client (no-footprint pattern): bucketc-onboard-deploy.sh "<Client>" <bd_company_id> <slug> — creates client+site, vaults the key, deploys to reachable Bitdefender machines. Domain-joined pattern: DC already in RMM -> AD is the authoritative list -> push the site installer to domain members via SC. If no BD/Datto footprint exists at all, onboard anyway (mints the site code) and deploy onsite or via the flash-drive kit.
• Deploy Datto EDR: create-group -> mint-key (via datto-edr skill) -> push Install-EDR through GuruRMM (POST /api/agents/:id/command, visible stdout only with -Interactive) rather than blind SC push. Re-run edr-deploy.py to sweep newly online machines; it recomputes coverage from a live EDR dump each run.
• SC metadata cleanup, per client: wiki first, then hostname role tokens, then UniFi switch/AP naming; run sc-cleanup.py for the safe no-guess pass; never touch Site without a topology source.
• Backup verification, per client: check the client wiki + backup-map.md first for which backend applies (never assume B2/MSP360); confirm with the matching skill (msp360/b2/seafile/owncloud/datto-workplace); only check clients that actually bill a backup line (gps-master-status.md "Billed for backup" table). Endpoint-scan directly (RMM PowerShell) when the destination-side tools are inconclusive or blocked (as done for the 4 Datto-pending clients).
• Onsite visit: use the flash-drive kit at Desktop\CLIENTS\<slug>\ — per-site installer scripts (Windows/Linux/macOS) + README; multi-site clients have per-location subfolders with the correct site code for that location.

History Highlights

• 2026-07-03 — Audit started; 40 active GPS clients identified; tracker.md built (Bucket A 7 / B 8 / C 25); GPS-RMM-Progress registered (baseline 46/189). IMC deploy established the ScreenConnect send-command channel as the working deploy pattern after 4 remote-exec methods failed. Bucket B/C pushed and onboarded (~60 agents).
• 2026-07-04 — Fleet-wide SC cleanup pass (~45 clients); found + remediated the shared-hostname cross-contamination bug. Corrected online-detection logic (SC ActiveConnections vs stale Syncro last_synced_at); Staging auto-enroll pipeline proven end-to-end. GuruRMM dup-agent bug found/fixed/deployed (v6.77); 40 orphans deleted + 39 moves, 0 failures. Phase 4 AV matrix built (9/22 NO-AV); EDR "Default RMM Org" dismantled (18 orgs created, 21 locations re-parented), NO-AV revised to 8 clients/20 devices; round-1 EDR remediation (4/5 active). Session closed at 111/189 enrollment.
• 2026-07-05 — CP-QB fixed (corrupt WMI class store) and enrolled. IMC-PRINTSERVER installed but found TLS-egress-blocked (escalated to Mike). Full Bitdefender -> Datto EDR mass deploy: edr-deploy.py protected ~78 machines across ~25 clients in one throttled run (authorized EDR agents 228 -> 302); 2 genuine legacy-OS blocks identified (AMT-PC Win7/PS2, SRV02 Server 2008 R2). Held billing findings verified and sent to Mike/Winter/Howard via Discord DM. Built the seafile, owncloud, and datto-workplace skills (+ shared backup_common.py) after Howard corrected the assumption that backup destinations were B2-only. Built the msp360 skill (full CRUD) once its API was found to be the authoritative backup-status source; code review (18 agents) fixed 10 findings including a credential-leak in dry-run output. Consolidated backup skill routing behind a wiki-first rule. Ran the first full 40-client backup verification (16 of 40 actually bill backup; 7 OK / 2 REVIEW / 3 confirmed-not-backing-up + 4 held pending Datto).
• 2026-07-06 — Deleted the wrong weekend findings DM to Winter (superseded by the MSP360 + Syncro-seat data). Endpoint-verified the 4 held clients (Reliant, Prairie Schooner, Janet Altschuler pending onboard, Design and Brand Envoys) — all confirmed NOT backing up. backup-map.md fully rewritten with MSP360 as source of truth.
• 2026-07-07 — Onboarded Janet Altschuler and Marty Ryan to GuruRMM (machines located via SC hostnames, agents pushed and enrolled); confirmed Janet's machines already have Datto EDR (left alone). Updated the Mike/Winter Discord messages in place (PATCH) with the endpoint-verified results rather than sending new ones. Built gps-master-status.md as the single current per-client list (superseding repeated reconciliation of 3 stale docs), consolidating GPS device count, RMM enrollment, backup billing + verdict, and wiki presence for all 40 clients. Created the 20 missing client wiki articles via two parallel sub-agents (wiki count 56 -> 76). Onboarded Business Services of Tucson (real machine) and, to generate site codes for the onsite kit, Andy's Mobile Fuel and Little Hearts Little Hands and Residential and Renovation Engineering (no machine yet). Confirmed GuruRMM has Windows, Linux, and macOS agents. Built the flash-drive onsite installer kit at Desktop\CLIENTS\ (20 site folders, per-location subfolders for Grabb & Durando and Glaz-Tech). Disabled the GPS-RMM-AutoEnroll 30-minute schedule (manual sweeps only, per Howard).

Pending / Next Steps

• Howard onsite tour with the Desktop\CLIENTS\ flash-drive kit to close the 7 zero-RMM-footprint clients in person (Andy's Mobile Fuel, Business Services of Tucson — machine KAT1 etc. queued for reconnect, Heieck/Little Hearts/Residential and Renovation Engineering/The Marc Group/MVAN — verify which still need a physical visit vs. a reconnect). Report machine details back into each client wiki + gps-master-status.md as they land.
• Mike/Winter still owe replies (as of 2026-07-07) on: the 4 endpoint-verified not-backing-up clients (Reliant, Prairie Schooner, Janet Altschuler, Design and Brand Envoys — decision needed: bill correctly, deploy real backup, or drop the line), the Datto Workplace Super Administrator API key (current key = 0 projects, blocking full Datto enumeration), and the general backup billing decisions below.
• 7 FAIL + 2 REVIEW backup verdicts need a billing/remediation decision: Grabb & Durando (failing cloud plan), Bill Tedards (359d stale), Arizona Medical Transit (0 plans), Reliant/Prairie Schooner/Janet Altschuler/Design and Brand Envoys (endpoint- confirmed none), Jimmy Company (12/1 seat mismatch), Glaz-Tech (2/1, 2nd server 251d stale). None of these are deploy instructions per the standing rule — Mike/Winter decide bill-vs-deploy-vs-drop.
• Revenue-leak decisions (bill vs. decommission, held, nothing sent): Cascades, Len's, Russo, Horseshoe, Business Services of Tucson; plus the 4 $0-priced lines (Ridgetop, Janet Altschuler, Design and Brand Envoys, AMT).
• Two legacy-OS EDR blocks remain unresolved: AMT-PC (Win7 SP1/PS2) and SRV02 (Server 2008 R2) — need OS prerequisites, the legacy-agent path, or an onsite visit.
• Bitdefender removal not started anywhere — per-client, only after EDR is confirmed ACTIVE for that client; do after the current tail is closed.
• IMC-PRINTSERVER still awaiting Mike's ruling on the TLS-egress lockdown (by-design vs. fix); no reinstall needed once unblocked.
• Manual SC-console cleanup: remove flagged duplicate sessions (Cascades RECEPTIONIST-PC needs a RENAME not a delete — confirmed two live machines sharing one hostname; Dataforth eng-dev-server; Grabb DESKTOP-NFK4F5P; Safesite 0226-Lenovo/MSI); manually tag the 5 unrecoverable blank sessions from the contamination incident.
• Identify remaining unknowns: ownCloud mara/jburger accounts (verify); Mineralogical Record's ownCloud vs. universal-minerals.md wiki article — confirm which is current (verify).
• Continue Phase 4 to email host verification (several "Exchange Hosted Email" lines across Bucket A clients still undocumented) once the backup tail closes.

Backlinks

(link from client wiki articles touched by this audit — Dataforth, Cascades of Tucson, Instrumental Music Center, Arizona Medical Transit, Janet Altschuler, Marty Ryan, Reliant Well Drilling, and the 20 newly created client articles as they come to reference this project)