claudetools/projects/gps-rmm-audit/session-logs/2026-07/2026-07-05-howard-gps-backup-map-msp360.md

GPS Backup Verification — Datto API Path + MSP360 Backup Map

User

• User: Howard Enos (howard)
• Machine: Howard-Home
• Role: tech

Session Summary

Resumed GPS -> GuruRMM audit Phase 4 (backup verification), picking up from the morning's backup-verification-skills session. Two threads: (1) determine whether the Datto Workplace API can be pushed past its file-only limitation, and (2) produce the corrected per-customer GPS backup map across all backends.

Confirmed the Datto Workplace file API is still a dead end for backup enumeration (0 projects visible to the current key). Researched Datto's actual API surface: there is no admin/team/user/device REST API — the public API is strictly file-sharing, and user/storage administration is Workplace-Manager-portal-only. However, the file API DOES have a "broader access" mode: enabling Super Administrator access on the API integration (or team-sharing projects to it) makes /file/projects enumerate the whole team. That is a portal action on the SaaS tenant, which Claude cannot perform. Howard chose that path; Mike is provisioning a Super-Administrator-scoped API integration. Datto enumeration is parked on that.

With Datto blocked, moved to building the backup map. Pulled the billed-backup client list from tracker.md (20 of 40 active GPS clients bill a Data Backup line), the live B2 bucket inventory (16 buckets), and the prior session's Seafile (5 accounts) and ownCloud (10 users) data. First pass resolved 7 clients cleanly against B2/ownCloud, left 1 as a name-guess (T&C Sorensen -> ACG-TCA), and 12 with no destination found. Identified the orphan destinations via the IX-server cPanel account map + client wiki: pavon (14.2 TB, the largest consumer) is a video-surveillance archive, not a backup; most others are non-GPS clients or ACG internal. Confirmed all 7 resolved billed buckets are non-empty (real backups), and flagged ACG-PST as a substantial 22K-file bucket with no known owner.

The breakthrough was finding the MSP360 Managed Backup Service API credentials in the vault (msp-tools/msp360-api.sops.yaml) and querying /api/Monitoring directly (no skill exists). This is the authoritative backup layer: 66 backup plans across 37 companies, each with computer, last-run, status, and data size. It collapsed the 12 unresolved clients to 5 truly missing (Datto candidates), confirmed 11 actively backing up, and surfaced 4 not-protected findings (AMT + T&C Sorensen have zero backup plans; Grabb GND-SERVER cloud plan failing; Bill Tedards ~12 months stale). MSP360 CompanyName also resolved the two orphan-bucket mysteries — ACG-PST = Peaceful Spirit, ACG-TCA = Tucson Coin and Autograph — which corrected an earlier tentative guess that ACG-TCA was T&C Sorensen (it is not; Sorensen is not backing up). Rewrote backup-map.md with MSP360 as the source of truth and saved a reference memory so future backup audits start there.

Key Decisions

• Pursue "broader Datto API access" via the existing integration's Super Administrator toggle rather than assuming a separate admin API exists — research confirmed no admin API exists, so the toggle (or project team-shares) is the only enumeration path.
• Treat MSP360 /api/Monitoring as the authoritative "is X backing up" source, not B2 bucket contents — a bucket with files does not prove a recent backup ran. B2/Seafile/ ownCloud are destinations; MSP360 LastStart+Status is the real signal.
• Queried MSP360 via direct API (read-only) since no skill covers it; flagged building a small read-only msp360 skill as a follow-up rather than doing it inline.
• Reported AMT and T&C Sorensen "zero backup plans" as findings routed to Mike/Winter/ client, not as a deploy instruction — backup targets are a per-client decision (feedback_backup_targets_never_guessed).
• Corrected the ACG-TCA -> T&C Sorensen guess once MSP360 proved ACG-TCA = Tucson Coin; did not ship the guess.

Problems Encountered

• Datto Workplace OpenAPI spec served is stripped (info: {}, no securitySchemes) — used Datto's published help docs to confirm the API is file-sharing-only and how the Super-Admin scope works.
• b2 usage (full per-bucket byte-size across 16 buckets) timed out at 400s (exit 124) — it lists every object version. Switched to per-bucket bucket-size with a 40s timeout; empty buckets return instantly, data buckets are the slow ones, which itself answers the empty-vs-real question. All 7 resolved billed buckets confirmed non-empty.
• MSP360 /api/Computers returns 404; used /api/Companies + /api/Users + /api/Monitoring for the full picture instead.

Configuration Changes

Created:

• projects/gps-rmm-audit/backup-map.md — the per-customer backup destination + health map (MSP360-authoritative; B2/Seafile/ownCloud destinations; Datto pending).
• .claude/memory/reference_msp360_backup_monitoring.md + index line in .claude/memory/MEMORY.md.

Scratch (not committed): MSP360 probe scripts in the session scratchpad (msp360_probe.py, msp360_report.py).

Credentials & Secrets

• MSP360 Managed Backup Service API — vault msp-tools/msp360-api.sops.yaml (already present, created by Mike 2026-05-18). Base https://api.mspbackups.com; auth POST /api/Provider/Login {UserName, Password} -> Bearer access_token; login kY9PvDdWki (password in vault). Endpoints used: /api/Monitoring, /api/Companies, /api/Users.
• Datto Workplace API — vault msp-tools/datto-workplace.sops.yaml (from morning session); file-access only, /file/projects empty until Super-Admin scope enabled.

Infrastructure & Servers

• MSP360 MBS — SaaS api.mspbackups.com. 37 companies / 59 users / 66 backup plans. Status codes: 0/1=Success, 2=Warning, 3=Failed, 4=Running, 6=Interrupted, 7=completed-with-warnings.
• Backblaze B2 — 16 buckets. Billed-GPS destinations: ACG-Dataforth, VWP-Backup, ACG-GLAZTECH, Horseshoe, ACG-Brett, ACG-BST. Orphans identified: ACG-PST=Peaceful Spirit (22,054 files), ACG-TCA=Tucson Coin and Autograph (404 files), ACG-REDNOUR=Rednour Law, ACG-Rohrbach=Rohrbach, ACG-IX/ACG-Internal/ACG-Tech03L=ACG internal, MSPBackups20200311=stale ex-client FSG.
• Datto Workplace — SaaS acg.workplace.datto.com (team 6); on-prem Datto Workplace Server VM ACG-DWP-X-BB on Jupiter (Birth Biologic).

Commands & Outputs

# Datto Workplace status (still 0 projects for current key)
py .claude/skills/datto-workplace/scripts/datto_workplace.py status

# B2 buckets + per-bucket size (usage full-scan times out; use bucket-size per bucket)
py .claude/skills/b2/scripts/b2.py buckets
py .claude/skills/b2/scripts/b2.py bucket-size ACG-PST   # 22,054 files

# MSP360 (direct API, read-only) — authoritative backup status
#   POST /api/Provider/Login {UserName,Password} -> Bearer token
#   GET  /api/Monitoring  -> 66 plans, per computer: LastStart/Status/DataCopied
#   GET  /api/Companies   -> 37 ; GET /api/Users -> 59

Key MSP360 findings: Arizona Medical Transit + T&C Sorensen = MSP360 company but 0 plans (not backing up); Grabb & Durando GND-SERVER Cloud plan Status=Failed 0 bytes; Bill Tedards DESKTOP-SUFJR0J last Success 2025-07-12 (~12 mo); Glaz-Tech GTI-INV-VMHOST last 2025-10-26; Len's LAB-SVR last 2026-06-17 (matches offline-since-2026-06-18 tracker note).

Pending / Incomplete Tasks

• Datto (blocked on Mike): enumerate once Super-Admin API integration is live, then check the 5 MISSING billed-backup clients against it: Quantum Wealth, Reliant Well Drilling, The Prairie Schooner, Janet Altschuler, Design & Brand Envoys.
• Backup-health findings to route (Mike/Winter/client): AMT + T&C Sorensen have no backup plans; Grabb GND-SERVER failing; Bill Tedards ~12 mo stale; Glaz-Tech GTI-INV-VMHOST ~8 mo stale; Len's LAB-SVR stale.
• Held revenue-leaks (Mike/Winter): Cascades (ACG-Cascades active, no B line), Len's (ACG-Lens, no B line), Russo (2.7 TB Cloud + 2.2 TB Seafile, no B line), Marc Group (ownCloud, billed email only).
• Cleanup: confirm MSPBackups20200311 (FSG) for deletion; identify ownCloud mara/jburger.
• Follow-up build: read-only msp360 skill (login + Monitoring) for repeatable fleet backup-status checks.
• Prior-session publish still applies: backup skills + backup_common.py uncommitted; Datto vault entry stored but not pushed.

Reference Information

• Backup map: projects/gps-rmm-audit/backup-map.md
• Memory: .claude/memory/reference_msp360_backup_monitoring.md, .claude/memory/feedback_backup_targets_never_guessed.md
• MSP360 API vault: msp-tools/msp360-api.sops.yaml; base https://api.mspbackups.com
• Datto WP API vault: msp-tools/datto-workplace.sops.yaml; OpenAPI https://acg.workplace.datto.com/api/v1/openapi.json
• Datto WP API scope docs: help.workplace.datto.com API Integrations (Super Administrator access enables full-team project visibility on the file API).

Update: 21:23 PT — built the msp360 skill (full CRUD) + consolidated backup routing

Built the msp360 skill (Howard asked for full management, not read-only), then reviewed, fixed, and published it; consolidated all backup-related skill routing behind a wiki-first rule.

MSP360 API surface — confirmed authoritative endpoint list against help.mspbackups.com/mbs-api-specification/methods and live Allow-header probing: full CRUD on Users, Companies, Licenses (Grant/Release/Revoke), Destinations, Accounts, Packages, Administrators; read-only Monitoring/Billing. /api/Computers returns HTTP 400 "Remote Management API methods are not enabled" (feature off on this account). Pulled live User/Company/License schemas so write payloads use real field names, not guesses.

Skill built (.claude/skills/msp360/): msp360_client.py (Bearer login + typed helpers, reuses backup_common.py), msp360.py (CLI), SKILL.md, .gitignore. Reads: status, monitoring (--failed/--company/--stale/--json), companies, users, licenses, billing, raw GET. Writes (all --confirm-gated, dry-run preview otherwise): create/delete company, create/delete user (--keep-data), grant/release/revoke license, raw passthrough. Errors -> errorlog.md; expected conditions (feature-gate, no-match) not.

Code review (workflow, high effort, 18 agents): 10 findings, all fixed + re-verified. Notable: (1) CREDENTIAL LEAK — create-user dry-run echoed plaintext --password; now redacted. (2) main() caught only Msp360Error, not parent bc.BackupError, so vault/HTTP failures were mislabeled "unexpected"; now catches bc.BackupError. (3) non-list 2xx body crashed reads; added _as_list guard. (4) licenses Number format crash on None; coalesced. (5) monitoring --failed hid never-run/Unknown plans (the exact dead-plan case); now included. Plus stale-date parse robustness, feature-gate substring broadening, dead --json on billing, dead update_* client methods, and _root() now uses bc.resolve_root(). Refuted 1 (SKILL.md does not document passwords on CLI).

Dry-test sweep: 14/14 reads OK; 8/8 writes correctly gated (nothing executed without --confirm); password-redaction confirmed.

Routing consolidation (.claude/SKILL_ROUTING.md): seafile + owncloud had each written their own provisioning rows. Collapsed all five backup rows (msp360/b2/seafile/owncloud/ datto) into ONE pointer row + a dedicated "Backups (wiki-first routing)" block: a backup request is a two-step route — (1) identify which backend THIS customer uses via the client wiki + backup-map.md (don't assume B2/MSP360), then (2) route to that backend's skill; provisioning uses the target backend's skill (all writes gated). Encodes both behaviors Howard asked for (check-the-wiki-for-status, and use-the-skills-to-set-up).

Published: commits 976bfe7 (skill) + a8e76ba (routing/memory) pushed to origin; Phase 5c installed msp360 to global. Memory reference_msp360_backup_monitoring updated (skill now exists, full not read-only).