claudetools/projects/gps-rmm-audit/session-logs/2026-07/2026-07-05-howard-backup-verification-skills.md

GPS Backup Verification — Seafile / ownCloud / Datto Workplace Skills

User

• User: Howard Enos (howard)
• Machine: Howard-Home
• Role: tech

Session Summary

Resumed the GPS -> GuruRMM coverage audit's Phase 4 (backup verification). Howard corrected an earlier assumption: the backup-destination list was B2-only, but many GPS customers back up through other channels — Datto Workplace, ownCloud, or Seafile (the last two run on Jupiter; "SeaCloud"/"SeaDrive" are the Seafile server + its desktop client). Decision was to build one query skill per backend so the audit can gather correct per-customer backup data instead of guessing.

Recovered context from wiki/projects/gps-rmm-audit.md, wiki/systems/jupiter.md, and .claude/memory/feedback_backup_targets_never_guessed.md. Confirmed the access surface for each backend: Seafile Pro has a documented Web API (admin creds vaulted), ownCloud is reachable only via root SSH to the VM (no web/OCS admin cred stored) so occ is the channel, and Datto Workplace had no vault credentials at all. Asked Howard two decisions: how to handle Datto (he supplied API creds mid-session) and what the source of truth should be (he chose "both" — server-side inventory plus GuruRMM endpoint detection).

Probed all three backends live. Seafile Web API authenticated and returned admin user/library data. ownCloud VM was reachable over SSH (paramiko 4.0.0 present). Datto Workplace API authenticated under HTTP Basic (key:secret) but is a file-access API only (projects/files/webhooks, per its OpenAPI spec) with no team-admin endpoints, and /file/projects returns empty for this key. Vaulted the Datto creds (msp-tools/datto-workplace.sops.yaml) via the vault skill.

Built a shared library .claude/scripts/backup_common.py (repo-root/vault resolution, minimal JSON HTTP, and a GuruRMM RmmClient that logs in, lists agents, and runs a read-only detection PowerShell to confirm a sync client is installed and signed in). Built and live-verified the seafile skill as the reference implementation, then delegated owncloud and datto-workplace to two parallel sub-agents using seafile + backup_common as the template. Independently verified all three against live data.

Findings overturn the B2-only picture: Seafile holds 5 active accounts / ~5 TB (Russo Law 2.2 TB, Stoltz Law, Sonoran Glass, + ACG internal); ownCloud holds 10 active users / 19.2 TB (Pavon 14.2 TB, Rohrbach, BST, Martell, MinRec, The Marc Group, and others); Datto Workplace is confirmed on Birth Biologic via endpoint detection. BST appears in both ownCloud and a B2 bucket, showing the cross-backend overlap the final map must reconcile.

Key Decisions

• Source of truth = "both": server-side inventory (who has an account + how much data) plus GuruRMM endpoint detection (which enrolled machine actually runs the client). Chosen by Howard.
• Three skills, not four: "SeaCloud" and "SeaDrive" are the same Seafile system, so one seafile skill covers them; b2 already covers Backblaze.
• Factored shared plumbing into .claude/scripts/backup_common.py rather than duplicating the GuruRMM client + vault reads across three skills (used identically by all three). Deviates from b2's fully-standalone style, justified by 3x reuse.
• ownCloud channel = SSH + occ (via paramiko), not the OCS HTTP API — the vault has only root SSH, no ownCloud web-admin account. Per-user "used bytes" is not exposed by occ on 10.16, so the sub-agent reads ownCloud's own filecache sizes from MySQL (config.php creds read live over SSH), with a du fallback for dirty-cache users.
• Datto Workplace: RMM endpoint detection is the primary source (the file API is scoped to what the key can see and returns no projects). Detection patterns MUST be ["Datto Workplace","Workplace2"], never bare "Datto" (false-matches Datto EDR/ RMM/AV — verified).
• Built seafile myself + verified, then delegated owncloud/datto to parallel sub-agents (independent, multi-file, parallelizable) with the verified skill as the template to keep the three consistent.

Problems Encountered

• Bare "Datto" detection pattern false-matched "Datto EDR Agent" on a test machine. Fixed by requiring precise patterns (Datto Workplace/Workplace2); confirmed live: ACG-DWP-X-BB detected=True (installed "Datto Workplace Server"), an EDR-only box detected=False.
• /tmp write blocked by the repo's block-tmp-path pre-hook. Switched probe temp files to repo-relative paths (./.dw_probe.json).
• Datto Workplace endpoint names unknown — guessed resource names (team/users/etc) all 404'd. Resolved by pulling the OpenAPI spec at /api/v1/openapi.json, which revealed it is a /file/* + /webhook/* API only.
• ownCloud usage is slow (SSH + per-user work); the verification command auto-backgrounded. Cause is the du fallback path for dirty-cache users; the filecache MySQL query is the fast primary. Completed fine (10 users / 19.2 TB).
• TaskCreate tool schema not loaded (wrong param shape); skipped the task tracker rather than fight it.

Configuration Changes

Created:

• .claude/scripts/backup_common.py — shared: root/vault resolve, JSON HTTP, GuruRMM RmmClient + endpoint detection PowerShell.
• .claude/skills/seafile/SKILL.md, scripts/seafile.py, scripts/seafile_client.py, .gitignore
• .claude/skills/owncloud/SKILL.md, scripts/owncloud.py, scripts/owncloud_client.py, .gitignore (built by sub-agent)
• .claude/skills/datto-workplace/SKILL.md, scripts/datto_workplace.py, scripts/datto_client.py, .gitignore (built by sub-agent)

Vault (separate repo, stored + verified, NOT yet pushed):

• msp-tools/datto-workplace.sops.yaml

Credentials & Secrets

• Datto Workplace API — vaulted at msp-tools/datto-workplace.sops.yaml.
  • Base: https://acg.workplace.datto.com/api/v1 (team path /6/api/v1 normalized to /api/v1).
  • Auth: HTTP Basic, username = api-key, password = api-secret.
  • api-key 6d1dbfa4-5652-4ae3-94ff-fdbbe312c36b, api-secret ed2cd873-5a49-4271-8043-a2d2e56e99f8.
  • File-access API only; /file/projects returns empty for this key. OpenAPI at /api/v1/openapi.json.
• Seafile Pro admin — existing vault services/seafile-pro.sops.yaml (credentials.username / credentials.password; also credentials.database.*).
• ownCloud VM — existing vault infrastructure/owncloud-vm.sops.yaml (credentials.username=root / credentials.password).
• GuruRMM API — existing vault infrastructure/gururmm-server.sops.yaml (credentials.gururmm-api.admin-email / admin-password).

Infrastructure & Servers

• Seafile Pro — Docker on Jupiter, http://172.16.3.20:8082 (direct) / https://sync.azcomputerguru.com (NPM). Version 12.0.19, edition pro. 11.8 TB storage.
• ownCloud VM — 172.16.3.22 (hostname cloud.acghosting.com), Rocky Linux 9.6, ownCloud 10.16.0 Community, install dir /var/www/owncloud, web user apache, occ = sudo -u apache php /var/www/owncloud/occ.
• Datto Workplace — SaaS acg.workplace.datto.com (team 6). On-prem "Datto Workplace Server" VM ACG-DWP-X-BB on Jupiter (Birth Biologic).
• GuruRMM API — http://172.16.3.30:3001 (356 agents, 235 online at session time).

Commands & Outputs

# Seafile skill
py .claude/skills/seafile/scripts/seafile.py usage
py .claude/skills/seafile/scripts/seafile.py audit [--client NAME] [--rmm] [--json]

# ownCloud skill
py .claude/skills/owncloud/scripts/owncloud.py usage
py .claude/skills/owncloud/scripts/owncloud.py audit [--client NAME] [--rmm] [--json]

# Datto Workplace skill
py .claude/skills/datto-workplace/scripts/datto_workplace.py status
py .claude/skills/datto-workplace/scripts/datto_workplace.py audit --client Birth --rmm

Live results:

• Seafile: 5 active accounts / ~5.02 TB (russo@rrs-law.com 2204.8 GB; mike 1469 GB; greg@gstoltzlaw.com 733 GB; debra@sonoranglass.org 448 GB; howard 165 GB).
• ownCloud: 10 users / 19189.2 GB (pavon 14187 GB; sysadmin 2893 GB; anaise 605 GB; rohrbach 603 GB; mara 308 GB; bst 253 GB; Martell 186 GB; minrec 98 GB; themarcgroup 29 GB; jburger 28 GB).
• Datto detection precision: ACG-DWP-X-BB (BirthBiologic) detected=True installed=['Datto Workplace Server']; DESKTOP-QNP3ON5 (EDR-only) detected=False.

Pending / Incomplete Tasks

• Paused per Howard: "we will come back to this."
• Commit/publish: new skills + backup_common.py are staged in the working tree (uncommitted); Datto vault entry stored+verified locally but NOT pushed. Run /scc or /sync to publish (sync.sh Phase 6 pushes the vault repo).
• Next step (Howard's call): produce the corrected GPS per-customer backup map — cross-reference all four backends (B2 + Seafile + ownCloud + Datto Workplace) against the 40 active GPS clients + targets.json, resolving every billed backup line to a real destination. Directly answers held findings (e.g. "AMT billed backup, no B2 destination found" — check Seafile/ownCloud/Datto).
• Datto audit --rmm is per-endpoint (slow); always scope with --client.
• Optional: reconcile cross-backend overlap (e.g. BST in both ownCloud and B2 bucket ACG-BST).

Reference Information

• Datto Workplace OpenAPI: GET https://acg.workplace.datto.com/api/v1/openapi.json (Basic auth).
• Datto WP endpoints: /file/projects, /file/{parentID}/files, /file/{fileID}, /file/{fileID}/data, /file/search, /webhook*.
• Seafile admin API: /api2/auth-token/ (form) -> token; /api/v2.1/admin/users/, /api/v2.1/admin/libraries/, /api/v2.1/admin/devices/.
• ownCloud: occ user:list --show-all-attributes --output=json; per-user size from oc_storages/oc_filecache (home::, path='').
• Detection engine: .claude/scripts/backup_common.py RmmClient.detect_client(agent, patterns).
• Related memory: feedback_backup_targets_never_guessed (billing mismatch is a Winter/Mike question, not a deploy instruction).