sync: auto-sync from HOWARD-HOME at 2026-07-05 15:29:59
Author: Howard Enos Machine: HOWARD-HOME Timestamp: 2026-07-05 15:29:59
This commit is contained in:
@@ -0,0 +1,159 @@
|
||||
# GPS Backup Verification — Seafile / ownCloud / Datto Workplace Skills
|
||||
|
||||
## User
|
||||
- **User:** Howard Enos (howard)
|
||||
- **Machine:** Howard-Home
|
||||
- **Role:** tech
|
||||
|
||||
## Session Summary
|
||||
|
||||
Resumed the GPS -> GuruRMM coverage audit's Phase 4 (backup verification). Howard
|
||||
corrected an earlier assumption: the backup-destination list was B2-only, but many
|
||||
GPS customers back up through other channels — Datto Workplace, ownCloud, or Seafile
|
||||
(the last two run on Jupiter; "SeaCloud"/"SeaDrive" are the Seafile server + its
|
||||
desktop client). Decision was to build one query skill per backend so the audit can
|
||||
gather correct per-customer backup data instead of guessing.
|
||||
|
||||
Recovered context from `wiki/projects/gps-rmm-audit.md`, `wiki/systems/jupiter.md`,
|
||||
and `.claude/memory/feedback_backup_targets_never_guessed.md`. Confirmed the access
|
||||
surface for each backend: Seafile Pro has a documented Web API (admin creds vaulted),
|
||||
ownCloud is reachable only via root SSH to the VM (no web/OCS admin cred stored) so
|
||||
`occ` is the channel, and Datto Workplace had no vault credentials at all. Asked
|
||||
Howard two decisions: how to handle Datto (he supplied API creds mid-session) and
|
||||
what the source of truth should be (he chose "both" — server-side inventory plus
|
||||
GuruRMM endpoint detection).
|
||||
|
||||
Probed all three backends live. Seafile Web API authenticated and returned admin
|
||||
user/library data. ownCloud VM was reachable over SSH (paramiko 4.0.0 present).
|
||||
Datto Workplace API authenticated under HTTP Basic (key:secret) but is a file-access
|
||||
API only (projects/files/webhooks, per its OpenAPI spec) with no team-admin
|
||||
endpoints, and `/file/projects` returns empty for this key. Vaulted the Datto creds
|
||||
(`msp-tools/datto-workplace.sops.yaml`) via the vault skill.
|
||||
|
||||
Built a shared library `.claude/scripts/backup_common.py` (repo-root/vault
|
||||
resolution, minimal JSON HTTP, and a GuruRMM `RmmClient` that logs in, lists agents,
|
||||
and runs a read-only detection PowerShell to confirm a sync client is installed and
|
||||
signed in). Built and live-verified the `seafile` skill as the reference
|
||||
implementation, then delegated `owncloud` and `datto-workplace` to two parallel
|
||||
sub-agents using seafile + backup_common as the template. Independently verified all
|
||||
three against live data.
|
||||
|
||||
Findings overturn the B2-only picture: Seafile holds 5 active accounts / ~5 TB
|
||||
(Russo Law 2.2 TB, Stoltz Law, Sonoran Glass, + ACG internal); ownCloud holds 10
|
||||
active users / 19.2 TB (Pavon 14.2 TB, Rohrbach, BST, Martell, MinRec, The Marc
|
||||
Group, and others); Datto Workplace is confirmed on Birth Biologic via endpoint
|
||||
detection. BST appears in both ownCloud and a B2 bucket, showing the cross-backend
|
||||
overlap the final map must reconcile.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- Source of truth = "both": server-side inventory (who has an account + how much
|
||||
data) plus GuruRMM endpoint detection (which enrolled machine actually runs the
|
||||
client). Chosen by Howard.
|
||||
- Three skills, not four: "SeaCloud" and "SeaDrive" are the same Seafile system, so
|
||||
one `seafile` skill covers them; `b2` already covers Backblaze.
|
||||
- Factored shared plumbing into `.claude/scripts/backup_common.py` rather than
|
||||
duplicating the GuruRMM client + vault reads across three skills (used identically
|
||||
by all three). Deviates from b2's fully-standalone style, justified by 3x reuse.
|
||||
- ownCloud channel = SSH + `occ` (via paramiko), not the OCS HTTP API — the vault
|
||||
has only root SSH, no ownCloud web-admin account. Per-user "used bytes" is not
|
||||
exposed by occ on 10.16, so the sub-agent reads ownCloud's own filecache sizes from
|
||||
MySQL (config.php creds read live over SSH), with a `du` fallback for dirty-cache
|
||||
users.
|
||||
- Datto Workplace: RMM endpoint detection is the primary source (the file API is
|
||||
scoped to what the key can see and returns no projects). Detection patterns MUST be
|
||||
`["Datto Workplace","Workplace2"]`, never bare `"Datto"` (false-matches Datto EDR/
|
||||
RMM/AV — verified).
|
||||
- Built seafile myself + verified, then delegated owncloud/datto to parallel
|
||||
sub-agents (independent, multi-file, parallelizable) with the verified skill as the
|
||||
template to keep the three consistent.
|
||||
|
||||
## Problems Encountered
|
||||
|
||||
- Bare `"Datto"` detection pattern false-matched "Datto EDR Agent" on a test machine.
|
||||
Fixed by requiring precise patterns (`Datto Workplace`/`Workplace2`); confirmed
|
||||
live: ACG-DWP-X-BB detected=True (installed "Datto Workplace Server"), an EDR-only
|
||||
box detected=False.
|
||||
- `/tmp` write blocked by the repo's block-tmp-path pre-hook. Switched probe temp
|
||||
files to repo-relative paths (`./.dw_probe.json`).
|
||||
- Datto Workplace endpoint names unknown — guessed resource names (team/users/etc)
|
||||
all 404'd. Resolved by pulling the OpenAPI spec at `/api/v1/openapi.json`, which
|
||||
revealed it is a `/file/*` + `/webhook/*` API only.
|
||||
- ownCloud `usage` is slow (SSH + per-user work); the verification command
|
||||
auto-backgrounded. Cause is the `du` fallback path for dirty-cache users; the
|
||||
filecache MySQL query is the fast primary. Completed fine (10 users / 19.2 TB).
|
||||
- TaskCreate tool schema not loaded (wrong param shape); skipped the task tracker
|
||||
rather than fight it.
|
||||
|
||||
## Configuration Changes
|
||||
|
||||
Created:
|
||||
- `.claude/scripts/backup_common.py` — shared: root/vault resolve, JSON HTTP, GuruRMM RmmClient + endpoint detection PowerShell.
|
||||
- `.claude/skills/seafile/SKILL.md`, `scripts/seafile.py`, `scripts/seafile_client.py`, `.gitignore`
|
||||
- `.claude/skills/owncloud/SKILL.md`, `scripts/owncloud.py`, `scripts/owncloud_client.py`, `.gitignore` (built by sub-agent)
|
||||
- `.claude/skills/datto-workplace/SKILL.md`, `scripts/datto_workplace.py`, `scripts/datto_client.py`, `.gitignore` (built by sub-agent)
|
||||
|
||||
Vault (separate repo, stored + verified, NOT yet pushed):
|
||||
- `msp-tools/datto-workplace.sops.yaml`
|
||||
|
||||
## Credentials & Secrets
|
||||
|
||||
- **Datto Workplace API** — vaulted at `msp-tools/datto-workplace.sops.yaml`.
|
||||
- Base: `https://acg.workplace.datto.com/api/v1` (team path `/6/api/v1` normalized to `/api/v1`).
|
||||
- Auth: HTTP Basic, username = api-key, password = api-secret.
|
||||
- api-key `6d1dbfa4-5652-4ae3-94ff-fdbbe312c36b`, api-secret `ed2cd873-5a49-4271-8043-a2d2e56e99f8`.
|
||||
- File-access API only; `/file/projects` returns empty for this key. OpenAPI at `/api/v1/openapi.json`.
|
||||
- Seafile Pro admin — existing vault `services/seafile-pro.sops.yaml` (credentials.username / credentials.password; also credentials.database.*).
|
||||
- ownCloud VM — existing vault `infrastructure/owncloud-vm.sops.yaml` (credentials.username=root / credentials.password).
|
||||
- GuruRMM API — existing vault `infrastructure/gururmm-server.sops.yaml` (credentials.gururmm-api.admin-email / admin-password).
|
||||
|
||||
## Infrastructure & Servers
|
||||
|
||||
- **Seafile Pro** — Docker on Jupiter, `http://172.16.3.20:8082` (direct) / `https://sync.azcomputerguru.com` (NPM). Version 12.0.19, edition pro. 11.8 TB storage.
|
||||
- **ownCloud VM** — `172.16.3.22` (hostname cloud.acghosting.com), Rocky Linux 9.6, ownCloud 10.16.0 Community, install dir `/var/www/owncloud`, web user `apache`, `occ` = `sudo -u apache php /var/www/owncloud/occ`.
|
||||
- **Datto Workplace** — SaaS `acg.workplace.datto.com` (team 6). On-prem "Datto Workplace Server" VM `ACG-DWP-X-BB` on Jupiter (Birth Biologic).
|
||||
- **GuruRMM API** — `http://172.16.3.30:3001` (356 agents, 235 online at session time).
|
||||
|
||||
## Commands & Outputs
|
||||
|
||||
```bash
|
||||
# Seafile skill
|
||||
py .claude/skills/seafile/scripts/seafile.py usage
|
||||
py .claude/skills/seafile/scripts/seafile.py audit [--client NAME] [--rmm] [--json]
|
||||
|
||||
# ownCloud skill
|
||||
py .claude/skills/owncloud/scripts/owncloud.py usage
|
||||
py .claude/skills/owncloud/scripts/owncloud.py audit [--client NAME] [--rmm] [--json]
|
||||
|
||||
# Datto Workplace skill
|
||||
py .claude/skills/datto-workplace/scripts/datto_workplace.py status
|
||||
py .claude/skills/datto-workplace/scripts/datto_workplace.py audit --client Birth --rmm
|
||||
```
|
||||
|
||||
Live results:
|
||||
- Seafile: 5 active accounts / ~5.02 TB (russo@rrs-law.com 2204.8 GB; mike 1469 GB; greg@gstoltzlaw.com 733 GB; debra@sonoranglass.org 448 GB; howard 165 GB).
|
||||
- ownCloud: 10 users / 19189.2 GB (pavon 14187 GB; sysadmin 2893 GB; anaise 605 GB; rohrbach 603 GB; mara 308 GB; bst 253 GB; Martell 186 GB; minrec 98 GB; themarcgroup 29 GB; jburger 28 GB).
|
||||
- Datto detection precision: ACG-DWP-X-BB (BirthBiologic) detected=True installed=['Datto Workplace Server']; DESKTOP-QNP3ON5 (EDR-only) detected=False.
|
||||
|
||||
## Pending / Incomplete Tasks
|
||||
|
||||
- Paused per Howard: "we will come back to this."
|
||||
- Commit/publish: new skills + backup_common.py are staged in the working tree
|
||||
(uncommitted); Datto vault entry stored+verified locally but NOT pushed. Run
|
||||
`/scc` or `/sync` to publish (sync.sh Phase 6 pushes the vault repo).
|
||||
- Next step (Howard's call): produce the corrected GPS per-customer backup map —
|
||||
cross-reference all four backends (B2 + Seafile + ownCloud + Datto Workplace)
|
||||
against the 40 active GPS clients + `targets.json`, resolving every billed backup
|
||||
line to a real destination. Directly answers held findings (e.g. "AMT billed
|
||||
backup, no B2 destination found" — check Seafile/ownCloud/Datto).
|
||||
- Datto `audit --rmm` is per-endpoint (slow); always scope with `--client`.
|
||||
- Optional: reconcile cross-backend overlap (e.g. BST in both ownCloud and B2 bucket ACG-BST).
|
||||
|
||||
## Reference Information
|
||||
|
||||
- Datto Workplace OpenAPI: `GET https://acg.workplace.datto.com/api/v1/openapi.json` (Basic auth).
|
||||
- Datto WP endpoints: `/file/projects`, `/file/{parentID}/files`, `/file/{fileID}`, `/file/{fileID}/data`, `/file/search`, `/webhook*`.
|
||||
- Seafile admin API: `/api2/auth-token/` (form) -> token; `/api/v2.1/admin/users/`, `/api/v2.1/admin/libraries/`, `/api/v2.1/admin/devices/`.
|
||||
- ownCloud: `occ user:list --show-all-attributes --output=json`; per-user size from `oc_storages`/`oc_filecache` (home::<uid>, path='').
|
||||
- Detection engine: `.claude/scripts/backup_common.py` `RmmClient.detect_client(agent, patterns)`.
|
||||
- Related memory: `feedback_backup_targets_never_guessed` (billing mismatch is a Winter/Mike question, not a deploy instruction).
|
||||
Reference in New Issue
Block a user