From 19b826306adb4818b8cc3cd0fca0d5eb9887abc0 Mon Sep 17 00:00:00 2001 From: Mike Swanson Date: Mon, 1 Jun 2026 16:49:10 -0700 Subject: [PATCH] sync: auto-sync from GURU-5070 at 2026-06-01 16:49:05 Author: Mike Swanson Machine: GURU-5070 Timestamp: 2026-06-01 16:49:05 --- .../2026-06-01-mike-b2-and-memory-tooling.md | 96 +++++++++++++++++++ 1 file changed, 96 insertions(+) create mode 100644 session-logs/2026-06-01-mike-b2-and-memory-tooling.md diff --git a/session-logs/2026-06-01-mike-b2-and-memory-tooling.md b/session-logs/2026-06-01-mike-b2-and-memory-tooling.md new file mode 100644 index 0000000..9c8a3d4 --- /dev/null +++ b/session-logs/2026-06-01-mike-b2-and-memory-tooling.md @@ -0,0 +1,96 @@ +# Session Log — 2026-06-01 — B2 skill, Tier-A storage purge, memory tooling, sync optimization + +## User +- **User:** Mike Swanson (mike) +- **Machine:** GURU-5070 +- **Role:** admin + +## Session Summary + +Started with a client-directed M365 password set for carla@rednourlaw.com (Rednour Law) via the remediation-tool / User Manager app — set to `Fiduciary1234!`, no force-change, no session revocation. Logged as a client-directed change (not breach remediation), recorded in `clients/rednour/reports/2026-06-01-carla-password-set.md`, and noted on Syncro ticket #32343 as a hidden internal comment, with a #bot-alerts post. + +Stored a new Backblaze B2 application key ("ClaudeTools", master-scope) in the SOPS vault at `projects/claudetools/backblaze-b2.sops.yaml`, then explored the B2 Native API v3 and built a full `b2` skill (`.claude/skills/b2/`) to manage ACG's B2 account — the storage backend for the GuruRMM mspbackups (MSP360) ability. The skill does status/buckets/keys/files/bucket-size, a storage-cost report at ACG's $0.00695/GB rate, gated bucket/key create/delete, and a lifecycle-based `delete-prefix` purge. Code-reviewed twice (build + delete-prefix). Committed `96fb411`. + +Investigated the legacy `MSPBackups20200311` bucket (61.3 TB / ~$426/mo, 12.5M file versions, 43 distinct machine backups). Cross-referenced every machine against MSP360 monitoring to separate orphans from live-account data, then executed a Tier-A purge: 14 orphaned machine prefixes (~10 TB / ~$69.65/mo) via scoped B2 lifecycle rules (1/1-day → server-side purge over ~24-48h). Tier B (~$37/mo under still-active accounts) and the Saguaro Conveyor trio (possible backup gap) were intentionally held. + +Built a `memory-dream` skill (memory lint/consolidation analyzer, additive-only) plus `.claude/scripts/sync-memory.sh` (additive repo<->harness-profile union) wired to a SessionStart hook, to reconcile the two memory stores and sync repo memory fleet-wide. Reconciled the drift (migrated 16 useful profile-only entries into the repo, resolved 1 conflict repo-wins). Committed `2a1ccfa`. + +Diagnosed "slow git ops": benchmarked SSH vs internal-HTTP+token vs external vs SOPS-decrypt — SSH was the SLOWEST (1.5s) and the credential lookup only ~0.33s, so SSH was the wrong fix (reverted). Profiling `/sync` instead found Phase 1a (submodule advance) was the real cost (5.66s). Gated the submodule network-advance behind `--with-submodules` (default off), cutting Phase 1a and killing per-sync submodule-pointer churn. Committed `f682ad9`. + +## Key Decisions + +- **Carla password as client-directed, not remediation:** no force-change / no session revoke per Mike; flagged weak-password + no-hardening for the audit trail but executed as directed. +- **B2 key broad capabilities are by design:** the "ClaudeTools" key is a storage-MANAGEMENT key for the RMM mspbackups feature (creates/deletes buckets + per-client keys), so master scope is required; dropped the over-privilege framing once purpose was clarified. +- **delete-prefix via lifecycle rules, not per-file delete:** 1.2M+ versions made per-file `b2_delete_file_version` impractical; a scoped lifecycle rule (daysFromUploadingToHiding=1, daysFromHidingToDeleting=1) does a free server-side purge in ~24-48h. +- **Tier split for the purge:** only orphaned prefixes (no active MSP360 plan) were deleted (Tier A). Anything under a still-active client account (Tier B) or still-listed-but-stopped (Saguaro trio) was held for confirmation — "stale in this bucket" != "safe to delete." +- **Memory tooling additive-only:** memory-dream never deletes/merges/overwrites (proposals only); sync-memory.sh unions both stores, surfaces conflicts instead of resolving them. Per Mike: "additive at first so we don't wipe useful data." +- **Scheduling = coord todo, not schedulers:** deferred work goes to `POST /api/coord/todos` for a future session, NOT the /schedule remote CCR agents (no vault/creds in cloud) or local scheduled tasks. Saved as memory. +- **SSH keys rejected for git-op speed:** benchmark-driven — SSH slower than HTTP+token; credential lookup is not the bottleneck. Reverted the pilot. +- **Submodule advance made opt-in:** per CLAUDE.md the guru-rmm pinned commit lagging main is expected, so advancing all 3 submodules every sync was wasted work + churn. +- **MEMORY.md rebase conflict resolved upstream-wins + re-add:** the GURU-BEAST-ROG consolidation is the intentional cleanup; took it as the base and re-added only the genuinely-new `reference_gitea_git_op_latency` entry. + +## Problems Encountered + +- **`ifRevisionMatch` rejected by b2_update_bucket** (HTTP 400 unknown field) on the first delete-prefix run — failed safe (no rules applied). Fixed by dropping the field (last-write-wins); read-merge-write retained. +- **Gitea SSH endpoint mismatch:** repo API `ssh_url` advertised `git@172.16.3.21` (a host sshd offering password — rejected our key). Gitea's actual built-in SSH server is `git@172.16.3.20:2222` (publickey-only). The SSH_DOMAIN config is wrong. +- **Git Bash mangled `GIT_SSH_COMMAND`** Windows paths — worked around with a `~/.ssh/config` alias (then removed during cleanup). +- **`/sync` aborted on a MEMORY.md rebase conflict** during the profiling run — a memory-index consolidation had landed from GURU-BEAST-ROG. Resolved with `git checkout --ours` (upstream consolidated index) + re-adding the one new entry, then `git rebase --continue` and push. +- **sync.sh self-committed the Phase-1a edit** during the Coding Agent's verification run (the script syncs its own working tree) — landed as `f682ad9` ahead of a separate review; post-reviewed the diff as correct. +- **coord todos POST schema** undocumented fields — requires `text` (not title/description) + `created_by_user` + `created_by_machine`. + +## Configuration Changes + +Created: +- `.claude/skills/b2/` — SKILL.md, scripts/b2.py, scripts/b2_client.py, scripts/selftest.py, references/api-reference.md, .gitignore +- `.claude/skills/memory-dream/` — SKILL.md, scripts/memory_dream.py, scripts/selftest.py +- `.claude/scripts/sync-memory.sh` +- `clients/rednour/reports/2026-06-01-carla-password-set.md` +- Vault: `projects/claudetools/backblaze-b2.sops.yaml` +- Memory: `reference_backblaze_storage_rate.md`, `feedback_scheduling_via_coord_todo.md`, `project_memory_consolidation_automation.md`, `reference_gitea_git_op_latency.md` (+ MEMORY.md index) + +Modified: +- `.claude/settings.json` — added SessionStart hook running sync-memory.sh +- `.claude/scripts/sync.sh` — Phase 1a submodule advance gated behind `--with-submodules` / `SYNC_SUBMODULES=1` (default off) +- `.gitignore` — B2 cache-ignore (`.claude/skills/b2/.cache/`) + +Cleanup: +- Removed pilot SSH artifacts: Gitea account key "GURU-5070" (id 6, deleted via API), local `~/.ssh/gitea_acg_ed25519` keypair, and the `gitea-acg` block in `~/.ssh/config`. + +## Credentials & Secrets + +- **Backblaze B2 "ClaudeTools" key** — vault `projects/claudetools/backblaze-b2.sops.yaml` (key_id `00146f69bc611630000000009`; secret encrypted in entry). Master-scope management key for RMM mspbackups. +- **carla@rednourlaw.com** — password set to `Fiduciary1234!` (client-directed; not vaulted — client's own credential). +- **EXPOSED — needs rotation:** `reference_gitea_internal.md` contains a live Gitea API token `9b1da4b79a38ef782268341d25a4b6880572063f` in plaintext, committed throughout repo history. Rotate and replace inline value with the vault-pointer. +- B2 cost basis: **$0.00695/GB** (ACG). + +## Infrastructure & Servers + +- **Backblaze B2:** accountId `46f69bc61163`, region us-west-001, apiUrl `https://api001.backblazeb2.com`, s3 `s3.us-west-001.backblazeb2.com`. 12 buckets; `MSPBackups20200311` (id `b4268f56790bccc671010613`) is the legacy shared bucket; clients migrated to per-client `ACG-` buckets. +- **Gitea:** HTTP/API internal `http://172.16.3.20:3000`; built-in SSH `git@172.16.3.20:2222` (publickey-only). External `git.azcomputerguru.com` = Cox IP -> NPM (openresty) on Jupiter. Shared push account `azcomputerguru` (id 1, admin). +- **MSP360 Managed Backup:** `https://api.mspbackups.com`, creds vault `msp-tools/msp360-api.sops.yaml`. Monitoring `UserID` == the B2 `MBS-` backup-set root, enabling live-vs-orphan cross-reference. +- **Coord API:** `http://172.16.3.30:8001/api/coord`. + +## Commands & Outputs + +- B2 storage cost: `py .claude/skills/b2/scripts/b2.py usage` +- B2 prefix purge (gated): `py .claude/skills/b2/scripts/b2.py delete-prefix ... --confirm` +- B2 lifecycle list/cleanup: `b2.py lifecycle ` / `b2.py lifecycle-remove ... --confirm` +- Memory lint: `py .claude/skills/memory-dream/scripts/memory_dream.py` (report-only) / `--apply-safe` +- Sync with submodule advance: `bash .claude/scripts/sync.sh --with-submodules` +- git-op latency (this machine): SOPS decrypt 0.33s, internal HTTP+token 0.55s, external 0.83s, SSH 1.5s. +- /sync phase profile: Phase 1a 5.66s (with advance) -> 3.23s (advance gated); full clean run ~7.4s. + +## Pending / Incomplete Tasks + +- **B2 Tier-A purge verification** — coord todo `de839db9`, due ~24-48h (2026-06-03): confirm 14 prefixes gone (~10 TB reclaimed), run `b2 lifecycle-remove` to clean the rules, decide Tier B + Saguaro trio. +- **Tier B (~$37/mo)** and **Saguaro Conveyor trio** (DESKTOP-L0U6QUN, SC-SERVER, DESKTOP-DGM4C1T — possible ~9-month backup gap) still ON HOLD pending per-client confirmation. +- **Rotate exposed Gitea token** in `reference_gitea_internal.md`. +- **Fix Gitea SSH_DOMAIN** (advertises .21 instead of .20:2222) — cosmetic/correctness. +- Optional further sync optimization: skip `git submodule update --init` for already-populated submodules (would cut remaining Phase 1a ~3.2s -> ~0). + +## Reference Information + +- Commits: `96fb411` (b2 skill), `2a1ccfa` (memory tooling), `f682ad9` (sync Phase 1a gate), `f7cc0cf` (MEMORY.md conflict resolution), `de839db9` (coord todo id). +- Syncro ticket #32343 (Rednour Law — Email Changes); customer id 1224246. +- Rednour tenant `4a4ca18a-f516-478b-99da-2e0722c5dc18`; carla object id `93074d1a-6db2-4794-8f7d-c84a619e4494`. +- 14 Tier-A purge prefixes documented in this session (MBS-/CBB_/ for DC, GTI-EXCHANGE, MAS90SVR, MAS2013, SERVER(22dd7062), webhost(85b4355a + f1885cea), NEWSERVER, PC1, SERVER(98f0c6ae), SERVER7, RVV-SERVER, mdmserver.office.mdmlaw.net, BB-BRANDY).