153 lines
19 KiB
Markdown
153 lines
19 KiB
Markdown
# Session Log: 2026-05-27
|
||
|
||
## User
|
||
- **User:** Mike Swanson (mike)
|
||
- **Machine:** GURU-5070
|
||
- **Role:** admin
|
||
|
||
## Session Summary
|
||
|
||
Continued from 2026-05-26 across the date boundary. Completed the identity.json Phase 2 migration on GURU-5070 (centralized Ollama/Python/platform config) directed by a coord message from the Mac session. `migrate-identity.sh` failed twice on Windows — it hardcoded `python3` instead of the detected `$PYTHON_CMD`, then passed a Git Bash POSIX path to native Windows Python. Fixed both (`$PYTHON_CMD` + `cygpath -m`), re-ran successfully, pushed the fix (251bb35), and sent Howard a heads-up to pull before running it on his Windows laptop. Pulled in Howard's GuruScan module refactor (GuruScan.psm1/.psd1, README.md, scanners.json, GURUSCAN_RESULT_JSON reporting) — it delivers on every gap and packaging suggestion from the prior coord thread. Saved a feedback memory to leave GuruScan alone until Howard requests review.
|
||
|
||
Ran a preemptive Valleywide health check (nothing reported by client). All six core hosts are UP: UDM, DC1, VWP-QBS (RDWeb 443 + RDP 3389 listening), HP iLO, ADSRVR, XenServer. The HP ProLiant — the recurring failure point (no UPS) — was confirmed powered ON via iLO. Key discovery: Tailscale silently hijacks VWP's `192.168.0.0/24` subnet (Tailscale route metric 5 beats the VWP VPN's 281), so `192.168.0.x` probes from any Tailscale-connected machine hit the wrong network; resolved the ambiguity with temporary `/32` routes via the VPN gateway. Valleywide has no GuruRMM agents (until an agent was deployed late in the session as a discovery/deployment testbed).
|
||
|
||
Investigated the GuruRMM "Network Deployment via discovery node" feature status: discovery (node designation + scanning + per-agent UI) is built, but deployment-to-discovered-devices is NOT (only a `deploying` status label exists; no push-install). The roadmap showed it as stale-unchecked — the same drift pattern as BUG-001.
|
||
|
||
That drift prompted the session's main work: making `FEATURE_ROADMAP.md` a living document. First added a roadmap-reconciliation pass (Agent F) to the `/rmm-audit` skill. Then, on Mike's decision, implemented three pieces: (1) a "Roadmap Is a Living Document" rule in GuruRMM's DESIGN.md + dev-principles memory making the roadmap update part of definition-of-done; (2) a one-time baseline reconcile flipping 44 verified-shipped core features `[ ]`→`[x]` (each proven against code by Agent F, conservative/end-to-end only); (3) flipped the audit's roadmap-pass default to reconcile-and-flip. The roadmap now reflects reality, dev work is the primary maintainer, and the audit is the backstop.
|
||
|
||
## Key Decisions
|
||
|
||
- **migrate-identity.sh: fixed both Windows bugs rather than just reporting** — they'd break every Windows machine in the fleet rollout; fix was unambiguous ($PYTHON_CMD + cygpath -m) and unblocks others.
|
||
- **Valleywide: used a scoped `/32` route override, not a routing-table reconfiguration** — minimal/reversible way to get a true reading of VWP's 192.168.0.x hosts past the Tailscale hijack; removed the routes immediately after.
|
||
- **GuruScan: hands-off until Howard asks** — declined to review his .psm1 refactor unprompted; saved the boundary to memory.
|
||
- **Roadmap convention = living status-and-plan tracker (Option B), maintained inline during dev.** The reconciliation revealed 0/705 feature lines were ever checked — the roadmap was a backlog. Mike chose to make it a true status doc maintained as part of definition-of-done, with the audit as backstop.
|
||
- **Baseline reconcile was conservative** — flipped only the 44 lines Agent F verified end-to-end; left ~661 (partials + genuinely-open) untouched. A wrongly-flipped line is worse than a missed one.
|
||
- **First roadmap pass run was annotate-only** (before the convention decision); the second run did the full flip after Mike chose Option B.
|
||
|
||
## Problems Encountered
|
||
|
||
- **migrate-identity.sh exit 127** (`python3: command not found`) then `FileNotFoundError` on `/d/...` path — Windows. Fixed with `$PYTHON_CMD` + `cygpath -m`; re-ran clean.
|
||
- **Valleywide 192.168.0.x hosts falsely showed DOWN** — Tailscale route for `192.168.0.0/24` (metric 5) overrides the VWP VPN route (metric 281), sending traffic to a different client's network. Disambiguated with `/32` routes via `192.168.4.1`; confirmed all hosts UP.
|
||
- **Misrouted an RMM bug to Howard earlier (BUG-001)** — corrected: RMM is Mike's; deleted the note; the GURU-KALI attribution-hardening pass (pulled this session) confirmed git history is clean (drift was reasoning-time inference).
|
||
- **Repeated push races** with concurrent GURU-KALI/Mac/HOWARD-HOME sessions — resolved by sync.sh rebase each time.
|
||
|
||
## Configuration Changes
|
||
|
||
- MODIFIED (gururmm repo) `docs/DESIGN.md` — new "The Roadmap Is a Living Document" rule (commit 3e114a0)
|
||
- MODIFIED (gururmm repo) `docs/FEATURE_ROADMAP.md` — 4 scope annotations on over-claiming lines (b6f7a49); baseline reconcile flipping 44 shipped lines `[ ]`→`[x]` + header note (3e114a0)
|
||
- CREATED (gururmm repo) `reports/2026-05-27-rmm-audit-roadmap.md` (b6f7a49)
|
||
- MODIFIED `.claude/skills/rmm-audit/SKILL.md` — Agent F roadmap-reconciliation pass + reconcile-and-flip default (14a6c09, a885b54)
|
||
- MODIFIED `.claude/memory/gururmm-development-principles.md` — "Living Roadmap (MANDATORY)" principle (a885b54)
|
||
- MODIFIED `.claude/memory/feedback_rmm_dev_is_mike.md` — added "leave GuruScan alone until Howard asks" (synced)
|
||
- MODIFIED `.claude/scripts/migrate-identity.sh` — Windows fixes (251bb35)
|
||
- MODIFIED (local, gitignored) `.claude/identity.json` — added python/ollama/platform/architecture fields (Phase 2 migration)
|
||
- PULLED: Howard's GuruScan module refactor; GURU-KALI attribution-hardening + identity Phase 2 (migrate-identity.sh, whoami-block.sh, sync.sh/syncro.md reading identity.json — no more Ollama curl probe on migrated machines)
|
||
|
||
## Credentials & Secrets
|
||
|
||
- **Valleywide HP iLO:** `clients/vwp/hp-ilo.sops.yaml` — host 172.16.9.125, Administrator / `EV2PBU6J` (iLO reset to factory 2026-04-22). SSH needs paramiko with `disabled_algorithms={'pubkeys':['rsa-sha2-256','rsa-sha2-512']}`.
|
||
- **Valleywide vault path is `clients/vwp/`** (NOT `clients/valleywide/` as the wiki states — wiki drift). Entries: adsrvr, dc1, udm, xenserver, hp-ilo, quickbooks-server-idrac, server2003, brother-mfc-l3780cdw.
|
||
- No other new secrets. identity.json (gitignored) now carries ollama.endpoint/prose_model + python.command.
|
||
|
||
## Infrastructure & Servers
|
||
|
||
- **Valleywide (VWP):** all UP as of 2026-05-27. UDM 172.16.9.1 (443 up), DC1 172.16.9.2, VWP-QBS 172.16.9.169 (RDWeb 443 + RDP 3389 listening), HP iLO 172.16.9.125 (ProLiant powered ON), ADSRVR 192.168.0.25, XenServer 192.168.0.104. OpenVPN client pool 192.168.4.0/24 (this machine got 192.168.4.3). **Tailscale hijacks 192.168.0.0/24** — use `/32` routes via 192.168.4.1 to reach VWP's 192.168.0.x reliably. No GuruRMM agents enrolled (1 deployed late as discovery/deployment testbed).
|
||
- **GuruRMM:** live main now 3e114a0; agent fleet 0.6.39/0.6.41. Discovery: node designation + scanning + per-agent DiscoveryTab built; fleet view + deployment-to-discovered-devices NOT built. `user_session` command context: migration 041, agent/src/watchdog/wts.rs.
|
||
- **Identity migration:** GURU-5070 + HOWARD-HOME both on Phase 2 (python.command=py, ollama.endpoint=localhost:11434, platform=windows, amd64; GURU-5070 prose_model qwen3:8b, HOWARD-HOME qwen3:14b).
|
||
|
||
## Commands & Outputs
|
||
|
||
- iLO power check (read-only): paramiko SSH to 172.16.9.125, `power` → "server power is currently: On"; `show /system1 enabledstate` → enabled.
|
||
- Scoped route workaround: `route add 192.168.0.25 mask 255.255.255.255 192.168.4.1` (+ .104), ping, then `route delete` — confirmed both UP, routes removed.
|
||
- Roadmap flip: exact-line-match Python script flipped 44 `- [ ]`→`- [x]` (each matched exactly 1x, 0 misses/dupes).
|
||
- migrate-identity fix: `"$PYTHON_CMD"` + `IDENTITY_PATH_PY=$(cygpath -m "$IDENTITY_PATH")`.
|
||
|
||
## Pending / Incomplete Tasks
|
||
|
||
- **VWP discovery/deployment testbed:** agent deployed; exercise discovery (designate node, scan LAN) and shake out the not-yet-built deployment path.
|
||
- **Roadmap convention now active** — going forward, RMM features must update FEATURE_ROADMAP.md in the same change (definition-of-done). Audit backstops.
|
||
- **Lonestar Apple MDM:** gather iPhone/iPad serials + iOS versions, choose APNs Apple ID, supervised-vs-unsupervised decision, targeted-invite enrollment.
|
||
- **Glabman wifi quote** (todo 1bf0cfef, due 2026-05-27).
|
||
- **GND-SERVER Datto alert:** confirm cleared (deletion synced).
|
||
- (Carried) quantumwms John Velez consent; 2x Business Premium before 2026-06-03; Autotask skill; Western Tire #32199; Kittle HIGH.
|
||
|
||
## Reference Information
|
||
|
||
- gururmm commits: b6f7a49 (roadmap annotations + report), 3e114a0 (living-roadmap principle + 44-flip reconcile).
|
||
- claudetools commits: a885b54 (living-roadmap memory + skill convention), 14a6c09 (rmm-audit Agent F pass), 251bb35 (migrate-identity Windows fix).
|
||
- Coord: Howard "Phase 2 migration done on HOWARD-HOME"; my replies 8618a252 (identity Phase 2), 5ab63a21 (migrate-identity heads-up to Howard). Deleted misrouted BUG-001 note (was 92468218).
|
||
- GuruScan (Howard's): projects/msp-tools/guru-scan/ — now GuruScan.psm1/.psd1 + README + scanners.json + GURUSCAN_RESULT_JSON. Hands-off until he asks (feedback_rmm_dev_is_mike.md).
|
||
- Report: projects/msp-tools/guru-rmm/reports/2026-05-27-rmm-audit-roadmap.md.
|
||
|
||
---
|
||
|
||
## Update: 08:40 PT — Vault-connectivity diagnosis, memory audit, RMM full audit + Phase 1 authz remediation (deployed)
|
||
|
||
### Session Summary
|
||
|
||
Diagnosed the reported external flap on `git.azcomputerguru.com`. SSHed IX (the ACG website host, unrelated) then traced the real path: the domain is served by **NPM (openresty) on Jupiter `172.16.3.20`** via the office Cox IP `72.194.62.10` — **not Cloudflare**. The flap was a transient NPM SSL-cert renewal (NPM log entry `14:14:36 UTC`). Corrected the machine-local auto-memory `reference_gitea_internal.md`, which wrongly claimed git.azcomputerguru.com sat behind Cloudflare and blocked curl.
|
||
|
||
Audited the shared in-repo memory (`.claude/memory/`): indexed 8 orphaned files into `MEMORY.md`, added frontmatter to 5 files, trimmed oversized index lines, de-duplicated, and fixed a broken backlink in the index (`../.claude/POWER_FAILURE_RUNBOOK` → `../POWER_FAILURE_RUNBOOK`).
|
||
|
||
Ran a full `/rmm-audit` pass (all six passes on Opus 4.7: parallel agents A–D + F, sequential E build-pipeline). **62 findings — 3 CRITICAL, 9 HIGH, 12 MEDIUM** + lows/info. Report: `projects/msp-tools/guru-rmm/reports/2026-05-27-rmm-audit.md`. The 3 CRITICALs are the same authorization class: handlers that take `_auth: AuthUser` (authenticate-only, **no** org-scope authorization) — a BOLA/IDOR hole on credentials, command dispatch, and script execution.
|
||
|
||
On Mike's "fix all → start Phase 1, TODO the rest" direction, implemented **Phase 1 (the 3 CRITICALs)** on branch `remediation/2026-05-27`, plus the create_credential gate that Code Review flagged. While building I discovered **main did not compile** — Howard's `3b19ff0` changed `db::logs::get_fleet_logs` to a 5-arg signature but left 4 stale callers in `logs.rs` (E0061 ×4). That compile break is exactly why Howard's server deploy was "stuck" (binary frozen at the May 25 build). Folded the caller fix into the same branch (`4961923`), so the deploy ships the build fix and the authz fixes together. Code Review returned **APPROVE-WITH-NITS** (caught create_credential ungated → HIGH → fixed). `cargo check` green at `bdefb1f`. Merged the branch to main (fast-forward), CI bumped to `de39e42` (v0.3.30), and deployed via `sudo /opt/gururmm/build-server.sh`. **Verified live:** release build 4m45s, systemd restarted `15:32 UTC`, `ExecStart=/opt/gururmm/gururmm-server` running the fresh binary. Phases 2–5 captured as coord TODOs. Notified Howard of the in-flight fix, the remediation task list, the living-roadmap definition-of-done expectation, and (post-deploy) that his fleet-log fix is now live.
|
||
|
||
### Key Decisions
|
||
|
||
- **Option B — merge the whole branch + deploy at once** (vs. cherry-picking just the build fix). Ships the get_fleet_logs fix and all Phase 1 authz together; Mike acknowledged the authz changes are behavior-changing (org-scoped 403s where before any authed user passed).
|
||
- **`authorize_agent_access` is fail-closed** — an agent with no site / orphaned client_id returns **403**, stricter than the reference `get_agent` handler which fails open. A credential/command/script path must never default-allow on missing scope.
|
||
- **`reveal_credential` gated dev_admin-only BEFORE the DB fetch** — don't even read the secret out of the DB if the caller isn't authorized.
|
||
- **New commit `bdefb1f` for the create_credential fix, not an amend** — keeps `4961923` (the build fix) byte-stable and cherry-pickable, after an earlier `--amend` mistake rewrote its SHA.
|
||
- **Roadmap-compliance verification of Howard's sessions = no violation** — his only post-rule commit (`3b19ff0`) was a bug fix to an already-`[x]` feature, which requires no roadmap flip. The rule is brand-new, so the action is forward-looking: confirm his sessions pulled the updated DESIGN.md + memory.
|
||
|
||
### Problems Encountered
|
||
|
||
- **main wouldn't compile (E0061 ×4 in logs.rs)** — pre-existing breakage from Howard's `3b19ff0` get_fleet_logs signature change; none of my authz files were in the errors. Root-caused, fixed callers to the 5-arg form (`&["ERROR"], None, since, 1000`), committed `4961923`.
|
||
- **Stale cargo check** — `git fetch origin <branch>` does NOT fast-forward the local branch, so checks ran old code. Fixed by checking out `origin/remediation/2026-05-27` detached.
|
||
- **`git commit --amend` mistake** — amended the build commit, folding in the credentials fix and changing the `4961923` SHA I'd told Howard to cherry-pick. Recovered with `git reset --hard origin/remediation/2026-05-27`, re-applied the one-liner as the new commit `bdefb1f`.
|
||
- **`internal_err` not in scope (E0425)** in credentials.rs create_credential gate — `internal_err` isn't imported there; switched to the inline `.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?` pattern the file already uses.
|
||
- **Deploy binary-path ambiguity** — post-deploy, `/opt/gururmm/gururmm-server` was fresh (May 27 15:32) but `/usr/local/bin/gururmm-server` was still May 25. Verified `systemctl cat` → `ExecStart=/opt/gururmm/gururmm-server`; the `/usr/local/bin` copy is vestigial and unused. No action needed (candidate cleanup item).
|
||
|
||
### Configuration Changes (gururmm repo, branch merged to main)
|
||
|
||
- MODIFIED `server/src/api/mod.rs` — new `pub async fn authorize_agent_access(state, auth, agent_id)` helper (admin bypass; agent→site→client_id→`can_access_org`; fail-closed 403). Added imports `AuthUser`, `db`, `uuid::Uuid`.
|
||
- MODIFIED `server/src/api/credentials.rs` — `authorize_credential_access(state, user, cred)` branching on scope_type (global→`is_dev_admin`; client→`is_admin`|`can_access_org`; site→resolve→`can_access_org`; unknown→403). Gated list_global/list_client/list_site/get_credential_meta/reveal_credential (dev_admin-only, pre-fetch)/update/delete AND create_credential.
|
||
- MODIFIED `server/src/api/commands.rs` — `send_command` calls `authorize_agent_access` before dispatch.
|
||
- MODIFIED `server/src/api/scripts.rs` — `run_script_on_agent` → `authorize_agent_access(req.agent_id)`; library CRUD → `is_admin()` gate.
|
||
- MODIFIED `server/src/api/logs.rs` — fixed 4 stale `get_fleet_logs` callers to 5-arg signature (build fix; was breaking main).
|
||
- Commits: `4961923` (build fix), `bdefb1f` (create_credential gate err-map fix). Merged FF to main; CI auto-bump → `de39e42` (v0.3.30).
|
||
|
||
### Configuration Changes (claudetools repo)
|
||
|
||
- MODIFIED `.claude/memory/MEMORY.md` — indexed 8 orphans, fixed POWER_FAILURE_RUNBOOK backlink, trimmed oversized lines, dedup.
|
||
- MODIFIED 5 memory files — added frontmatter.
|
||
- MODIFIED (machine-local auto-memory) `reference_gitea_internal.md` — corrected the Cloudflare claim (git.azcomputerguru.com = office Cox 72.194.62.10 → NPM/openresty on Jupiter 172.16.3.20).
|
||
|
||
### Infrastructure & Servers
|
||
|
||
- **git.azcomputerguru.com path:** office Cox IP `72.194.62.10` → **NPM (openresty) on Jupiter `172.16.3.20`** → Gitea `172.16.3.20:3000`. NOT Cloudflare. External flaps = NPM SSL renewal events.
|
||
- **GuruRMM server:** `172.16.3.30:3001`, systemd `gururmm-server`, `ExecStart=/opt/gururmm/gururmm-server` (NOT `/usr/local/bin/`). Now **v0.3.30 / de39e42**, restarted `2026-05-27 15:32:28 UTC`, MainPID 598071. Deploy is manual: `sudo /opt/gururmm/build-server.sh` (git reset --hard origin/main → cargo build --release → stop/cp/start). No Phase 1 migrations, so `.sqlx` cache untouched.
|
||
|
||
### Commands & Outputs
|
||
|
||
- Deploy verify: `systemctl cat gururmm-server | grep ExecStart` → `/opt/gururmm/gururmm-server`; `ActiveEnterTimestamp=Wed 2026-05-27 15:32:28 UTC` (== fresh binary mtime); `SubState=running`.
|
||
- cargo check (warm, origin/remediation/2026-05-27 @ bdefb1f): `CARGO_EXIT=0`, Finished in 25.53s, 0 errors.
|
||
- get_fleet_logs caller fix shape: `get_fleet_logs(&state.db, &["ERROR"], None, since, 1000)` (was 4-arg `"ERROR", since, 1000`).
|
||
|
||
### Pending / Incomplete Tasks (remediation Phases 2–5, coord TODOs)
|
||
|
||
- **Phase 2** (`9a1ed577`, HIGH authz/IDOR): org-scope checks.rs / inventory / user_inventory / commands reads / registry; auth on `/agents/status-stream` SSE.
|
||
- **Phase 3** (`54239760`, HIGH): `sqlx::query!`/`query_as!` → runtime (mspbackups, updates); build-linux.sh stray `n#` + duplicate beta block.
|
||
- **Phase 4** (`58c3fcad`, HIGH/MED): `internal_err` sweep (~127 sites); log redaction; MSPBackups mappings UI; React error boundary; AgentDetail client enrichment row.
|
||
- **Phase 5** (`fd677411`, MED/LOW): discovery IP validation, registry wire fields, defer_hours, ws api-key char-boundary, TS `any`, aria-labels, localhost fallback, /metrics+stats wiring.
|
||
- **Cleanup candidate:** remove the stale `/usr/local/bin/gururmm-server` (unused by systemd).
|
||
- (Carried) Lonestar Apple MDM enrollment; Glabman wifi quote (todo `1bf0cfef`, due 2026-05-27); quantumwms John Velez consent; 2× Business Premium before 2026-06-03; Western Tire #32199; Kittle HIGH; VWP discovery/deployment testbed.
|
||
|
||
### Reference Information
|
||
|
||
- gururmm: `4961923` (build fix), `bdefb1f` (create_credential gate), merged to main → `de39e42` (v0.3.30, deployed).
|
||
- Reports: `reports/2026-05-27-rmm-audit.md` (62 findings), `reports/2026-05-27-rmm-audit-roadmap.md`.
|
||
- Coord TODOs (gururmm, assigned mike): `9a1ed577` `54239760` `58c3fcad` `fd677411`.
|
||
- Coord messages to Howard: `114e6209` (fix in flight), `b14e1793` (task list + roadmap guidance + build-check nit), `44ac8984` (server deployed / log fix live). Component `gururmm/server` → `deployed` v0.3.30.
|