From 09c52a7d095c46ea87d79fdca166db42be301360 Mon Sep 17 00:00:00 2001 From: Mike Swanson Date: Wed, 27 May 2026 06:11:16 -0700 Subject: [PATCH] feat(rmm-audit): add roadmap reconciliation pass (Agent F) + cleanup step MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The skill's frontmatter claimed it updated FEATURE_ROADMAP.md, but the body had no roadmap-reconciliation logic — so stale checkboxes slipped through (Network Discovery Node backend + BUG-001 temperature both shipped while marked [ ]). Added: - Agent F (parallel, read-only): cross-references every roadmap checkbox against code artifacts; classifies STALE-INCOMPLETE / PARTIAL / STALE-COMPLETE / ACCURATE with proving artifact. Conservative — only flips when end-to-end evidence is unambiguous; backend/scaffolding-only is PARTIAL, never flipped. - Living-docs step: actually flip stale checkboxes, annotate partials, flag [x]-but-missing as [HIGH] regressions; every change logged in the report's new "FEATURE_ROADMAP.md Delta" section (no silent edits). - Phase 0 extracts the roadmap claims list; --pass=roadmap added. Co-Authored-By: Claude Opus 4.7 (1M context) --- .claude/skills/rmm-audit/SKILL.md | 88 +++++++++++++++++++++++++++++-- 1 file changed, 84 insertions(+), 4 deletions(-) diff --git a/.claude/skills/rmm-audit/SKILL.md b/.claude/skills/rmm-audit/SKILL.md index 99c04a0..ca3abbf 100644 --- a/.claude/skills/rmm-audit/SKILL.md +++ b/.claude/skills/rmm-audit/SKILL.md @@ -10,7 +10,8 @@ description: | FEATURE_ROADMAP.md). Takes 10-20 minutes. Invoke explicitly only — no auto-trigger. Use /rmm-audit for a full audit. - Optional arg: --pass= to run a single pass (api, ui, rust, ts, security, pipeline). + Optional arg: --pass= to run a single pass (api, ui, rust, ts, security, pipeline, roadmap). + The roadmap pass reconciles FEATURE_ROADMAP.md checkboxes against the code and cleans up stale ones. --- # GuruRMM End-to-End Audit @@ -58,12 +59,17 @@ Before spawning agents, read these yourself: Capture from `server/src/api/mod.rs` the complete route list (all `.route(...)` calls). This becomes the **authority route list** passed to both the API and UI audit agents. +Also extract every checkbox line from `FEATURE_ROADMAP.md` (both `[ ]` and `[x]`, with +section + priority) into a **roadmap claims list** — passed to Agent F for reconciliation +against the code. + --- ## Phase 1: Parallel Audit Agents -Spawn all four agents simultaneously in a single message. Each agent receives the -full context it needs inline — do not assume they share context. +Spawn all five parallel agents (A, B, C, D, F) simultaneously in a single message. +Each agent receives the full context it needs inline — do not assume they share context. +(Agent E — Build Pipeline — runs sequentially afterward; it SSHes the live build server.) --- @@ -226,6 +232,48 @@ Return structured findings with file:line references. --- +### Agent F — Roadmap Reconciliation + +**Goal:** Make `FEATURE_ROADMAP.md` tell the truth. The roadmap drifts from reality: +features ship in code while their checkbox stays `[ ]` (verified cases: the Network +Discovery Node backend, and temperature monitoring / BUG-001 — both shipped while marked +incomplete), and occasionally an item is `[x]` but the code was never written or was +reverted. This pass reconciles every checkbox against the code. + +**Runs in parallel with A–D (read-only code search, no SSH).** + +**Instructions for agent:** + +1. Read `docs/FEATURE_ROADMAP.md` and extract every checkbox line with its state + (`[ ]` / `[x]`), section, and priority. + +2. For EACH item, find the implementing artifact in the code — do NOT trust the + checkbox. Map feature → evidence: + - API/endpoint features → `server/src/api/mod.rs` routes + the handler module + - DB/schema features → `server/migrations/*.sql` + `server/src/db/*.rs` + - Agent capabilities → `agent/src/` modules + - Dashboard/UI features → `dashboard/src/pages|components/*.tsx` + `api/client.ts` + Use Grep/GrepAI. Cite the exact artifact (file:line, migration name, route path). + +3. Classify each item: + - **STALE-INCOMPLETE** — `[ ]` but the code fully implements it end-to-end → + recommend flip to `[x]`. Cite the proving artifact. + - **PARTIAL** — `[ ]`, backend/some layers exist but not end-to-end for the stated + scope (e.g. backend + API done but no UI; scaffolding present but unwired — like + safe-rollout, or discovery with a per-agent tab but no fleet view). Keep `[ ]` but + recommend an inline annotation, e.g. `[ ] ... (backend done; UI/fleet-view pending)`. + Do NOT flip partials to `[x]`. + - **STALE-COMPLETE** — `[x]` but the code does not implement it (never built or + reverted) → recommend flip to `[ ]`, flag `[HIGH]` (the roadmap is lying). + - **ACCURATE** — checkbox matches code. No change. + +4. Be conservative: only flip `[ ]`→`[x]` when the evidence is unambiguous AND the + feature is complete end-to-end for its stated scope. When in doubt → PARTIAL with a + note, never a flip. A backend-only or scaffolding-only implementation is PARTIAL. + +Return a table: +`item text | section | current state | verdict (STALE-INCOMPLETE / PARTIAL / STALE-COMPLETE / ACCURATE) | proving-or-missing artifact` + --- ### Agent E — Build Pipeline Health @@ -375,7 +423,7 @@ Write to: `projects/msp-tools/guru-rmm/reports/YYYY-MM-DD-rmm-audit.md` # GuruRMM Audit Report — YYYY-MM-DD **Auditor:** Claude (claude-opus-4-7) -**Passes:** API Coverage, UI Gaps, Rust Quality, TypeScript Quality, Data Integrity, Build Pipeline +**Passes:** API Coverage, UI Gaps, Rust Quality, TypeScript Quality, Data Integrity, Build Pipeline, Roadmap Reconciliation **Previous audit:** [link to prior report if one exists, else "First audit"] --- @@ -452,6 +500,19 @@ New gaps discovered this audit: --- +## FEATURE_ROADMAP.md Delta (Agent F) + +Checkboxes corrected to match code reality this audit: +- `[ ]` → `[x]` **** — was marked incomplete but is shipped. Proof: ``. +- `[x]` → `[ ]` **** — marked complete but NOT in code (regression / never built). `[HIGH]` + +Annotated as partial (left `[ ]`, scope clarified): +- `[ ]` **** (backend done; UI/fleet-view pending) — `` + +Verified accurate (no change): N items. + +--- + ## Recommended Action Order 1. [CRITICAL items, sorted by impact] @@ -467,6 +528,24 @@ After writing the report, update `docs/UI_GAPS.md`: - Add any newly discovered gaps under the appropriate priority section - Do NOT remove completed items — move them to the "Completed Features" section +### Update FEATURE_ROADMAP.md (roadmap cleanup) + +Apply Agent F's reconciliation to `docs/FEATURE_ROADMAP.md` — this is the roadmap +cleanup the audit is responsible for. The roadmap is a living doc, so editing it fits +the "living docs are updated" exception to read-only. + +- **STALE-INCOMPLETE → flip `[ ]` to `[x]`** for every item Agent F proved is shipped + end-to-end. Keep the line text; optionally append `(verified )`. +- **PARTIAL → leave `[ ]`, append the scope annotation** Agent F recommends + (e.g. `(backend done; UI/fleet-view pending)`). Never flip a partial to `[x]`. +- **STALE-COMPLETE → flip `[x]` to `[ ]`** and add `[REGRESSION — flagged YYYY-MM-DD]` + so it's visible; mirror it as a `[HIGH]` finding in the report. +- Update any "Last Updated" / status line in the roadmap. +- Every checkbox change MUST also appear in the report's "FEATURE_ROADMAP.md Delta" + section with its proving (or missing) artifact — no silent edits. +- When evidence is ambiguous, do NOT change the checkbox; record it as PARTIAL/INFO + and leave the cleanup for human review. Bias toward under-flipping. + --- ## Phase 5: User Summary @@ -482,6 +561,7 @@ MEDIUM (N): Batched in report. Pipeline: [one-line status — e.g. "all green" or highest-severity finding] UI_GAPS.md: N items marked complete, N new gaps added. +Roadmap: N checkboxes corrected (N stale-incomplete flipped to done, N partial annotated, N regressions flagged). Recommended first action: [the single highest-priority finding] ```