Builds the false-positive/true-positive proof the plan requires before the guard can be promoted to blocking, and fixes the one false-positive it surfaced. - test-harness-guard.sh: 12-case matrix in a throwaway repo, runs the REAL guard, asserts WARN/clean for real conflicts/secrets/keys vs legit content (setext underlines, dividers, docs that mention a marker, encrypted sops, public keys, .example templates). - harness-guard.sh: conflict rule now requires a real hunk (BOTH ^<<<<<<< AND ^>>>>>>>), dropping the lone =======$ trigger that false-positived on a 7-char setext underline / divider. Identical true-positive power (git writes all three markers); FP surface -> 0. - /self-check: new harness.guard_selftest runs the matrix in an isolated temp repo (read-only vs the real tree) so guard correctness is continuously proven. Verified 12/12 pass, true positives intact, real-tree FP surface = 0. FATAL flip (todo f1c11d0d, on/after 2026-06-22) is now evidence-backed + one-step. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
174 lines
11 KiB
Markdown
174 lines
11 KiB
Markdown
---
|
|
name: self-check
|
|
description: "Self-diagnose this machine's harness conformance vs the fleet baseline: identity.json, tooling, env/paths, hooks, skill/command/script set, vault decrypt, coord/Gitea reachability, capability tier. Grades RED/AMBER/GREEN; can publish a census. Triggers: self check/test, doctor, health check, am I configured right, harness/fleet conformance."
|
|
|
|
---
|
|
|
|
# Self-Check — ClaudeTools Harness Self-Diagnosis
|
|
|
|
A top-to-bottom evaluation of how *this* machine's ClaudeTools harness is wired,
|
|
graded against a checked-in **baseline manifest** so every machine behaves the
|
|
same way — while explicitly allowing for architecture, OS, and hardware
|
|
differences via a **capability tier** model.
|
|
|
|
This is the skill the user asked for when a session needs to "make sure
|
|
everything is as it should be."
|
|
|
|
## The model in one paragraph
|
|
|
|
`identity.json` is the foundational, per-machine map of **where things live and
|
|
what this box can do** (vault path, repo root, platform, arch, python command,
|
|
Ollama endpoints). The **baseline manifest**
|
|
(`baseline/manifest.json`) declares what *every* machine must have — required
|
|
tools, identity fields, scripts, hook files, the wired `settings.json` hooks, the
|
|
canonical skill/command set, and the **capability rules** that say what to do when
|
|
a capability is absent (e.g. no local Ollama → use the remote endpoint, or if that
|
|
is also down, route Tier-0 work to haiku instead of blocking). The probe compares
|
|
the live machine against the manifest, resolves the machine's capability tier, and
|
|
grades RED/AMBER/GREEN. Required things missing = RED. Advisory drift = AMBER.
|
|
Capability differences are **never** failures — they select a ruleset.
|
|
|
|
## V1 is a CENSUS tool (read this)
|
|
|
|
There is no ratified fleet baseline yet. `baseline/manifest.json` is **provisional**,
|
|
generated from a single known-good machine (GURU-5070). So V1's job is to gather
|
|
ground truth from every machine and help Mike build the real baseline:
|
|
|
|
1. **Probe** — each machine runs the check and produces a structured census.
|
|
2. **Publish** — `--publish` PUTs the census to coord as component
|
|
`selfcheck_<host>` (state = grade, notes = full JSON). One row per machine =
|
|
a live fleet conformance view.
|
|
3. **Fan out** — `fanout` broadcasts a request to `ALL_SESSIONS` so every active
|
|
instance reports.
|
|
4. **Aggregate** — `aggregate` reads all censuses back and proposes a baseline
|
|
(tools/skills/commands present on *all* machines = "required everywhere";
|
|
present on *some* = "capability-gated"), and lists machines with FAILs.
|
|
|
|
Mike reviews the aggregate and ratifies `manifest.json`. From then on the same
|
|
probe enforces conformance. **V1 does not auto-fix anything** — it reports the
|
|
exact fix command for each finding (per the decision on record).
|
|
|
|
## Running it
|
|
|
|
The probe is `scripts/self-check.sh` (bash; runs on Git Bash/Windows, macOS,
|
|
Linux; deps: jq + curl). Always pass a real UTC timestamp:
|
|
|
|
```bash
|
|
SELFCHECK_TS="$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
|
|
bash .claude/skills/self-check/scripts/self-check.sh <mode>
|
|
```
|
|
|
|
| Mode | Purpose |
|
|
|------|---------|
|
|
| `report` (default) | Human RED/AMBER/GREEN report. Exit 0/1/2 = GREEN/AMBER/RED. |
|
|
| `--json` | Structured census JSON to stdout (for piping). |
|
|
| `--publish` | Run + publish census to coord (component `selfcheck_<host>`). Softfails to `.claude/coord-queue.jsonl`. |
|
|
| `fanout` | Broadcast a census request to ALL_SESSIONS. |
|
|
| `aggregate` | Fleet table + proposed-baseline summary from published censuses. |
|
|
|
|
`/self-check` is the slash-command runner for the same script.
|
|
|
|
## What it checks
|
|
|
|
| Category | Checks |
|
|
|----------|--------|
|
|
| **identity** | identity.json exists + valid JSON; **all required fields present**; `claudetools_root` exists and equals the running repo; `vault_path` exists; `machine` == hostname; git user.name/email match identity. |
|
|
| **tooling** | required everywhere: bash, git, jq, curl, sops, age, ssh, a python. Missing = FAIL. |
|
|
| **capability** | ollama, cargo, node, gh, docker, op — presence is INFO, never a failure. Resolves the **Ollama tier** (local / remote / none) and prints the effective Tier-0 ruleset. |
|
|
| **files** | required scripts + hook files present and executable. |
|
|
| **hooks** | the three `settings.json` hooks are wired (block-backslash PreToolUse, check-messages UserPromptSubmit, sync-memory SessionStart); `current-mode` present. |
|
|
| **git** | origin points at ACG Gitea (internal IP preferred); main-repo post-commit hook installed (AMBER if not). |
|
|
| **skills/commands** | every skill dir and command file in the baseline is present; extras are reported as census candidates. |
|
|
| **duplicates** | command/skill names present in BOTH the repo and `~/.claude`. Divergent content = WARN (the "same `/cmd`, different behaviour on the Mac" bug); identical = INFO (redundant, will drift). CRLF-only differences are ignored. |
|
|
| **memory** | `MEMORY.md` index exists; no orphaned memory files; manifest-declared contradiction patterns (see semantic pass below). Never FAILs the grade. |
|
|
| **harness** | the 1.4.0 invariants (read-only): VERSION marker present + not older than `manifest.harness.min_version`; **skill-registry description budget** (sum of all SKILL.md `description:` fields under `registry_desc_budget_chars` — WARN on regrowth); global deploy targets `~/.claude/skills` + `~/.claude/commands` populated (the "Mac wiped global skills" failure); `harness-guard.sh` present + wired into `sync.sh`; core scripts parse (`bash -n`); `now-phoenix.sh --date` emits a valid date; **guard self-test** runs the full `test-harness-guard.sh` false-positive/true-positive matrix in an isolated temp repo (proves the guard still catches real conflicts/secrets and does not false-positive — the standing prerequisite for promoting the guard to FATAL). Budget/min-version/script-list are tunable in `manifest.harness`. |
|
|
| **consistency** | the **command-restates-standard** lint (deterministic half): for each `manifest.command_standard_links` pair, the standard must still contain its defer-to-SSOT pointer to the owning command. A lost pointer = WARN (the standard likely drifted back into restating the command — the Syncro-timers failure mode). The semantic contradiction judgement is delegated to the model (see below). |
|
|
| **vault** | vault repo exists; sops+age present; `vault.sh list` succeeds (decrypt wired). |
|
|
| **connectivity** | coord API (required), main API + internal Gitea (advisory; off-network is OK). |
|
|
|
|
## Rogue-memory contradiction — semantic pass (do this when asked, or on a full check)
|
|
|
|
The engine's memory check is deterministic and conservative (index + orphans +
|
|
declared patterns) so it never produces false alarms. A *true* contradiction
|
|
check — "does any memory directly contradict what this machine's settings say?"
|
|
— is a judgment task, so the model does it (route the prose/classification to
|
|
Ollama Tier-0 per the house rules; Claude reviews the result):
|
|
|
|
1. Read `identity.json` (where things live + this box's capabilities),
|
|
`settings.json` (wired hooks/permissions), and `baseline/manifest.json`.
|
|
2. Read the memory index `.claude/memory/MEMORY.md`, then open any memory whose
|
|
one-line hook touches: paths/roots, python launcher, endpoints/IPs, OS/arch
|
|
assumptions, tool choices, or model routing.
|
|
3. Flag memories that **directly contradict** this machine's reality, e.g.:
|
|
- prescribes `python3`/`python` when `identity.python.command` is `py` (or vice-versa),
|
|
- hardcodes a repo/vault path that isn't this machine's `claudetools_root`/`vault_path`,
|
|
- names an endpoint/IP that conflicts with `identity.coord_api` or the manifest,
|
|
- assumes a capability (local Ollama) this machine's tier says is absent.
|
|
4. Report each as: memory file, the contradicting claim, the setting it violates,
|
|
and a suggested correction. **Do not edit memories** — surface for the operator
|
|
(deletions/rewrites go through the human, mirroring memory-dream's posture).
|
|
|
|
Genuinely machine-specific guidance in a *shared* memory is the usual culprit —
|
|
the fix is to scope it ("on Windows…") or split it, not to globally flip it.
|
|
|
|
### Semantic pass 2 — command vs standard contradiction
|
|
|
|
The `consistency` category only checks that the defer-to-SSOT *pointer* is present.
|
|
Whether a command and its standard actually **say contradictory things** is a
|
|
judgement task — do it the same way (Ollama Tier-0 for the read/classify, Claude
|
|
reviews):
|
|
|
|
1. For each `manifest.command_standard_links` pair, read BOTH the standard and the
|
|
owning command it points to.
|
|
2. Flag any rule the standard states that **conflicts** with the command (e.g. the
|
|
standard mandates a timer for routine billing while `/syncro` says line-item is
|
|
normal and timers are outlier-only — the original drift this lint exists to catch).
|
|
3. Report: the topic, the conflicting claims (quote both sides), and which one is the
|
|
SSOT. **Do not edit** — surface for the operator; the SSOT (the command) wins, so
|
|
the fix is almost always to correct the standard, not the command.
|
|
|
|
New links are cheap to add — drop another `{topic, standard, must_reference, why}`
|
|
into `manifest.command_standard_links` whenever a command and a standard speak to the
|
|
same rule.
|
|
|
|
## Fleet self-remediation loop (machines fix themselves)
|
|
|
|
We never fix a remote machine. The flow is:
|
|
|
|
1. `fanout` — broadcast asks every instance to self-check + self-fix + re-publish.
|
|
2. Each operator runs `/self-check` locally, applies the printed fix commands on
|
|
their own box, re-runs to confirm GREEN, then `/self-check --publish`.
|
|
3. `aggregate` — shows who is still RED/AMBER and prints each machine's own fix
|
|
list. Relay it to that operator; do not run it for them.
|
|
4. Repeat until the fleet is consistently GREEN, then ratify the manifest.
|
|
|
|
## How to interpret a run
|
|
|
|
After running, summarize for the user:
|
|
- The **grade** and the PASS/WARN/FAIL/INFO tallies.
|
|
- Each **FAIL** and **WARN** with its exact fix command. Do not auto-apply.
|
|
- The **capability tier** line — confirm the machine knows its Tier-0 fallback.
|
|
- If publishing/aggregating, note how many machines have reported and which are RED.
|
|
|
|
Capability differences (no Ollama, no gh, ARM vs amd64, macOS vs Windows) are
|
|
expected and must never be reported as broken — they are the whole point of the
|
|
tier model.
|
|
|
|
## Files
|
|
|
|
```
|
|
.claude/skills/self-check/
|
|
SKILL.md this file
|
|
scripts/self-check.sh the probe engine (report / --json / --publish / fanout / aggregate)
|
|
baseline/manifest.json the provisional fleet baseline (single source of truth)
|
|
baseline/README.md the baseline model + how to refine/ratify it
|
|
.claude/commands/self-check.md the /self-check runner
|
|
```
|
|
|
|
## Extending the baseline
|
|
|
|
When a new tool/skill/command/hook becomes mandatory fleet-wide, edit
|
|
`baseline/manifest.json`, commit, and `/sync`. Every machine's next self-check
|
|
enforces it. Capability-only tools go in `capability_tools` with a matching entry
|
|
in `capability_rules` describing the fallback. See `baseline/README.md`.
|