feat(harness-guard): FATAL-promotion prerequisite — test matrix + pair-required conflict rule (VERSION 1.4.3)
Builds the false-positive/true-positive proof the plan requires before the guard can be promoted to blocking, and fixes the one false-positive it surfaced. - test-harness-guard.sh: 12-case matrix in a throwaway repo, runs the REAL guard, asserts WARN/clean for real conflicts/secrets/keys vs legit content (setext underlines, dividers, docs that mention a marker, encrypted sops, public keys, .example templates). - harness-guard.sh: conflict rule now requires a real hunk (BOTH ^<<<<<<< AND ^>>>>>>>), dropping the lone =======$ trigger that false-positived on a 7-char setext underline / divider. Identical true-positive power (git writes all three markers); FP surface -> 0. - /self-check: new harness.guard_selftest runs the matrix in an isolated temp repo (read-only vs the real tree) so guard correctness is continuously proven. Verified 12/12 pass, true positives intact, real-tree FP surface = 0. FATAL flip (todo f1c11d0d, on/after 2026-06-22) is now evidence-backed + one-step. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -81,7 +81,7 @@ SELFCHECK_TS="$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
|
||||
| **skills/commands** | every skill dir and command file in the baseline is present; extras are reported as census candidates. |
|
||||
| **duplicates** | command/skill names present in BOTH the repo and `~/.claude`. Divergent content = WARN (the "same `/cmd`, different behaviour on the Mac" bug); identical = INFO (redundant, will drift). CRLF-only differences are ignored. |
|
||||
| **memory** | `MEMORY.md` index exists; no orphaned memory files; manifest-declared contradiction patterns (see semantic pass below). Never FAILs the grade. |
|
||||
| **harness** | the 1.4.0 invariants (read-only): VERSION marker present + not older than `manifest.harness.min_version`; **skill-registry description budget** (sum of all SKILL.md `description:` fields under `registry_desc_budget_chars` — WARN on regrowth); global deploy targets `~/.claude/skills` + `~/.claude/commands` populated (the "Mac wiped global skills" failure); `harness-guard.sh` present + wired into `sync.sh`; core scripts parse (`bash -n` on sync/guard/now-phoenix); `now-phoenix.sh --date` emits a valid date. Budget/min-version/script-list are tunable in `manifest.harness`. |
|
||||
| **harness** | the 1.4.0 invariants (read-only): VERSION marker present + not older than `manifest.harness.min_version`; **skill-registry description budget** (sum of all SKILL.md `description:` fields under `registry_desc_budget_chars` — WARN on regrowth); global deploy targets `~/.claude/skills` + `~/.claude/commands` populated (the "Mac wiped global skills" failure); `harness-guard.sh` present + wired into `sync.sh`; core scripts parse (`bash -n`); `now-phoenix.sh --date` emits a valid date; **guard self-test** runs the full `test-harness-guard.sh` false-positive/true-positive matrix in an isolated temp repo (proves the guard still catches real conflicts/secrets and does not false-positive — the standing prerequisite for promoting the guard to FATAL). Budget/min-version/script-list are tunable in `manifest.harness`. |
|
||||
| **consistency** | the **command-restates-standard** lint (deterministic half): for each `manifest.command_standard_links` pair, the standard must still contain its defer-to-SSOT pointer to the owning command. A lost pointer = WARN (the standard likely drifted back into restating the command — the Syncro-timers failure mode). The semantic contradiction judgement is delegated to the model (see below). |
|
||||
| **vault** | vault repo exists; sops+age present; `vault.sh list` succeeds (decrypt wired). |
|
||||
| **connectivity** | coord API (required), main API + internal Gitea (advisory; off-network is OK). |
|
||||
|
||||
@@ -13,7 +13,8 @@
|
||||
"syntax_check_scripts": [
|
||||
".claude/scripts/sync.sh",
|
||||
".claude/scripts/harness-guard.sh",
|
||||
".claude/scripts/now-phoenix.sh"
|
||||
".claude/scripts/now-phoenix.sh",
|
||||
".claude/scripts/test-harness-guard.sh"
|
||||
],
|
||||
"guard_wired_in": ".claude/scripts/sync.sh"
|
||||
},
|
||||
|
||||
@@ -659,6 +659,21 @@ check_harness_smoke() {
|
||||
"Check .claude/scripts/now-phoenix.sh"
|
||||
fi
|
||||
fi
|
||||
|
||||
# 6. Guard self-test: run the full false-positive/true-positive matrix in an isolated
|
||||
# temp repo (writes only under mktemp, never the real tree). Proves the guard still
|
||||
# detects real conflicts/secrets AND does not false-positive on legit content — the
|
||||
# standing prerequisite for promoting the guard to FATAL.
|
||||
local gt="$REPO_ROOT/.claude/scripts/test-harness-guard.sh" gres
|
||||
if [ -f "$gt" ] && command -v git >/dev/null 2>&1; then
|
||||
gres="$(bash "$gt" 2>/dev/null | grep 'RESULT:' | head -1 | sed 's/^[[:space:]]*RESULT:[[:space:]]*//')"
|
||||
if echo "$gres" | grep -q 'FAIL 0'; then
|
||||
emit harness.guard_selftest harness PASS "guard FP/TP matrix clean ($gres)"
|
||||
elif [ -n "$gres" ]; then
|
||||
emit harness.guard_selftest harness WARN "guard self-test reported failures ($gres)" \
|
||||
"Run: bash .claude/scripts/test-harness-guard.sh — a detection case regressed"
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
Reference in New Issue
Block a user