claudetools/specs/claudetools-harness-optimization/review-3way.md

# ClaudeTools Harness Optimization — 3-Way Review Record

Independent reviews on 2026-06-08: Claude (firsthand session friction), Grok (xAI),
Gemini (Google). The analysis converged on the same P0/P1/P2/P3 priorities; the spec
draft was then put back to Grok and Gemini for a review pass. This file records each
model's input, the consensus, the dissent + how it was reconciled, and the resulting
revisions applied to `shape.md`/`plan.md`.

## Consensus (all three, analysis phase)
- Two core weaknesses: distributed-write safety (`git add -A` + submodule + concurrent
  wiki recompiles) and context economics (repeated injection of the skill/command
  registry, CLAUDE.md, big command bodies).
- P0 = reliability footguns (submodule-safe sync, decouple/serialize wiki, SSOT for
  command-vs-standard, guards). P1 = context diet (one-line descriptions, CLAUDE
  CORE/EXTENDED, thin save/sync). P2 = re-tune over-delegation. P3 = tier the knowledge
  layers. Accuracy first; never compress safety/billing/vault rules for tokens.

## Grok — spec review highlights
- GAPS: no fleet rollout/escape-hatch under heterogeneity; no harness VERSION marker;
  coord-lock failure modes unanalyzed; incomplete "where is text injected" inventory;
  discoverability after the diet; no fleet-level success criteria.
- SEQUENCING: Task 3 "confirm with Mike" must be a prerequisite, not inside the edit
  task. Task 2 — build the gated/serialized path and prove it under concurrent load
  BEFORE excising the inline recompile. Task 1+4 touch the same staging path -> same
  branch. Task 9 is a behavioral-contract change under heterogeneity.
- HIGHEST RISK: Task 4 (pre-commit guard) can fail-closed and brick every machine's
  `/save`. De-risk: standalone script, WARN-ONLY for 7-14 days, `SKIP_HARNESS_GUARD=1`
  bypass (logged+broadcast), test matrix, wire to fatal only after a clean warn period.
- CUT/SIMPLIFY: defer Task 8 (shard syncro/rmm) past P1; consolidate per-task verifies
  into one rollout checklist; drop ongoing token telemetry (one-off before/after is
  enough); drop the arbitrary 40-60 memory number.
- ADD: Task 0.5 — `.claude/harness/VERSION` + CORE reference, bumped on every
  fleet-visible behavioral change, so sessions can detect partial rollout.

## Gemini — spec review highlights
- GAPS: coord-lock lifecycle/orphan-eviction undefined; the lazy-load mechanism for
  sharded command files is undefined (models will glob-load them anyway).
- SEQUENCING: put the guard (Task 4) in EARLY so you don't commit the very footguns you
  are fixing while developing Tasks 1-2.
- HIGHEST RISK: Task 2 "wiki merge-not-rewrite" via automation -> progressive structural
  corruption / duplicated headers / hallucinated artifacts. De-risk: a STAGING pattern
  — background job writes proposed updates to `.claude/wiki_staging/` + notifies; an
  operator or focused validation agent reviews + applies the diff.
- CUT/SIMPLIFY: cut Task 11 (Python port) from this spec -> a separate future spec; fix
  Bash determinism first. Simplify Task 10 — NO monolithic `session-logs/INDEX.md`
  (recreates the bloat); use date-folder structure (`session-logs/YYYY-MM/`) + scoped
  grep instead.
- ADD: an out-of-band recovery script (`force-pull-raw.sh`) distributed BEFORE P0, so a
  node can rescue itself if a P0 change bricks `sync.sh`/`save.sh` (the update vector).

## Dissent + reconciliation
1. **Pre-commit guard timing** — Grok: wire late, warn-only (it can brick saves).
   Gemini: put it in first (catch footguns during dev). RESOLVED: build it EARLY as a
   standalone script in **WARN-ONLY** mode (present during Task 1-2 dev, non-fatal),
   with a logged bypass; promote to fatal only after a clean warn period. Satisfies both.
2. **Highest-risk task** — Grok: Task 4 (guard fail-closed). Gemini: Task 2 (wiki
   auto-merge corruption). RESOLVED: both are top risks; address both — guard is
   warn-only, and wiki uses the staging-review pattern (no blind auto-merge).
3. **Session-log INDEX** — Claude proposed a monolithic INDEX; Gemini: that recreates
   bloat. RESOLVED: drop the monolithic index; reorganize logs into `YYYY-MM/` folders
   and recall via scoped grep.
4. **Python port** — Grok accepted it as a P4 stretch; Gemini: cut to a separate spec.
   RESOLVED: this spec lands the deterministic Bash fixes (jq JSON, phoenix-date helper,
   skills-deploy parity) only; a full Python port is a SEPARATE future spec.

## Revisions applied to the spec (see plan.md)
- NEW Task 0.5: harness `VERSION` marker (+ CORE reference) for partial-rollout detection.
- NEW Task 0.6: OOB `force-pull-raw.sh` recovery script, distributed before any P0 change.
- Task 3 split: Task 3a (confirm the billing SSOT truth with Mike) is a hard prerequisite.
- Task 2 rewritten: staging pattern (`.claude/wiki_staging/` + review/apply) + prove
  under concurrent load + coord-lock lifecycle (orphan eviction, coord-down fallback)
  BEFORE removing the inline recompile.
- Task 4 rewritten: standalone guard, WARN-ONLY first, bypass flag, test matrix; built
  early, co-developed with Task 1 on one branch, promoted to fatal only after a clean
  warn window.
- Task 7 constraint: keep any LLM/narrative step OUT of the critical mechanical save/sync
  path (no new failure domain in the hot loop).
- Task 8 (shard syncro/rmm) demoted to P2/deferred; needs a real lazy-load gate to be
  worth it.
- Task 9 gated on fleet harness VERSION (behavioral-contract change).
- Task 10: date-folder logs + scoped grep (no monolithic index); drop the 40-60 number;
  keep graduate + memory-dream.
- Task 11 (Python port) moved OUT of this spec to a future one; deterministic Bash fixes
  stay (jq, phoenix-date, skills-deploy).
- Verification consolidated into one P0/P1 rollout checklist with fleet-level success
  criteria (e.g., 2 consecutive days, zero conflict markers / submodule accidents).
- Added a "where is text injected" inventory pass as a prerequisite to the diet (P1).