claudetools/.claude/docs/graduation-pipeline.md

Scratch Graduation Pipeline (spec)

Status: draft / in progress (2026-06-15). Push side built + tested; triage validated on the flarum test case; scheduled-on-BEAST wiring + execute helper are the remaining work.

Problem

Scratch dirs (tmp/, temp/, .claude/tmp/) are gitignored, so anything in them is invisible to git and lost on cleanup. The old approach — a synchronous tmp-promotion-check.sh run inside /save and /scc — had two fatal flaws:

1. Too slow on Windows. At ~240 scratch files it forked basename/wc/grep -r per file; the "referenced in a session log" check recursed clients/ + projects/ (Rust target/, node_modules/, .git) once per file and hung /save for 4 minutes (errorlog 2026-06-15).
2. Too dumb. Extension/size heuristics can't answer the real question — which of flarum_do_insert.py / do_insert2.py / search_insert.py is canonical, what's a superseded debug dupe, what holds secrets, where each belongs. That's semantic judgment.

The interim tmp-promotion-check.sh is now a fast (0.4s) pure-builtin "N scripts in scratch" nudge. The real triage is offloaded and asynchronous, per this spec.

Architecture

workstation                         BEAST (GURU-BEAST-ROG, best GPU)            any Claude session
-----------                         --------------------------------            ------------------
graduation-push.sh                  Ollama @ :11434 (GPU)                       review proposal
  tar scratch ──SCP/Tailscale──▶  graduation-inbox/<machine>/*.tgz             sanitize secrets
  (soft-fail if BEAST off)          graduation-triage (Ollama classify)        git mv keepers
                                    ─▶ proposal manifest ──coord msg/todo──▶    delete junk → commit

1. Push (graduation-push.sh, built): tars scratch and scps ONE tarball to guru@100.101.122.4:graduation-inbox/<machine>/scratch-<utc>.tgz over Tailscale. Decoupled from /save; soft-fails if BEAST is unreachable. Centralizes every machine's scratch on the GPU box (archive + lets BEAST batch-process even when the origin machine is off).
2. Triage (Ollama on BEAST's GPU): for each file, classify {disposition: graduate|delete|keep-data, canonical?, superseded_by, has_secrets, suggested_home, why}. Emits a proposal manifest (the supersession/secret reasoning the old heuristics couldn't do). The orchestration can run on BEAST (Git-bash, scheduled) or on any machine against BEAST's Ollama API — the GPU is reached over the HTTP API either way.
3. Review + execute: a Claude session (or human) reads the manifest, sanitizes secrets (hardcoded creds → vault lookups), git mvs keepers to permanent homes, deletes junk, commits. Ollama proposes, human/Claude disposes (same contract as memory-dream + the Tier-0 routing rule).

Transport / environment facts (verified 2026-06-15)

• BEAST = guru-beast-rog, Tailscale 100.101.122.4. SSH key auth works as guru (no password).
• BEAST default SSH shell = cmd.exe; home C:\Users\guru. The harness/triage run under Git-for-Windows MSYS bash — NOT WSL. (bash on PATH resolves to the WindowsApps WSL stub; invoke Git-bash explicitly. The WSL stub also can't reach the Windows-host Ollama on localhost — another reason to avoid it.)
• Ollama runs on BEAST's Windows side, bound so it's reachable fleet-wide over Tailscale at http://100.101.122.4:11434. Models incl. qwen3:32b, qwen3.6:latest (36B), gemma3:27b, codestral:22b, qwen3:14b, nomic-embed-text.
• Inbox: C:\Users\guru\graduation-inbox\<machine>\ (cmd path) — per-machine namespaced.

Security (non-negotiable)

• Secrets never enter git. Raw scratch can contain hardcoded creds (the flarum scripts hold the IX root SSH password). It rides the WireGuard-encrypted Tailscale/SSH link and lands ONLY on BEAST (trusted). The transport is deliberately NOT the git repo or a multi-tenant store.
• Sanitize before commit. Any file graduated into a tracked home gets hardcoded secrets swapped for vault lookups first (vault.sh get-field ...). harness-guard would block a plaintext-secret commit.
• Manifest-only returns. Only the proposal manifest comes back toward git — never the raw files.

Components

Piece	Path	State
Push	.claude/scripts/graduation-push.sh	built + tested (241 files → BEAST)
Interim nudge	.claude/scripts/tmp-promotion-check.sh	fast builtin-only (0.4s)
Triage	.claude/scripts/graduation-triage.*	validated ad-hoc on flarum; productize next
Execute	manual (Claude session)	flarum = first test case
Schedule	BEAST cron/loop calling triage	TODO
Return	coord message/todo to origin machine	TODO

Open items

• Productize graduation-triage (general file loop + Ollama classify + manifest) and a --execute helper that sanitizes + git mvs per an approved manifest.
• Wire a scheduled triage run on BEAST (or a /loop) + coord-message return.
• Decide retention/cleanup of the BEAST inbox + auto-deleting obvious junk to keep scratch bounded.
• Consider dropping tmp-promotion-check from /save entirely once the pipeline is routine.