claudetools/session-logs/2026-06/2026-06-18-mike-sync-submodule-autoheal-and-thoughts-rescue.md

User

• User: Mike Swanson (mike)
• Machine: GURU-BEAST-ROG
• Role: admin

Session Summary

Continuation of the error-logging session started 2026-06-14 (see session-logs/2026-06/2026-06-14-mike-errorlog-rule.md). This span added a non-destructive auto-heal for the recurring sync.sh submodule abort, then used it to rescue real un-committed design content the abort had been masking.

Mike asked to fix the git submodule update abort that sync.sh hit during Phase 3 (post-rebase submodule reconcile). Root cause: when the parent's incoming gitlink points a submodule at a commit that TRACKS a file the submodule's working tree holds as UNTRACKED, git submodule update aborts ("untracked working tree files would be overwritten by checkout"). It was non-fatal (sync still exited 0) but left the submodule pointer stale and recurred every sync. The live case was projects/msp-tools/guru-rmm stuck at ed92097 while the parent wanted f38da05, blocked by an untracked docs/RMM_THOUGHTS.md.

Investigated before patching: confirmed the blocking file was a divergent local copy (13091 bytes) vs the committed version f38da05 carries (29621 bytes). A blind git submodule update --force would have discarded the local file — wrong, because RMM_THOUGHTS.md is human idea-capture. Built resolve_submodule_collisions() in sync.sh: on the abort it enumerates each submodule, computes the intersection of (untracked files) x (paths the target gitlink commit tracks) = exactly the files that would block checkout, and moves ONLY those aside to <file>.synced-aside-<UTCstamp> (content preserved, locale- independent, never --force), then retries the update once. Wired into the Phase 3 failure branch with a single retry; refined to capture the first attempt's output and only surface the raw git error if the resolve+retry also fails (clean happy-path output). Dry-ran the detection read-only first (targeted exactly guru-rmm's RMM_THOUGHTS.md, left the other two submodules untouched), then ran the real sync: guru-rmm advanced ed92097 -> f38da05, the local copy preserved as docs/RMM_THOUGHTS.md.synced-aside-20260615T132540Z. Pushed and broadcast to the fleet.

The non-destructive choice paid off immediately. Diffing the aside backup against the now- canonical 29621-byte version showed the local copy held 94 unique non-blank lines — a coherent "Re-grounding pass — 2026-06-08" plus 8 Raw thought sections (Users->Settings + partner scoping, command provenance split, left-nav/IA rationalization, Settings agent- deploy block, Watchdog Alerts redundancy, Updates->Settings, Reports hub, AgentDetail offline-staleness DEFECT) — all ABSENT from the committed doc, i.e. genuinely orphaned, not superseded. A --force would have erased them. Mike chose "merge + stage, you review", so appended the aside's lines 11-182 (the 2026-06-08 block, skipping the duplicate header) to the canonical RMM_THOUGHTS.md via shell append (preserves the canonical's exact bytes, including a non-UTF8 char that makes git treat it as binary), then git added it in the submodule WITHOUT committing or pushing. Left for Mike to review the staged diff (+174 lines, 9 sections) and commit in the submodule's own flow.

Also surfaced this span: an incoming GURU-KALI broadcast (py.sh interpreter resolver, building directly on the $CLAUDETOOLS_ROOT fix from 2026-06-14), and the observation that the errorlog matured fleet-wide — a log-skill-error.sh helper plus [correction] / [friction] categories, now logging across GURU-5070, Howard-Home, and GURU-KALI.

Key Decisions

• Move-aside, never --force. The blocking file is human idea-capture; discarding it to unblock a checkout is unacceptable. Move only the precise colliding untracked files to a timestamped backup, preserving content. Validated by the 94 rescued lines.
• Compute collisions by tree intersection, not by parsing git's English error. For each submodule: untracked files INTERSECT paths tracked by the target gitlink commit. Locale- independent and precise — only moves files that would actually block checkout.
• Capture first-attempt output; only print raw git error if retry also fails. Avoids the alarming "fatal: Unable to checkout" appearing right before "SUCCESS" on the auto-healed happy path.
• Stage the thoughts rescue, do not commit/push. It is Mike's idea doc in a submodule with its own commit flow + pre-commit hooks; placement and commit are his call.
• Append rescued block verbatim with only a --- separator. No editorial insertions — keep Mike's voice; the block is self-documenting (every section dated 2026-06-08 / Raw).

Problems Encountered

• Process substitution <(...) as a command argument fails in this Git-bash (Could not access '/proc/<pid>/fd/63') — hit when diffing git show output. Worked around with temp files. Note: input redirection done < <(...) DOES work in the sync.sh execution context (already used at lines 314/361), so the resolver safely uses that form.
• Near-miss: Write to settings.local.json would have clobbered an existing permissions block (the file existed despite an earlier grep miss). The Read-before-Write guard caught it; switched to an Edit that preserved the block. (From the 2026-06-14 envroot work.)
• errorlog.md "modified since read" on Edit — the fleet upgraded the file (helper + categories) and other machines appended entries between my read and edit. Re-read, then re-applied the RESOLVED mark.

Configuration Changes

• Modified: .claude/scripts/sync.sh — added resolve_submodule_collisions() helper (after reconcile_git_identity); rewired the Phase 3 submodule-update failure branch to resolve + retry once, with captured-output (raw error only on final failure).
• Modified: errorlog.md — marked the guru-rmm submodule-abort entry [RESOLVED 2026-06-15].
• Modified (STAGED, NOT committed): projects/msp-tools/guru-rmm/docs/RMM_THOUGHTS.md — +174 lines (rescued 2026-06-08 re-grounding pass + 8 Raw sections). Staged in the submodule for Mike's review.
• Untracked (preserved backup, in submodule): docs/RMM_THOUGHTS.md.synced-aside-20260615T132540Z.

Earlier in the same overall session (2026-06-14, logged separately): .claude/CLAUDE.md error-logging rule, errorlog.md created, .claude/scripts/ensure-settings-env.py, .claude/settings.local.json env, .claude/ONBOARDING.md env row.

Credentials & Secrets

None created, discovered, or rotated this session.

Infrastructure & Servers

• Coord API: http://172.16.3.30:8001/api/coord (broadcasts + todos).
• Submodule: projects/msp-tools/guru-rmm — advanced ed92097bb61c... -> f38da05b3535... (working tree; parent gitlink intentionally lags per sync.sh design).
• guru-rmm origin: git.azcomputerguru.com/azcomputerguru (Gitea).

Commands & Outputs

• Read-only collision dry-run (intersection of untracked x target-tree) correctly flagged only docs/RMM_THOUGHTS.md in guru-rmm; guru-connect and youtube-sync-docker clean.
• Live heal: [WARNING] projects/msp-tools/guru-rmm: untracked 'docs/RMM_THOUGHTS.md' collided with incoming commit; preserved as 'docs/RMM_THOUGHTS.md.synced-aside-20260615T132540Z' -> [OK] Submodules reconciled (set aside colliding untracked file(s)). Sync exit 0.
• git diff --cached --stat (submodule): docs/RMM_THOUGHTS.md | 174 +++.. 1 file, 174 insertions.
• Orphan-content check: all 5 distinctive 2026-06-08 topics returned ABSENT from canonical before the merge; grep -c "Re-grounding pass — 2026-06-08" = 1 after.

Pending / Incomplete Tasks

• Mike to review + commit the staged RMM_THOUGHTS.md rescue in the guru-rmm submodule: cd projects/msp-tools/guru-rmm && git diff --cached docs/RMM_THOUGHTS.md then git commit + git push (runs submodule pre-commit hooks; advances guru-rmm main). Delete docs/RMM_THOUGHTS.md.synced-aside-20260615T132540Z once satisfied.
• Fleet machines pick up the sync.sh auto-heal on next pull + /sync (broadcast sent; no per-machine action).

Reference Information

• Coord broadcasts this session: errorlog rule 5514a2d3, envroot fix 8e1dec6a (todo 5c51cad2), submodule auto-heal 091025d7. (All 2026-06-14/15.)
• Incoming broadcasts addressed: GURU-KALI tmp/ gitignore (f495b08f), GURU-KALI py.sh resolver (9b1c5c39).
• Key files: .claude/scripts/sync.sh (resolve_submodule_collisions), projects/msp-tools/guru-rmm/docs/RMM_THOUGHTS.md, errorlog.md, .claude/scripts/ensure-settings-env.py.
• Prior log this session: session-logs/2026-06/2026-06-14-mike-errorlog-rule.md.