# Session: GuruRMM policy-controlled tray deployment — diagnose, build, ship, promote

## User
- **User:** Mike Swanson (mike)
- **Machine:** GURU-5070
- **Role:** admin

## Session Summary

Started from a Windows Security screenshot on GURU-5070: "Part of this app has been blocked …
can't confirm who published gururmm-tray.exe." Diagnosed it as **Smart App Control (SAC)** in
enforcement mode (`VerifiedAndReputablePolicyState=1`), which blocks a trusted parent (the signed
agent) from loading an unsigned child. The published tray IS signed (Azure Trusted Signing,
CN=Arizona Computer Guru LLC); the blocked file was a stale pre-signing tray dated 2026-05-13.
Mike hand-replaced it (elevated). SAC was then found to also be blocking `grok.exe`, `rustc`/`cargo`,
`sops`, `openssl` — Mike turned SAC **off** (one-way; needs a Windows reset to re-enable).

A fleet census (113-host sample via /rmm) showed the blast radius was ~0: the tray was **never
deployed fleet-wide** — ~98% of hosts had no tray binary at all; only GURU-BEAST-ROG had a stale
unsigned one. That reframed the work: the real requirement (per Mike) is that the tray be
**pushed to every machine but only shown when a client policy enables it**, versioned in lockstep
with the agent.

A first inline attempt (fold tray refresh into `do_update`) was rejected by a 17-agent code review
(9 defects). The design was then re-shaped with **multi-AI input**: Claude's review + Gemini 3.1 Pro
(caught "no network in the SYSTEM watchdog") + Grok/xAI (caught "materialize ≠ enable"; they
disagreed on ownership, resolved toward agent-stages/watchdog-installs). Wrote a 4-file spec
(`specs/tray-policy-deploy/`) and built the feature across agent + watchdog + tray + server:
Tasks 1 (policy gate + default-OFF), 2 (materialize FSM + staging + versioned mutex), 3 (server
capability-gate + default-OFF), 5 (observability) + Authenticode verify. Every step compile-verified
on Beast (amd64 + i686 + server + tray). Security review clean; a second 17-agent code review found
10 findings — the 5 correctness ones fixed (notably the cross-restart cycle and the SCM-poll lock
stall), 3 cleanups fixed, 3 documented follow-ups.

Merged to main and deployed **v0.6.77**. The build initially failed — but only on the **legacy Rust
1.77 variant**, which re-resolves deps unpinned and now pulls edition2024 crates (chacha20/rand_core
0.10.1) that 1.77 can't parse (BUG-021 recurrence). Proved it reproduces at the pre-feature base, so
NOT caused by the change. Per Mike, **disabled the legacy variant** (`LEGACY_ENABLED=false`),
preserving existing legacy artifacts at 0.6.76 so 2008R2/Win7 endpoints stay supported. Rebuild
succeeded; 0.6.77 shipped + signed. Finally **promoted the agent 0.6.77 to stable** (flipped
`.channel` files) and **promoted the dashboard beta → prod** (live at rmm.azcomputerguru.com), both
done as `guru` without sudo.

## Key Decisions

- **Architecture: agent-staged, watchdog-enforced** — the agent (already SYSTEM + network for
  self-update) downloads/verifies/stages the tray; the SYSTEM watchdog does only the local
  install + policy-gated launch. Resolves the Gemini-vs-Grok disagreement (keeps network out of the
  supervisor without a second trust story).
- **Materialize is policy-agnostic; only launch is policy-gated** (Grok's catch) — the binary lands
  on every machine regardless of `enabled`; the icon shows only when a client policy sets it.
- **Default OFF at the leaf** — flipped `system_defaults` tray.enabled false; the agent now ENFORCES
  the gate, so a true default would light up the whole fleet on first update.
- **Separate tray_download_url + server-authenticated checksum** (not a sidecar, not bundled) —
  reuses the existing publish pipeline + scanner pattern; fixes the CDN-sidecar supply-chain gap.
- **Dropped the `revision` field** — review + security review both flagged it as dead state;
  single-writer + atomic rename makes it unnecessary.
- **Deferred Authenticode publisher-pinning + updater/win_swap dedup + tray_stage URL-allowlist** —
  documented follow-ups; the server-authenticated SHA256 is the implemented primary control.
- **Kill the legacy Rust 1.77 variant for now** (Mike) — 2008R2 support returns via a separate
  legacy agent (C++/.NET or clean rewrite), NOT by fighting the 1.77 + edition2024 treadmill.
  Existing legacy artifacts preserved (frozen at 0.6.76).

## Problems Encountered

- **SAC blocking everything** — Smart App Control enforcing on GURU-5070 blocked tray/grok/rustc/sops.
  Diagnosed via `HKLM:\SYSTEM\CurrentControlSet\Control\CI\Policy\VerifiedAndReputablePolicyState`.
  Mike turned SAC off (Windows Security → App & browser control).
- **Local MSVC build broken** — post-SAC `rustc` runs but `link.exe` env isn't loaded in Git-bash;
  used Beast (guru@100.101.122.4) via throwaway `git worktree` + patch + `cargo check` for all verify.
- **`git diff` excluded new files** — patches to Beast missed new `.rs` files until switching to
  `git add -A && git diff --cached <base>`.
- **windows 0.58 feature-gates `WINTRUST_DATA`** — hand-declared the C-ABI structs (`#[repr(C)]`) and
  passed a raw pointer to the windows-rs `WinVerifyTrust` binding.
- **Legacy 1.77 build fail-closed the whole Windows build** — edition2024 deps (chacha20/rand_core
  0.10.1). Proved pre-existing (reproduces at base 9de44d3). Disabled via `LEGACY_ENABLED=false`.
- **Git-bash path mangling** — `$SSH "/opt/gururmm/..."` mangled the leading `/opt` to `C:/Program`;
  fixed with `MSYS_NO_PATHCONV=1 MSYS2_ARG_CONV_EXCL='*'`.
- **ClaudeTools push to 172.16.3.20:3000 unreachable** — the memory+gitlink commit is local
  (later reconciled via fleet sync; `/sync` shows clean).
- **sudo needs a password on .30** — did the agent-channel flip + dashboard promote as `guru`
  (dirs are guru-owned); dashboard promote couldn't append to the root-owned build log (cosmetic).

## Configuration Changes

Agent (guru-rmm, all on branch merged to main):
- NEW `agent/src/tray_state.rs`, `agent/src/tray_stage.rs`, `agent/src/win_swap.rs`,
  `agent/src/authenticode.rs`, `agent/src/watchdog/tray_install.rs`
- MOD `agent/src/main.rs` (module decls), `agent/src/transport/mod.rs` + `websocket.rs`
  (TrayConfigUpdate fields + desired-state writer), `agent/src/watchdog/monitor.rs` (gate + FSM +
  telemetry), `agent/src/watchdog/wts.rs` (session-based terminate_all + tracked_count),
  `agent/Cargo.toml` (Win32_Security_WinTrust)
- MOD `tray/Cargo.toml` (pinned 0.6.76), `tray/src/winsingleton.rs` (versioned mutex)
Server:
- MOD `server/src/policy/merge.rs` (default-OFF), `config_update.rs` (materialize fields + gate),
  `server/src/updates/scanner.rs` (tray index), `server/src/ws/mod.rs` (capability-gated send)
Pipeline:
- MOD `deploy/build-pipeline/build-shared.sh` (tray version lockstep bump),
  `deploy/build-pipeline/build-windows.sh` (LEGACY_ENABLED=false + preserve legacy artifacts)
Spec/docs:
- NEW `specs/tray-policy-deploy/{shape,plan,references,standards}.md`
- MOD `docs/RMM_THOUGHTS.md` (Feature 12 — legacy rewrite)
Memory (ClaudeTools):
- NEW `.claude/memory/project_gururmm_legacy_disabled.md` + MEMORY.md index line

## Credentials & Secrets

None newly created or discovered. Tray signing uses the existing Azure Trusted Signing SP
(`/etc/gururmm-signing.env` on the build server, root-readable only) via jsign — unchanged.

## Infrastructure & Servers

- **Build server:** 172.16.3.30 (guru@) — webhook-handler.py orchestrates; build scripts in
  `/opt/gururmm/`; logs `/var/log/gururmm-build-{windows,server,dashboard,linux}.log`;
  downloads `/var/www/gururmm/downloads/`; dashboard `/var/www/gururmm/dashboard{,-beta}`.
- **Beast (primary Windows build host):** guru@100.101.122.4 (Tailscale); sccache `C:\sccache`;
  checkout `C:\gururmm`. **Pluto (fallback):** Administrator@172.16.3.36.
- **Gitea:** guru-rmm pushes to git.azcomputerguru.com (HTTPS, worked); ClaudeTools origin
  172.16.3.20:3000 (was unreachable this session).
- **Live URLs:** dashboard prod https://rmm.azcomputerguru.com, beta https://rmm-beta.azcomputerguru.com;
  agent API wss://rmm-api.azcomputerguru.com/ws.

## Commands & Outputs

- SAC state: `Get-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Control\CI\Policy" | VerifiedAndReputablePolicyState` → 1 (enforce); after off → 0.
- Beast verify loop: `git add -A && git diff --cached <base> > patch` → scp → `git worktree add --detach C:\gururmm-verify <base>` → `git apply` → `cargo +stable check` (amd64 + `--target i686-pc-windows-msvc`).
- Legacy repro at base: `move Cargo.lock aside && cargo +1.77 build --features legacy` → `feature edition2024 is required`.
- Agent promote: `for f in /var/www/gururmm/downloads/*0.6.77*.channel; do rm -f "$f"; echo stable > "$f"; done`.
- Dashboard promote: `MSYS_NO_PATHCONV=1 ssh guru@172.16.3.30 /opt/gururmm/promote-dashboard.sh --confirm` → backup dashboard-20260704-204039, prod==beta verified.

## Pending / Incomplete Tasks

- **Enable the tray for a client:** set `tray.enabled=true` on a client policy — until then it
  deploys dark (default-OFF, by design). Nothing shows an icon yet.
- **2008R2/Win7 legacy agent rewrite** (RMM_THOUGHTS Feature 12) — separate C++/.NET or clean
  rewrite; do NOT re-enable the Rust 1.77 variant as the fix.
- **Documented follow-ups (accepted):** Authenticode publisher-pinning; migrate
  `updater::replace_binary` onto `win_swap` (+ MOVEFILE_DELAY_UNTIL_REBOOT); restore updater's
  HTTPS/domain-allowlist in `tray_stage`; best-effort desired-state write (C3).
- **build-shared.sh tray-lockstep bump** won't take effect until /opt/gururmm/build-shared.sh is
  manually synced (build-shared self-excludes from the auto-sync); cosmetic (tray file named by
  agent version regardless).
- **Harness (GURU-5070):** `CLAUDETOOLS_ROOT=D:/claudetools` (wrong case) breaks `log-skill-error.sh`.

## Reference Information

- **Version shipped:** agent/server/tray 0.6.77 (server 0.3.93). Legacy frozen at 0.6.76.
- **Key commits (guru-rmm main):** spec 29b2696; feature 20cb05c/42d4651/b7599fd/9294591; fixes 21097bc; merge f6a4251; version-bump 6b8febc; legacy-disable 84a82a3; RMM_THOUGHTS 23a7bcc. Base 9de44d3.
- **Spec:** `projects/msp-tools/guru-rmm/specs/tray-policy-deploy/`.
- **Downloads:** `gururmm-{agent-windows-amd64,agent-windows-x86,tray-windows-amd64,agent-base}-0.6.77.*` (stable channel); legacy `*-0.6.76.exe` preserved.
