sync: auto-sync from GURU-5070 at 2026-07-05 16:34:34
Author: Mike Swanson Machine: GURU-5070 Timestamp: 2026-07-05 16:34:34
This commit is contained in:
@@ -0,0 +1,150 @@
|
||||
# Session: GuruRMM policy-controlled tray deployment — diagnose, build, ship, promote
|
||||
|
||||
## User
|
||||
- **User:** Mike Swanson (mike)
|
||||
- **Machine:** GURU-5070
|
||||
- **Role:** admin
|
||||
|
||||
## Session Summary
|
||||
|
||||
Started from a Windows Security screenshot on GURU-5070: "Part of this app has been blocked …
|
||||
can't confirm who published gururmm-tray.exe." Diagnosed it as **Smart App Control (SAC)** in
|
||||
enforcement mode (`VerifiedAndReputablePolicyState=1`), which blocks a trusted parent (the signed
|
||||
agent) from loading an unsigned child. The published tray IS signed (Azure Trusted Signing,
|
||||
CN=Arizona Computer Guru LLC); the blocked file was a stale pre-signing tray dated 2026-05-13.
|
||||
Mike hand-replaced it (elevated). SAC was then found to also be blocking `grok.exe`, `rustc`/`cargo`,
|
||||
`sops`, `openssl` — Mike turned SAC **off** (one-way; needs a Windows reset to re-enable).
|
||||
|
||||
A fleet census (113-host sample via /rmm) showed the blast radius was ~0: the tray was **never
|
||||
deployed fleet-wide** — ~98% of hosts had no tray binary at all; only GURU-BEAST-ROG had a stale
|
||||
unsigned one. That reframed the work: the real requirement (per Mike) is that the tray be
|
||||
**pushed to every machine but only shown when a client policy enables it**, versioned in lockstep
|
||||
with the agent.
|
||||
|
||||
A first inline attempt (fold tray refresh into `do_update`) was rejected by a 17-agent code review
|
||||
(9 defects). The design was then re-shaped with **multi-AI input**: Claude's review + Gemini 3.1 Pro
|
||||
(caught "no network in the SYSTEM watchdog") + Grok/xAI (caught "materialize ≠ enable"; they
|
||||
disagreed on ownership, resolved toward agent-stages/watchdog-installs). Wrote a 4-file spec
|
||||
(`specs/tray-policy-deploy/`) and built the feature across agent + watchdog + tray + server:
|
||||
Tasks 1 (policy gate + default-OFF), 2 (materialize FSM + staging + versioned mutex), 3 (server
|
||||
capability-gate + default-OFF), 5 (observability) + Authenticode verify. Every step compile-verified
|
||||
on Beast (amd64 + i686 + server + tray). Security review clean; a second 17-agent code review found
|
||||
10 findings — the 5 correctness ones fixed (notably the cross-restart cycle and the SCM-poll lock
|
||||
stall), 3 cleanups fixed, 3 documented follow-ups.
|
||||
|
||||
Merged to main and deployed **v0.6.77**. The build initially failed — but only on the **legacy Rust
|
||||
1.77 variant**, which re-resolves deps unpinned and now pulls edition2024 crates (chacha20/rand_core
|
||||
0.10.1) that 1.77 can't parse (BUG-021 recurrence). Proved it reproduces at the pre-feature base, so
|
||||
NOT caused by the change. Per Mike, **disabled the legacy variant** (`LEGACY_ENABLED=false`),
|
||||
preserving existing legacy artifacts at 0.6.76 so 2008R2/Win7 endpoints stay supported. Rebuild
|
||||
succeeded; 0.6.77 shipped + signed. Finally **promoted the agent 0.6.77 to stable** (flipped
|
||||
`.channel` files) and **promoted the dashboard beta → prod** (live at rmm.azcomputerguru.com), both
|
||||
done as `guru` without sudo.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Architecture: agent-staged, watchdog-enforced** — the agent (already SYSTEM + network for
|
||||
self-update) downloads/verifies/stages the tray; the SYSTEM watchdog does only the local
|
||||
install + policy-gated launch. Resolves the Gemini-vs-Grok disagreement (keeps network out of the
|
||||
supervisor without a second trust story).
|
||||
- **Materialize is policy-agnostic; only launch is policy-gated** (Grok's catch) — the binary lands
|
||||
on every machine regardless of `enabled`; the icon shows only when a client policy sets it.
|
||||
- **Default OFF at the leaf** — flipped `system_defaults` tray.enabled false; the agent now ENFORCES
|
||||
the gate, so a true default would light up the whole fleet on first update.
|
||||
- **Separate tray_download_url + server-authenticated checksum** (not a sidecar, not bundled) —
|
||||
reuses the existing publish pipeline + scanner pattern; fixes the CDN-sidecar supply-chain gap.
|
||||
- **Dropped the `revision` field** — review + security review both flagged it as dead state;
|
||||
single-writer + atomic rename makes it unnecessary.
|
||||
- **Deferred Authenticode publisher-pinning + updater/win_swap dedup + tray_stage URL-allowlist** —
|
||||
documented follow-ups; the server-authenticated SHA256 is the implemented primary control.
|
||||
- **Kill the legacy Rust 1.77 variant for now** (Mike) — 2008R2 support returns via a separate
|
||||
legacy agent (C++/.NET or clean rewrite), NOT by fighting the 1.77 + edition2024 treadmill.
|
||||
Existing legacy artifacts preserved (frozen at 0.6.76).
|
||||
|
||||
## Problems Encountered
|
||||
|
||||
- **SAC blocking everything** — Smart App Control enforcing on GURU-5070 blocked tray/grok/rustc/sops.
|
||||
Diagnosed via `HKLM:\SYSTEM\CurrentControlSet\Control\CI\Policy\VerifiedAndReputablePolicyState`.
|
||||
Mike turned SAC off (Windows Security → App & browser control).
|
||||
- **Local MSVC build broken** — post-SAC `rustc` runs but `link.exe` env isn't loaded in Git-bash;
|
||||
used Beast (guru@100.101.122.4) via throwaway `git worktree` + patch + `cargo check` for all verify.
|
||||
- **`git diff` excluded new files** — patches to Beast missed new `.rs` files until switching to
|
||||
`git add -A && git diff --cached <base>`.
|
||||
- **windows 0.58 feature-gates `WINTRUST_DATA`** — hand-declared the C-ABI structs (`#[repr(C)]`) and
|
||||
passed a raw pointer to the windows-rs `WinVerifyTrust` binding.
|
||||
- **Legacy 1.77 build fail-closed the whole Windows build** — edition2024 deps (chacha20/rand_core
|
||||
0.10.1). Proved pre-existing (reproduces at base 9de44d3). Disabled via `LEGACY_ENABLED=false`.
|
||||
- **Git-bash path mangling** — `$SSH "/opt/gururmm/..."` mangled the leading `/opt` to `C:/Program`;
|
||||
fixed with `MSYS_NO_PATHCONV=1 MSYS2_ARG_CONV_EXCL='*'`.
|
||||
- **ClaudeTools push to 172.16.3.20:3000 unreachable** — the memory+gitlink commit is local
|
||||
(later reconciled via fleet sync; `/sync` shows clean).
|
||||
- **sudo needs a password on .30** — did the agent-channel flip + dashboard promote as `guru`
|
||||
(dirs are guru-owned); dashboard promote couldn't append to the root-owned build log (cosmetic).
|
||||
|
||||
## Configuration Changes
|
||||
|
||||
Agent (guru-rmm, all on branch merged to main):
|
||||
- NEW `agent/src/tray_state.rs`, `agent/src/tray_stage.rs`, `agent/src/win_swap.rs`,
|
||||
`agent/src/authenticode.rs`, `agent/src/watchdog/tray_install.rs`
|
||||
- MOD `agent/src/main.rs` (module decls), `agent/src/transport/mod.rs` + `websocket.rs`
|
||||
(TrayConfigUpdate fields + desired-state writer), `agent/src/watchdog/monitor.rs` (gate + FSM +
|
||||
telemetry), `agent/src/watchdog/wts.rs` (session-based terminate_all + tracked_count),
|
||||
`agent/Cargo.toml` (Win32_Security_WinTrust)
|
||||
- MOD `tray/Cargo.toml` (pinned 0.6.76), `tray/src/winsingleton.rs` (versioned mutex)
|
||||
Server:
|
||||
- MOD `server/src/policy/merge.rs` (default-OFF), `config_update.rs` (materialize fields + gate),
|
||||
`server/src/updates/scanner.rs` (tray index), `server/src/ws/mod.rs` (capability-gated send)
|
||||
Pipeline:
|
||||
- MOD `deploy/build-pipeline/build-shared.sh` (tray version lockstep bump),
|
||||
`deploy/build-pipeline/build-windows.sh` (LEGACY_ENABLED=false + preserve legacy artifacts)
|
||||
Spec/docs:
|
||||
- NEW `specs/tray-policy-deploy/{shape,plan,references,standards}.md`
|
||||
- MOD `docs/RMM_THOUGHTS.md` (Feature 12 — legacy rewrite)
|
||||
Memory (ClaudeTools):
|
||||
- NEW `.claude/memory/project_gururmm_legacy_disabled.md` + MEMORY.md index line
|
||||
|
||||
## Credentials & Secrets
|
||||
|
||||
None newly created or discovered. Tray signing uses the existing Azure Trusted Signing SP
|
||||
(`/etc/gururmm-signing.env` on the build server, root-readable only) via jsign — unchanged.
|
||||
|
||||
## Infrastructure & Servers
|
||||
|
||||
- **Build server:** 172.16.3.30 (guru@) — webhook-handler.py orchestrates; build scripts in
|
||||
`/opt/gururmm/`; logs `/var/log/gururmm-build-{windows,server,dashboard,linux}.log`;
|
||||
downloads `/var/www/gururmm/downloads/`; dashboard `/var/www/gururmm/dashboard{,-beta}`.
|
||||
- **Beast (primary Windows build host):** guru@100.101.122.4 (Tailscale); sccache `C:\sccache`;
|
||||
checkout `C:\gururmm`. **Pluto (fallback):** Administrator@172.16.3.36.
|
||||
- **Gitea:** guru-rmm pushes to git.azcomputerguru.com (HTTPS, worked); ClaudeTools origin
|
||||
172.16.3.20:3000 (was unreachable this session).
|
||||
- **Live URLs:** dashboard prod https://rmm.azcomputerguru.com, beta https://rmm-beta.azcomputerguru.com;
|
||||
agent API wss://rmm-api.azcomputerguru.com/ws.
|
||||
|
||||
## Commands & Outputs
|
||||
|
||||
- SAC state: `Get-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Control\CI\Policy" | VerifiedAndReputablePolicyState` → 1 (enforce); after off → 0.
|
||||
- Beast verify loop: `git add -A && git diff --cached <base> > patch` → scp → `git worktree add --detach C:\gururmm-verify <base>` → `git apply` → `cargo +stable check` (amd64 + `--target i686-pc-windows-msvc`).
|
||||
- Legacy repro at base: `move Cargo.lock aside && cargo +1.77 build --features legacy` → `feature edition2024 is required`.
|
||||
- Agent promote: `for f in /var/www/gururmm/downloads/*0.6.77*.channel; do rm -f "$f"; echo stable > "$f"; done`.
|
||||
- Dashboard promote: `MSYS_NO_PATHCONV=1 ssh guru@172.16.3.30 /opt/gururmm/promote-dashboard.sh --confirm` → backup dashboard-20260704-204039, prod==beta verified.
|
||||
|
||||
## Pending / Incomplete Tasks
|
||||
|
||||
- **Enable the tray for a client:** set `tray.enabled=true` on a client policy — until then it
|
||||
deploys dark (default-OFF, by design). Nothing shows an icon yet.
|
||||
- **2008R2/Win7 legacy agent rewrite** (RMM_THOUGHTS Feature 12) — separate C++/.NET or clean
|
||||
rewrite; do NOT re-enable the Rust 1.77 variant as the fix.
|
||||
- **Documented follow-ups (accepted):** Authenticode publisher-pinning; migrate
|
||||
`updater::replace_binary` onto `win_swap` (+ MOVEFILE_DELAY_UNTIL_REBOOT); restore updater's
|
||||
HTTPS/domain-allowlist in `tray_stage`; best-effort desired-state write (C3).
|
||||
- **build-shared.sh tray-lockstep bump** won't take effect until /opt/gururmm/build-shared.sh is
|
||||
manually synced (build-shared self-excludes from the auto-sync); cosmetic (tray file named by
|
||||
agent version regardless).
|
||||
- **Harness (GURU-5070):** `CLAUDETOOLS_ROOT=D:/claudetools` (wrong case) breaks `log-skill-error.sh`.
|
||||
|
||||
## Reference Information
|
||||
|
||||
- **Version shipped:** agent/server/tray 0.6.77 (server 0.3.93). Legacy frozen at 0.6.76.
|
||||
- **Key commits (guru-rmm main):** spec 29b2696; feature 20cb05c/42d4651/b7599fd/9294591; fixes 21097bc; merge f6a4251; version-bump 6b8febc; legacy-disable 84a82a3; RMM_THOUGHTS 23a7bcc. Base 9de44d3.
|
||||
- **Spec:** `projects/msp-tools/guru-rmm/specs/tray-policy-deploy/`.
|
||||
- **Downloads:** `gururmm-{agent-windows-amd64,agent-windows-x86,tray-windows-amd64,agent-base}-0.6.77.*` (stable channel); legacy `*-0.6.76.exe` preserved.
|
||||
Reference in New Issue
Block a user