sync: auto-sync from GURU-5070 at 2026-06-23 08:17:44

Author: Mike Swanson
Machine: GURU-5070
Timestamp: 2026-06-23 08:17:44

session-logs/2026-06/2026-06-23-mike-rmm-universal-installer.md

@@ -0,0 +1,139 @@
+# GuruRMM Win10-on-ARM install diagnosis -> Universal self-detecting installer (Feature 9 P1 built+deployed)
+
+## User
+- **User:** Mike Swanson (mike)
+- **Machine:** GURU-5070
+- **Role:** admin
+
+## Session Summary
+
+Started as a GuruRMM agent-install failure on a new Win11-suspected box ("Jeff's House",
+site UPPER-FALCON-8240): the generated `install.ps1` threw `ERROR_BAD_EXE_FORMAT` at
+`Start-Process $stagingPath install`. Confirmed the served binary was a valid signed x64 PE,
+so the fault was box-side. Through a sequence of Discord-DM'd diagnostics (the box is reached
+via ScreenConnect's command runner, which is a 32-bit PowerShell host), determined the machine
+is **Windows 10 Pro build 19045 on ARM64 (Snapdragon 8cx)**. Windows 10 on ARM has ONLY x86
+emulation (x64 emulation arrived with Win11-on-ARM), so the x64 agent can never load there.
+Immediate fix: install the existing `windows-x86` variant (verified i386 PE 0x014C, signed),
+delivered by fetching the official install script and swapping `/download/windows` ->
+`/download/windows-x86`. That succeeded; the agent enrolled.
+
+Mike then asked for a "universal" installer (like the MSP360 one) that auto-detects 32/64-bit,
+legacy, and ARM and installs the right agent. Logged the design as RMM_THOUGHTS Feature 9 (and
+Feature 8, native ARM64 build), discussed the approach, captured decisions, and ran `/shape-spec`
+to produce `specs/universal-installer/`. Decisions: build BOTH the arch-aware `/windows` script
+AND a native i386 bootstrapper `.exe` together; Windows-first; replace `/windows` in place;
+online bootstrapper as P1, offline-bundled as P2; priority P1.
+
+On Mike's "build it", implemented P1 across the guru-rmm submodule: the self-detecting `/windows`
+script (Sysnative relaunch + CIM arch/OS detection), a new `installer/bootstrap` i386 crate, a
+`download_bootstrap_exe` server endpoint, landing-page button, and build-pipeline wiring. Pushed
+to the submodule's origin/main; the webhook pipeline built + deployed (server v0.6.71). Verified
+server-side: deployed decision tree correct, `/install/<site>/download/exe` serves a signed i386
+PE with the GRMM_CFG site-code trailer. Mike asked "UI on portal updated?" -> found the dashboard
+portal (EnrollmentModal + Sites pages) still pointed at x64-only downloads; updated them to the
+Universal Installer (primary) with x64/x86 as "force" options. Dashboard built + deployed to BETA
+(verified the rebuilt bundle contains `download/exe`). Advanced the parent ClaudeTools submodule
+pin (was ~30 commits stale) and synced all repos.
+
+## Key Decisions
+- **Win10-on-ARM root cause, not a corrupt download.** The signed x64 binary was fine; the OS
+  simply cannot load an x64 PE (no x64 emulation on Win10 ARM). Confirmed via
+  `Win32_Processor.Name`=Snapdragon 8cx + OS build 19045.
+- **`windows-x86` is the working variant for Win10-ARM** (x86 emulation), not a native ARM64 build.
+  This reframed Feature 8 (native ARM64) from "only option" to a Win11-ARM/perf nice-to-have.
+- **Detect via CIM `Win32_Processor.Architecture` + OS build, never `$env:PROCESSOR_ARCHITECTURE`**
+  (it reports the process arch and reads x86 inside a 32-bit/ScreenConnect host).
+- **Relaunch via `Sysnative`** from a 32-bit host before detecting/installing.
+- **One i386 bootstrapper stub is universal on Windows** (x86 native, x64 WOW64, ARM64 emulation
+  on both Win10 and Win11), so a single binary covers every Windows PC.
+- **Reuse `build_signed_site_exe` + the PE trailer** for the stub (new `bootstrap` platform) — no
+  new signing/embedding machinery.
+- **Replace `/install/<site>/windows` in place** (vs a new `/auto`) so every existing one-liner +
+  the landing page are fixed with nothing to migrate.
+- **Renumbered RMM_THOUGHTS features to 8/9** after discovering origin/main already had Feature 6
+  (multi-vendor security) and Feature 7 (ScreenConnect deploy) — the pinned submodule HEAD I first
+  edited was stale (only had 1-5).
+- **Advance the parent submodule pin to current main** (it was ~30 commits behind), pushed.
+
+## Problems Encountered
+- **ScreenConnect quote-stripping (repeat of a known rule).** First diagnostics handed Mike PS
+  one-liners with embedded double-quotes; ScreenConnect's command runner strips them (same
+  CommandLineToArgvW class as curl.exe/plink). Fixed by switching to `-EncodedCommand` (UTF-16LE
+  base64). Logged to errorlog as `--friction` referencing `feedback_windows_quote_stripping`.
+- **Stale pinned submodule HEAD.** The parent pinned guru-rmm at `2e469f1b` (Features 1-5 only),
+  but real origin/main was 29 commits ahead with its own Features 6/7. Caught before committing;
+  rebased onto origin/main and renumbered my additions to 8/9.
+- **Cross-host submodule auth.** guru-rmm origin is `git.azcomputerguru.com` (Cloudflare-fronted);
+  anon fetch/push fails. Used the vaulted `services/gitea` api-token via a GIT_ASKPASS helper.
+- **No local compile validation.** GURU-5070 lacks the MSVC linker / i686 setup and sqlx offline;
+  per the build-pipeline standard the build host is the validator. Relied on the webhook build +
+  endpoint polling; both server and the new i686 crate compiled cleanly.
+- **`.30` build host is publickey-only** from GURU-5070 (no key, password auth disabled), so build
+  logs were unreachable; monitored via public endpoints (and the beta dashboard bundle) instead.
+- **Dashboard had its own install UI** (separate from the server-rendered landing page) still
+  pointing at x64-only downloads — found only when Mike asked "UI on portal updated?".
+
+## Configuration Changes
+guru-rmm submodule (origin/main, commits a2a0d99 / 4194b0a / 30e14d8 / 53bb682 / b6b283d):
+- NEW `installer/bootstrap/Cargo.toml`, `installer/bootstrap/src/main.rs` (i386 stub crate).
+- MOD `server/src/api/install.rs` — self-detecting `install_script_windows`; `download_bootstrap_exe`;
+  `bootstrap` arm in `site_binary_names` + `build_signed_site_exe` `platform_tag`; landing-page button.
+- MOD `server/src/main.rs` — route `GET /install/:site_code/download/exe`.
+- MOD `deploy/build-pipeline/build-windows.sh` — build (i686) + sign + publish + symlink + cleanup
+  for `gururmm-bootstrap-i386`. (Change-gate already watched `installer/`.)
+- MOD `dashboard/src/lib/install-urls.ts`, `dashboard/src/components/EnrollmentModal.tsx`,
+  `dashboard/src/pages/Sites.tsx` — Universal Installer as primary Windows download.
+- NEW `specs/universal-installer/{plan,shape,references,standards}.md`.
+- MOD `docs/RMM_THOUGHTS.md` — Features 8 (native ARM64) + 9 (universal installer).
+Parent ClaudeTools: submodule pin advanced (commits f8f57d7e, 1bc6c40d); `errorlog.md` friction entry.
+
+## Credentials & Secrets
+- No new secrets created. Used existing vault entries (read-only): `services/gitea`
+  (`credentials.username` + `credentials.api.api-token`) for cross-host submodule push via
+  GIT_ASKPASS; `infrastructure/gururmm-server.sops.yaml` (build host has publickey-only SSH, no
+  usable cred from GURU-5070); `projects/discord-bot/bot-token` (`credentials.bot_token`) to read
+  #dev-alerts. No plaintext secrets committed.
+
+## Infrastructure & Servers
+- GuruRMM server / build host: `172.16.3.30` (gururmm / gururmm-build), publickey-only SSH.
+  Webhook-handler on port 9000; build logs `/var/log/gururmm-build{,-windows,-server,-dashboard}.log`.
+- Downloads served from `https://rmm.azcomputerguru.com/downloads/`; site installers at
+  `https://rmm.azcomputerguru.com/install/<SITE-CODE>` (+ `/windows`, `/download/<variant>`, `/download/exe`).
+- Dashboard: prod `/var/www/gururmm/dashboard/`; BETA `/var/www/gururmm/dashboard-beta` ->
+  `https://rmm-beta.azcomputerguru.com`. Auto-build deploys to BETA only; prod via `promote-dashboard.sh`.
+- guru-rmm git: `git.azcomputerguru.com/azcomputerguru/gururmm.git` (Cloudflare). ClaudeTools git:
+  `172.16.3.20:3000/azcomputerguru/claudetools.git` (internal Gitea). Agent v0.6.71.
+- Field box: UPPER-FALCON-8240 (Jeff's House) = Win10 Pro 19045 on ARM64 (Snapdragon 8cx); running
+  the windows-x86 agent.
+
+## Commands & Outputs
+- Win10-ARM unblock (run via ScreenConnect): `powershell -NoProfile -EncodedCommand <b64>` where the
+  decoded body does `iex (irm '.../install/UPPER-FALCON-8240/windows')` with `/download/windows` ->
+  `/download/windows-x86`.
+- Verify a served PE: download, read machine at e_lfanew+4 (0x014C=i386, 0x8664=x64, 0xAA64=ARM64),
+  `Get-AuthenticodeSignature`.
+- Cross-host submodule push: GIT_ASKPASS helper emitting `services/gitea` username/api-token;
+  `git push origin HEAD:main`.
+- Deployed `/windows` decision tree verified live: ARM64+build>=22000->windows, ARM64+Win10->windows-x86,
+  x64+6.1->windows-legacy else windows, x86 similarly.
+- `/install/UPPER-FALCON-8240/download/exe`: 787,272 bytes, PE 0x014C, sig Valid (CN=Arizona Computer
+  Guru LLC), GRMM_CFG trailer + embedded site code present.
+
+## Pending / Incomplete Tasks
+- **On-box runtime test (Task 8):** run the one-liner + the Universal Installer .exe on real x64,
+  Win10-ARM, Win11-ARM, and legacy boxes; confirm correct variant installs + enrolls.
+- **Dashboard prod promotion:** changes are on BETA (`rmm-beta`); run `promote-dashboard.sh` on .30
+  after review to land them on the prod portal. (Human-gated; not runnable from GURU-5070.)
+- **P2 offline-bundled installer** (specs/universal-installer plan.md Task 7) — not built.
+- **Feature 8 (native ARM64 agent)** — optional Win11-ARM/perf follow-on; not built.
+- Delete plaintext `C:\Users\guru\Downloads\OIT-API.txt` (carryover, Mike to do).
+
+## Reference Information
+- Spec: `projects/msp-tools/guru-rmm/specs/universal-installer/` (plan/shape/references/standards).
+- Backlog: `docs/RMM_THOUGHTS.md` Feature 9 (P1 Built+Deployed), Feature 8 (Raw).
+- guru-rmm commits: a2a0d99 (spec), 4194b0a (P1 impl), 30e14d8 (docs), 53bb682 (dashboard UI),
+  b6b283d (ci-version-bump v0.6.71). Parent: f8f57d7e + 1bc6c40d (submodule pin advances).
+- Endpoints added: `GET /install/:site_code/download/exe` (download_bootstrap_exe). Base binary
+  `gururmm-bootstrap-i386-latest.exe`. Bootstrap crate name `gururmm-bootstrap` (i686-pc-windows-msvc).
+- Memory refs: [[feedback_windows_quote_stripping]], [[feedback_submodule_autosync_discipline]].