sync: auto-sync from HOWARD-HOME at 2026-07-06 17:39:50
Author: Howard Enos Machine: HOWARD-HOME Timestamp: 2026-07-06 17:39:50
This commit is contained in:
@@ -19,6 +19,8 @@ Categories (the `[type]` tag): _(none)_ = skill/command execution failure ·
|
||||
|
||||
<!-- Append entries below this line -->
|
||||
|
||||
2026-07-07 | Howard-Home | rmm/windows-edition-upgrade | [correction] hand-rolled changepk Home->Pro instead of following reference_windows_edition_upgrade_rmm; changepk does NOT auto-reboot (flips edition live + sets RebootPending) so the long offline was the flaky RMM agent, not a reboot in progress - must trigger Restart-Computer -Force explicitly [ctx: host=DALLAS ref=reference_windows_edition_upgrade_rmm]
|
||||
|
||||
2026-07-06 | Howard-Home | ssh/gururmm-server | [friction] vault infrastructure/gururmm-server.sops.yaml lists username/password + 'Sudo password same as SSH', but 172.16.3.30 SSH is publickey-only (password auth disabled) and no fleet key authenticates guru/root/mike/howard from workstations. Watching build logs / running promote-dashboard.sh must be done on the box. Fix: add SSH pubkey to guru@.30 or correct the vault note. [ctx: ref=infrastructure/gururmm-server.sops.yaml host=172.16.3.30]
|
||||
|
||||
2026-07-06 | Howard-Home | gururmm-build/webhook | [friction] linux webhook build SKIPPED after merge: change-detector excludes agent/Cargo.toml, so a Cargo.toml-only agent change (BUG-024 rustls swap) never rebuilds; had to force build-linux.sh manually as root [ctx: commit=a73dcad ref=build-linux.sh:29]
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
## User
|
||||
- **User:** Howard Enos (howard)
|
||||
- **Machine:** Howard-Home
|
||||
- **Role:** tech
|
||||
|
||||
## Session Summary
|
||||
|
||||
Started by diagnosing a stray `C:\Program Files\Git\bin\bash.exe` console window on the Howard-Home desktop. Traced it to the `GPS-RMM-AutoEnroll` scheduled task (runs `gps-rmm-autoenroll.sh` every 30 min), which was visible because its principal used `LogonType: Interactive`. Confirmed the running instance was mid-`sleep 150` (normal, after pushing 3 installers), not hung. Changed the principal to `S4U` (runs hidden in Session 0), verified via a test run, then — per Howard's follow-up — disabled the task entirely (manual enrollment going forward) and killed the leftover Session-0 bash processes.
|
||||
|
||||
Built the structure for a fleet-wide client Wi-Fi inventory so techs can connect onsite without asking. Security-driven split: passwords go encrypted in the SOPS vault (`clients/<slug>/wifi.sops.yaml`, `credentials.<key>_ssid/_password`), and a human-readable index (no passwords) lives at `wiki/reference/client-wifi.md` (registered under a new "Reference" section in `wiki/index.md`, exempt from `/wiki-compile`). Per Howard's choices via AskUserQuestion: structure-only for now (the `/wifi` importer is deferred), dedicated wiki doc. Added a `reference_client_wifi_inventory` memory documenting the convention.
|
||||
|
||||
Deployed the BUG-024 fix (Linux agent glibc-floor crash-loop). Verified GURU-5070's `fix/linux-musl-glibc-floor` branch (rustls on non-Windows, `x86_64-unknown-linux-musl` static, objdump build gate), merged it to gururmm `main` (`a73dcad`). Discovered the webhook SKIPPED the Linux rebuild — the change-detector excluded `agent/Cargo.toml`, and the fix was Cargo.toml-only. Forced `build-linux.sh` as root on `.30`; the objdump gate PASSED and published a verified glibc-free static `0.6.79` to beta (Linux un-quarantined). Then fixed the detector gap itself (BUG-025, `fe5468e`) in both `build-linux.sh` and `build-windows.sh`, validated on real commits.
|
||||
|
||||
Attempted to recover the last still-dead Linux agent, Scileppi's SL-SERVER (Synology RS2212+). Howard stood up OpenVPN into Scileppi's network; SL-SERVER (192.168.242.5) became reachable but SSH resets at the banner (sshd restricts to the LAN subnet; my VPN IP 10.137.42.119 is off-subnet), and the vault DSM/user creds all fail DSM login (400, likely stale since the 2025-12-29 migration). The Mac mini jump-host path proved unreliable (T-Mobile public IP, and ScreenConnect returns no command output). Escalated to Mike via Discord DM.
|
||||
|
||||
Closed the session by reviewing the three bugs touched: marked BUG-024 and BUG-025 FIXED in `FEATURE_ROADMAP.md`, then dug into why BUG-024's crash-loop never self-recovered. Root-caused it: the unix rollback watchdog is spawned inside the agent's systemd cgroup (`setsid` changes session, not cgroup), so `KillMode=control-group` (default; unit sets none) SIGKILLs it during the restart it's meant to guard — and it's the only rollback path on unix. Filed as BUG-026 (OPEN/HIGH) and, per Howard, flagged it for Mike to fix (coord todo + Discord DM) rather than implementing it.
|
||||
|
||||
## Key Decisions
|
||||
- Disabled (not deleted) `GPS-RMM-AutoEnroll` so it can be re-run manually; used S4U before deciding to disable, proving the hidden-run path works if re-enabled.
|
||||
- Wi-Fi passwords go in the vault only; the wiki holds a passwordless index. Placed the index under `wiki/reference/` specifically because `/wiki-compile` only touches clients/projects/systems slugs, so it won't be clobbered.
|
||||
- Forced the Linux build manually on `.30` rather than waiting on the webhook, because the change-detector gap meant the webhook would never rebuild a Cargo.toml-only change. Rebuilt as `0.6.79` (same version GURU-5070 validated), relying on the objdump gate as the safety net.
|
||||
- Fixed BUG-025 by counting meaningful `Cargo.toml` lines (excluding only the `[package] version =` line) instead of excluding the manifest wholesale — preserves the version-bump false-positive guard while catching real dep/feature changes.
|
||||
- Did NOT implement the BUG-026 watchdog fix; flagged for Mike per Howard. The fix (systemd-run --scope) touches the self-update path and needs testing on a real systemd host.
|
||||
- Left `0.6.79` on beta to soak; stable promotion is a deliberate separate step.
|
||||
|
||||
## Problems Encountered
|
||||
- **Webhook skipped the BUG-024 Linux build.** `build-linux.sh`/`build-windows.sh` excluded `agent/Cargo.(toml|lock)`; the fix was Cargo.toml-only → `AGENT_CHANGED=0` → skipped. Resolved by forcing the build (cleared `/opt/gururmm/last-built-commit-linux` so the skip guard's `-n "$LAST_SHA"` was false), then fixed the detector (BUG-025).
|
||||
- **Repeated sync races + a MEMORY.md rebase conflict.** GURU-5070 was auto-syncing frequently; three push races and one line-ending conflict in `.claude/memory/MEMORY.md` (Mike's normalized version vs my one-line add). Resolved by `git checkout --ours` + re-adding the single line. Also caught two empty `.ocu.err`/`.ocu.json` scratch files that `git add -A` swept in; removed from tracking and added `.ocu.*` to `.gitignore`.
|
||||
- **Scileppi SL-SERVER unreachable for remediation.** SSH banner-reset (source-subnet restriction), DSM API creds fail (400), Mac mini on a foreign network + no SC command-output retrieval. Not resolvable without Mike changing NAS/VPN access or providing current creds; escalated.
|
||||
- **`/tmp` write blocked by hook.** Wrote a JSON dump to `/tmp/mini.json`; `block-tmp-path.sh` correctly rejected it (Windows path mismatch rule). Switched to the scratchpad.
|
||||
|
||||
## Configuration Changes
|
||||
- `wiki/reference/client-wifi.md` — created (Wi-Fi inventory doc, no passwords).
|
||||
- `wiki/index.md` — added "Reference" section + row.
|
||||
- `.claude/memory/reference_client_wifi_inventory.md` — created; `.claude/memory/MEMORY.md` — index line added (survived a rebase conflict).
|
||||
- `.gitignore` (claudetools) — added `.ocu.*`.
|
||||
- Scheduled task `GPS-RMM-AutoEnroll` (Howard-Home) — principal set to S4U, then task Disabled.
|
||||
- guru-rmm `deploy/build-pipeline/build-linux.sh` + `build-windows.sh` — BUG-025 detector fix.
|
||||
- guru-rmm `docs/FEATURE_ROADMAP.md` — BUG-024 → FIXED, BUG-025 filed FIXED, BUG-026 filed OPEN.
|
||||
- Server `.30`: `/opt/gururmm/last-built-commit-linux` cleared then repopulated; `/var/www/gururmm/downloads/gururmm-agent-linux-amd64-0.6.79` (+ .channel beta + .sha256) published; `/tmp/musl-build` worktree removed; `fix/linux-musl-glibc-floor` remote branch deleted.
|
||||
|
||||
## Credentials & Secrets
|
||||
- SL-SERVER (Scileppi Synology RS2212+) vault entry `clients/scileppi-law/rs2212.sops.yaml`: `credentials.username: sysadmin`, `credentials.password: Gptf*77ttb123!@#-sl-server` (the `\!` in the vault is shell-escape notation), users chris/andrew/sylvia/rose = `Scileppi2025!`. NOTE: **all of these FAIL DSM login (400)** — likely stale since the 2025-12-29 migration. No SSH private key is stored in the entry despite the "SSH key auth configured" note. Flag for update once working access is regained.
|
||||
- gururmm server SSH: `infrastructure/gururmm-server-physical.sops.yaml`, `credentials.username: guru`, `credentials.ssh-private-key` (ed25519), sysadmin password `credentials.password`. Reaches `guru@172.16.3.30` (hostname gururmm). Key extracted to scratchpad during the session and deleted after use.
|
||||
|
||||
## Infrastructure & Servers
|
||||
- GuruRMM build/deploy server: `172.16.3.30` (hostname `gururmm`, Ubuntu 26.04, guru user, SSH key auth). Downloads dir `/var/www/gururmm/downloads`, quarantine `/var/www/gururmm/quarantine`, linux build log `/var/log/gururmm-build-linux.log`, last-built marker `/opt/gururmm/last-built-commit-linux`.
|
||||
- Scileppi: SL-SERVER Synology RS2212+ at `192.168.242.5` (LAN 192.168.242.0/24); DSM API on :5000/:5001 reachable but auth fails; SSH :22 TCP-connects but resets at banner (subnet-restricted). Vault `host` field says 172.16.1.59 (unreachable — stale/mgmt IP). OpenVPN client IP this session: `10.137.42.119`.
|
||||
- Mac mini "Mac-mini-2" (Sylvia, Apple M4) — ScreenConnect session `b9fb0065-a00e-08fe-bdbc-5d9d93c17245`, Company "The Law Offices of Chris Scileppi"; online but public IP `172.59.206.208` (T-Mobile, not Cox), so possibly off the office LAN.
|
||||
|
||||
## Commands & Outputs
|
||||
- `Set-ScheduledTask -TaskName 'GPS-RMM-AutoEnroll' -Principal (New-ScheduledTaskPrincipal -UserId 'Howard' -LogonType S4U -RunLevel Limited)` then `Disable-ScheduledTask`.
|
||||
- Force build: `ssh guru@172.16.3.30 "sudo -S bash -c 'rm -f /opt/gururmm/last-built-commit-linux && /home/guru/gururmm/deploy/build-pipeline/build-linux.sh'"` → `BUG-024 gate PASSED — static musl binary, no glibc floor` → `Linux build complete: v0.6.79`.
|
||||
- Verify static: `objdump -T <bin> | grep -c GLIBC_` = 0; `readelf -l <bin> | grep -c INTERP` = 0; `ldd` = "statically linked"; `file` = "static-pie linked".
|
||||
- Detector validation: BUG-024 change `2471946` → 25 meaningful Cargo.toml lines (build); version bump `c81eeb5` → 0 (skip).
|
||||
- Scileppi SSH: `kex_exchange_identification: read: Connection reset` (system ssh + paramiko), from first attempt. DSM login: all accounts `{"error":{"code":400}}`.
|
||||
|
||||
## Pending / Incomplete Tasks
|
||||
- **Scileppi SL-SERVER recovery (BUG-024 straggler)** — BLOCKED on access. With Mike (Discord DM `1523821940047876146`): needs VPN subnet `10.137.42.0/24` whitelisted for SSH on the NAS (or VPN NAT to LAN), OR current DSM creds, OR the authorized SSH key location. Vault creds `clients/scileppi-law/rs2212` are stale and need updating once in.
|
||||
- **BUG-026** — flagged for Mike (coord todo `f7382334`, Discord DM `1523834569760178259`). Fix: spawn watchdog via `systemd-run --scope --collect` + bounded retry loop + verify agent stays up.
|
||||
- **`0.6.79` stable promotion** — left on beta to soak; promote deliberately later (`/opt/gururmm/promote-*` or the pipeline).
|
||||
- **`/wifi` importer** — deferred; build once a few real networks are captured. Schema in `wiki/reference/client-wifi.md`.
|
||||
|
||||
## Reference Information
|
||||
- gururmm commits: `a73dcad` (BUG-024 merge), `fe5468e` (BUG-025 detector fix), `c248214` (roadmap 024/025), `66152da` (roadmap 026). Branch `fix/linux-musl-glibc-floor` deleted post-merge.
|
||||
- claudetools commits: `c9ea7e8` (wifi structure + .ocu ignore), later syncs `0b11a40`.
|
||||
- Coord: 4 BUG-024 messages marked read (`cf35bf16`, `20daedf9`, `e2b01835`, `9659c5fc`); sent mike `5f8b33f7` (BUG-024 deploy) + `8c45b4dd` (detector fixed); coord todo `f7382334` (BUG-026, mike/gururmm).
|
||||
- Discord DMs to mike: `1523821940047876146` (Scileppi access), `1523834569760178259` (BUG-026).
|
||||
- Vault also synced an unrelated incoming entry `clients/altschuler/gururmm-site-main.sops.yaml`.
|
||||
Reference in New Issue
Block a user