wiki: compile lonestar-electrical (full) — Unraid VM-loss root cause (/boot/config/go hijack), IP fix 172.16.1.188->192.168.120.177, hours 0
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -2,14 +2,18 @@
|
||||
type: client
|
||||
name: lonestar-electrical
|
||||
display_name: Lone Star Electrical Systems LLC
|
||||
last_compiled: 2026-06-03
|
||||
compiled_by: Mikes-MacBook-Air/claude-main
|
||||
last_compiled: 2026-07-06
|
||||
compiled_by: GURU-5070/claude-main
|
||||
sources:
|
||||
- clients/lonestar-electrical/session-logs/2026-07/2026-07-06-mike-unraid-vm-recovery.md
|
||||
- clients/lonestar-electrical/session-logs/2026-06-03-session.md
|
||||
- clients/lonestar-electrical/session-logs/2026-06-02-session.md
|
||||
- clients/lonestar-electrical/session-logs/2026-06-01-session.md
|
||||
- clients/lonestar-electrical/session-logs/2026-05-29-sophos-removal.md
|
||||
- clients/lonestar-electrical/docs/apple-mdm-setup-reference.md
|
||||
- session-logs/2026-06/2026-06-10-mike-lonestar-unraid-vm-restore.md
|
||||
- session-logs/2026-06/2026-06-10-mike-kittle-mfa-passkey-lonestar-rdp.md
|
||||
- session-logs/2026-06-01-mike-lonestar-sophos-coordination.md
|
||||
- session-logs/2026-03-23-session.md
|
||||
- session-logs/2026-03-24-session.md
|
||||
- credentials.md
|
||||
@@ -19,6 +23,7 @@ sources:
|
||||
- temp/lonestar-russ-setup.py
|
||||
- temp/lonestar-kyla-reset.py
|
||||
- temp/lonestar-kyla-2fa-fix.py
|
||||
- Syncro live: GET /customers/33809612 (2026-07-06)
|
||||
backlinks: []
|
||||
aliases: [lonestar]
|
||||
---
|
||||
@@ -32,26 +37,36 @@ Electrical contractor in Tucson, AZ. ACG-managed client. Distinctive in the flee
|
||||
## Profile
|
||||
|
||||
- **Company type:** Electrical contractor (field service)
|
||||
- **Contract type:** Prepaid hour block
|
||||
- **Hours remaining:** 13.5 hrs as of 2026-06-02 (Syncro live — always re-check `GET /customers/33809612` before billing).
|
||||
- **Billing rate:** (verify — check recent Syncro invoices; not captured in available sources)
|
||||
- **Contract type:** Prepaid hour block — **depleted as of 2026-07-06 (0 hrs remaining); effectively break-fix now.** Re-check `GET /customers/33809612` before any billing and confirm with the client whether the block will be replenished or the account moves formally to break-fix/hourly.
|
||||
- **Hours remaining:** **0 hrs as of 2026-07-06** (Syncro live — down from 13.5 hrs recorded 2026-06-02; no intervening top-up documented in available sources — verify what consumed the remaining balance).
|
||||
- **Billing rate:** (verify — not captured in available sources)
|
||||
- **Syncro customer ID:** `33809612` (Lone Star Electrical Systems LLC)
|
||||
- **Address:** 3774 North Warren Avenue, Tucson, AZ
|
||||
- **Managed assets (Syncro):** 1 asset on record
|
||||
- **Sites:** Norris site (location of the LS-1 / LS-2 Win11 workstations)
|
||||
- **Sites:** Norris site (LS-1 / LS-2 Win11 workstations); Warren site (Unraid server "Tower" + LONESTAR-VM, LAN `192.168.120.0/24`)
|
||||
- **Key contacts:**
|
||||
- Robin Eneix — robine@lonestarelectrical.net (Syncro primary contact)
|
||||
- Robin Eneix — robine@lonestarelectrical.net, 520-248-8436 (AZ ROC #318060 CR-11) — office manager / billing + scheduling; Syncro primary contact as of prior compiles
|
||||
- Amanda B — amandab@lonestarelectrical.net — contact seen in Syncro as of 2026-07-06 (verify role vs. Robin Eneix)
|
||||
- Jose R. (joser@lonestarelectrical.net) — field user; subject of the 2026-03 personal-phone MDM issue
|
||||
- sysadmin@lonestarelectrical.net — Google Workspace admin account (ACG-managed)
|
||||
- James — account compromised 2026-03-10 (Syncro #32010); [verify current name/role]
|
||||
- Kyla, Russ — GWS user accounts touched via provisioning/2FA scripts (temp/); [verify roles]
|
||||
- Main phone on file (Syncro): 520-730-3642
|
||||
- **Active ticket:** None open in Syncro as of 2026-06-02
|
||||
- **Active ticket:** None open in Syncro as of 2026-07-06
|
||||
|
||||
---
|
||||
|
||||
## Infrastructure
|
||||
|
||||
### Servers & Services
|
||||
|
||||
| Host | IP | Role | OS | Notes |
|
||||
|---|---|---|---|---|
|
||||
| Tower | 192.168.120.177 (Warren LAN) | Unraid server — array + VM host | Unraid 7.1.4 | Vault entry's `172.16.1.188` is STALE (temporary ACG-office address during 2026-06 USB migration). No active UPS. |
|
||||
| LONESTAR-VM ("Windows 11" domain) | 192.168.120.197/24 | Windows VM (guest on Tower), GuruRMM-managed | Windows | Recovered + autostart 2026-07-06 after libvirt root-cause fix (see Patterns) |
|
||||
| LS-1 | (Norris site, verify) | Workstation | Windows 11 | Sophos removed 2026-06-02; GuruRMM-managed |
|
||||
| LS-2 | (Norris site, verify) | Workstation | Windows 11 | Sophos removed 2026-06-02; GuruRMM-managed |
|
||||
|
||||
### Email & Identity
|
||||
|
||||
- **Platform:** Google Workspace (domain `lonestarelectrical.net`). NOT Microsoft 365 — the M365 remediation tool does not apply here.
|
||||
@@ -64,6 +79,7 @@ Electrical contractor in Tucson, AZ. ACG-managed client. Distinctive in the flee
|
||||
- **Platform:** ManageEngine MDM (Zoho) — https://mdm.manageengine.com/webclient
|
||||
- **MDM admin:** mike@azcomputerguru.com (Zoho account, Super Admin)
|
||||
- **Enrolled devices:** 2 company tablets (named **Zach** and **JOSE**), enrolled 2025-12-04 via QR code, fully managed. These are direct enrollments and are unaffected by the Google third-party-EMM integration.
|
||||
- **Apple/iOS enrollment (planned, not yet executed):** reference plan compiled 2026-05-27 for enrolling the client's iPhone/iPads into the same ManageEngine tenant — requires an APNs certificate upload and targeted invite/QR enrollment (NOT re-enabling blanket self-enrollment, which was deliberately disabled 2026-03-24). Full checklist: `clients/lonestar-electrical/docs/apple-mdm-setup-reference.md`. Status of execution: [verify — not confirmed complete in available sources].
|
||||
|
||||
### Workstations
|
||||
|
||||
@@ -71,26 +87,27 @@ Electrical contractor in Tucson, AZ. ACG-managed client. Distinctive in the flee
|
||||
- **LS-1 GuruRMM agent:** `6b9617fa-5c77-40e1-8b64-a1545e730895`
|
||||
- **LS-2 GuruRMM agent:** `97fe5582-aa3d-4132-94a6-f4c8582bca31`
|
||||
- **Windows Defender:** active and real-time protection enabled on both as of 2026-06-02.
|
||||
- **Current location (2026-06-02):** both desktops are physically at the **Computer Guru office** for repair (hence they appear on the ACG `172.16.0.0/22` network). To be taken back **onsite to Norris and reconnected the week of 2026-06-08**.
|
||||
- **Location:** returned onsite to Norris the week of 2026-06-08 per plan (prior compile had both at the ACG office for repair; current onsite status [verify] — no confirmation logged since).
|
||||
|
||||
### Virtual Machine — LONESTAR-VM
|
||||
|
||||
- **Host:** `LONESTAR-VM` — Windows VM at the Warren site. **GuruRMM-managed** (agent `a4d39a9d-2210-483c-9b1e-6348efdba627`, v0.6.54, online as of 2026-06-10). Discovered via the GuruRMM fleet on 2026-06-10 (was not previously documented). Also reachable via ScreenConnect.
|
||||
- **IP:** `192.168.120.197/24` (interface "Ethernet 2"), gateway `192.168.120.1`. **Warren-site LAN is `192.168.120.0/24`** — the live production subnet, distinct from the `172.16.x` ACG-office addresses LS-1 / LS-2 / Tower carried while they were in the shop for repair.
|
||||
- **Host:** `LONESTAR-VM` (libvirt domain name **"Windows 11"**) — Windows VM guest on the **Tower** Unraid server at the Warren site. **GuruRMM-managed** (agent `a4d39a9d-2210-483c-9b1e-6348efdba627`, v0.6.54 as of 2026-06-10). Also reachable via ScreenConnect.
|
||||
- **IP:** `192.168.120.197/24` (interface "Ethernet 2"), gateway `192.168.120.1`. **Warren-site LAN is `192.168.120.0/24`** — the live production subnet.
|
||||
- **Network profile:** Private.
|
||||
- **RDP:** Enabled. Windows Defender Firewall "Remote Desktop" rules (TCP/UDP 3389 + Shadow) are Enabled and scoped **Domain + Private** — tightened off the `Any`/Public profile on 2026-06-10. RDP listening is on (`fDenyTSConnections=0`). Reachable on the Warren LAN / site VPN, not over a Public network.
|
||||
- **RDP:** Enabled. Windows Defender Firewall "Remote Desktop" rules (TCP/UDP 3389 + Shadow) tightened to **Domain + Private** on 2026-06-10 (off the prior `Any`/Public profile). RDP listening is on (`fDenyTSConnections=0`).
|
||||
- **Vdisk:** `/mnt/user/domains/Windows 11/vdisk1.img` (64 GiB).
|
||||
- **Stability history:** offline 2026-06-06 to 2026-06-10 (restored via manual libvirt/TPM repair — see History); recurring "`/VMs` page empty / VM disappears" through early July, **root-caused and fixed 2026-07-06** — see Patterns below. Recovered + **autostart enabled** as of 2026-07-06; clean-reboot validation was scheduled for the night of 2026-07-06 to confirm the fix holds through a full stock boot.
|
||||
|
||||
### Unraid Server
|
||||
|
||||
- **Status:** Running Unraid **7.1.4** as of 2026-06-02 (migrated to new USB flash drive). **GuruRMM agent enrolled 2026-06-03.**
|
||||
- **Hostname:** Tower
|
||||
- **LAN IP:** 172.16.1.188 (currently at ACG office, to be returned to client site week of 2026-06-08)
|
||||
- **LIVE LAN IP: `192.168.120.177`** (Warren site, `192.168.120.0/24`), reached over the client VPN. **The vaulted IP `172.16.1.188` is STALE** — that address only applied while the box was temporarily at the ACG office for the 2026-06 USB migration; it has since returned to the Warren site. Correct the vault entry.
|
||||
- **Status:** Unraid **7.1.4**, kernel 6.12.24, libvirt 11.2.0 (monolithic libvirtd). Array: 2 XFS data disks + parity (STARTED). Cache: single btrfs pool on a Crucial MX500 500GB SSD (`/dev/sdb1`), SMART PASSED.
|
||||
- **Power protection:** **no active UPS** on this box as of 2026-07-06 — flag as a risk item (unclean shutdowns/parity risk on outage).
|
||||
- **Boot device:** USB flash drive replaced 2026-06-02 (Unraid USB Creator, 7.1.4) after a recurring `bzfirmware` checksum failure on the original stick; **registered and confirmed stable**. Old failing stick (label `UNRAID`, `/dev/sda1`, Generic Flash Disk 8GB) retired.
|
||||
- **License type:** [verify — Basic / Plus / Pro]
|
||||
- **Boot device:** New USB flash drive (written via Unraid USB Creator, 7.1.4), **registered and confirmed stable**. The original failed stick (label `UNRAID`, `/dev/sda1`, Generic Flash Disk 8GB) has been **RETIRED** as of 2026-06-02.
|
||||
- **Config:** Old `config/` folder (array assignments `super.dat`, shares, network settings, license `.key`) copied from the failing stick onto the new one. Disk layout and array configuration preserved; only the OS files are fresh.
|
||||
- **License:** Re-registered to the new USB GUID via Unraid webGUI Tools > Registration > Replace Key on 2026-06-02.
|
||||
- **Root credentials:** Vaulted at `clients/lonestar-electrical/unraid-server.sops.yaml` as of 2026-06-03.
|
||||
- **GuruRMM agent:** Docker container `gururmm-agent` installed 2026-06-03 (agent ID `e827f798-bab1-484b-b641-98da7ff5af87`, version 0.6.28, status: online). Container uses host networking, auto-restart policy, and proper volume mounts for device-id persistence (`/mnt/user/appdata/gururmm/lib`).
|
||||
- **Root credentials:** Vaulted at `clients/lonestar-electrical/unraid-server.sops.yaml` (root / SSH port 22 standard OpenSSH) — do not inline the password; read via the vault skill. **The `ip` field in this vault entry is stale (`172.16.1.188`) and needs correcting to `192.168.120.177`.**
|
||||
- **GuruRMM agent (host):** Docker container `gururmm-agent` deployed 2026-06-03/04 (host networking, auto-restart, volume mounts for device-id persistence at `/mnt/user/appdata/gururmm/lib`). RMM agent IDs seen: `e827f798-bab1-484b-b641-98da7ff5af87` (vault-noted) and a host ID `00b48379-...` seen offline 2026-07-06. **As of 2026-07-06 this container was crash-looping**, leaving the host RMM agent offline; being handled by a separate session — verify resolution before relying on host-level RMM alerting for this box.
|
||||
- **Array/disk layout:** [verify — deferred to future documentation]
|
||||
|
||||
---
|
||||
@@ -101,13 +118,22 @@ Electrical contractor in Tucson, AZ. ACG-managed client. Distinctive in the flee
|
||||
- **ManageEngine MDM:** mike@azcomputerguru.com (Zoho Super Admin) — https://mdm.manageengine.com/webclient
|
||||
- **GWS service account (programmatic):** `ACG-MSP-Access (Google Workspace)` (vault: MSP Tools); key file `temp/acg-msp-access-8f72339997e5.json`
|
||||
- **Vault root:** `clients/lonestar-electrical/` in vault repo
|
||||
- **Unraid server:** root credentials vaulted at `clients/lonestar-electrical/unraid-server.sops.yaml` (2026-06-03)
|
||||
- **Unraid server (Tower):** root credentials vaulted at `clients/lonestar-electrical/unraid-server.sops.yaml` — **live IP is `192.168.120.177`, not the `172.16.1.188` currently recorded in that vault entry (correct on next vault touch)**. Reached over the client VPN; `plink -batch -pw` (PuTTY) is the known-working non-interactive path from GURU-5070 since system OpenSSH lacks non-interactive password auth and no `sshpass` is installed there.
|
||||
- **GuruRMM Warren site:** enrollment key vaulted at `clients/lonestar-electrical/gururmm-site-warren.sops.yaml` (site code: WARM-FALCON-5891)
|
||||
- **Unraid web UI:** `http://192.168.120.177` (LAN/VPN only)
|
||||
|
||||
---
|
||||
|
||||
## Patterns & Known Issues
|
||||
|
||||
- **Unraid "Tower": recurring `/VMs` page empty / VM vanishes — ROOT-CAUSED and FIXED 2026-07-06.** This was NOT a storage or hardware problem (vdisk, nvram, swtpm, and the cache SSD all checked out healthy every time). The actual cause was a **leftover hand-hack in `/boot/config/go`**, labeled `"LibVirt Fix (final 2026-06-10)"`, added during an earlier repair attempt on 2026-06-10. On every boot it wrote `/etc/libvirt/*.conf` files and manually started `libvirtd` against the still-**unmounted** `/etc/libvirt` path, racing ahead of `emhttpd`. This hijacked libvirt before Unraid's own array/emhttpd process could mount `libvirt.img` there, so emhttpd's mount then failed and the VM(s) disappeared from the UI. **A prior "fix" was the actual cause of the recurrence** — which is why it only ever happened on this box and why earlier repairs (manual libvirt.img mount, qemu.conf creation, TPM directory fixes, VM Manager toggling) never held past the next reboot.
|
||||
- **Fix applied 2026-07-06:** rewrote `/boot/config/go` to remove the hijack block entirely, restoring the stock Unraid boot path (emhttpd handles the libvirt mount natively). Backup saved as `/boot/config/go.bak-2026-07-06`.
|
||||
- **Belt-and-suspenders:** also pinned `domain.cfg`'s `IMAGE_FILE` directly to `/mnt/cache/system/libvirt/libvirt.img` (bypassing the `/mnt/user` FUSE path) to remove any FUSE-timing edge case, even though the go-hack was confirmed as the primary cause. Backup saved as `/boot/config/domain.cfg.bak-2026-07-06`.
|
||||
- VM "Windows 11" (= LONESTAR-VM, `192.168.120.197`) was recovered and autostart re-enabled the same day. **A clean reboot was scheduled for the night of 2026-07-06 to validate the fix under a full stock boot** — confirm this held before considering the issue closed.
|
||||
- **Reusable lesson:** on any Unraid box with a history of hand-patched `/boot/config/go` libvirt fixes, check for leftover blocks that start libvirtd or touch `/etc/libvirt` before emhttpd runs — a "fix" that races the array mount will look exactly like a flaky storage/VM problem on every subsequent boot.
|
||||
- Related, now-explained history: the 2026-06-06 to 2026-06-10 VM outage and the 2026-06-10 libvirt/TPM repair session (missing `qemu.conf`, stale TPM directory, VM Manager toggle) were all downstream symptoms of the same underlying boot-order hijack later found and removed 2026-07-06.
|
||||
- **No active UPS** on this Unraid box compounds the risk of any future boot-order issue turning into an unclean shutdown; consider adding one.
|
||||
|
||||
- **Inherited Sophos with no Central access — kernel-driver tamper-protection removal (procedure proven and COMPLETE on LS-1 and LS-2, 2026-06-02).** LS-1 and LS-2 came from the previous MSP running Sophos Endpoint Protection managed via the previous MSP's Sophos Central account — ACG has **no Central access**, so no remote uninstall and no way to disable tamper protection from the management plane. The procedure is now proven end-to-end and reusable. Key findings from the full execution:
|
||||
|
||||
- **SophosZap's gate is a registry flag, not just the driver.** SophosZap checks `HKLM\SYSTEM\CurrentControlSet\services\Sophos Endpoint Defense\TamperProtection\Config\SEDEnabled` — if this is `1`, SophosZap exits with "does not run with tamper protection on" even when the kernel driver is renamed/disabled. The driver disable alone is not sufficient; `SEDEnabled=0` must be set.
|
||||
@@ -144,14 +170,20 @@ Electrical contractor in Tucson, AZ. ACG-managed client. Distinctive in the flee
|
||||
|
||||
## Active Work
|
||||
|
||||
No open Syncro tickets as of 2026-06-02.
|
||||
No open Syncro tickets as of 2026-07-06. **Prepaid hour block is depleted (0 hrs remaining)** — clarify with the client whether to replenish the block or move the account to formal break-fix/hourly billing before the next ticket.
|
||||
|
||||
- **Sophos removal on LS-1 / LS-2 — COMPLETE (2026-06-02).** Both machines are fully clean: no Sophos services, drivers, folders, or Add/Remove entries; Windows Defender real-time protection active on both. Billed and closed on Syncro #32347 (2.0h in-shop, prepaid). See Patterns for the full reusable procedure including the critical SophosEL ELAM boot-driver lesson.
|
||||
- **Unraid "Tower" recurring VM-disappearance issue — root cause found and fixed 2026-07-06.** Leftover `/boot/config/go` libvirt hijack (from a 2026-06-10 hand-patch) removed; `domain.cfg` `IMAGE_FILE` pinned to `/mnt/cache/...`; VM "Windows 11" (LONESTAR-VM) recovered with autostart. **Pending: the clean-reboot validation scheduled for the night of 2026-07-06** — confirm the box comes up clean (emhttpd mounts libvirt.img, VM autostarts) before closing this out. See Patterns for full detail.
|
||||
|
||||
- **Unraid server USB replacement — COMPLETE (2026-06-02).** New stick running Unraid 7.1.4, config/ preserved, license re-registered to the new GUID, **old stick retired**. Documented and billed on Syncro #32372 (1.5h in-shop, prepaid, Closed).
|
||||
- **Remaining Unraid items handed to Mike** (todo `de75eec6`): set + vault the root password, document hostname/IP/license type, verify array integrity, and investigate a LimeTech/Unraid API skill. **Deferred until Mike posts a note** on what he has done with the machine — do not chase these until then.
|
||||
- **`gururmm-agent` Docker container on Tower is crash-looping (as of 2026-07-06)**, leaving the Unraid host's own RMM agent offline. Being handled in a separate session — verify resolution and confirm the host agent is back online.
|
||||
|
||||
- **LS-1 / LS-2 logistics:** both desktops are at the **Computer Guru office** for repair as of 2026-06-02; to be returned **onsite to Norris and reconnected the week of 2026-06-08**.
|
||||
- **Unraid vault/documentation cleanup (carried forward, still open):** correct the stale `172.16.1.188` IP in `clients/lonestar-electrical/unraid-server.sops.yaml` to the live `192.168.120.177`; document license type and full array/disk layout; consider enabling Settings > Syslog Server > "Mirror syslog to flash" so future boot failures leave evidence (Unraid syslog is RAM-only and had already rotated out key evidence once).
|
||||
- Original todo `de75eec6` (root pw vault, server docs, array verify, LimeTech/Unraid API investigation) — the root password has since been vaulted; the remaining documentation items are still outstanding.
|
||||
|
||||
- **No UPS on the Unraid host** — flagged as a risk item; no remediation scheduled yet.
|
||||
|
||||
- **LS-1 / LS-2 logistics:** returned onsite to Norris the week of 2026-06-08 per the 2026-06-02 plan; current physical/network status not re-confirmed since — verify.
|
||||
|
||||
- **Apple MDM enrollment for iPhone/iPads** — plan compiled 2026-05-27 (`clients/lonestar-electrical/docs/apple-mdm-setup-reference.md`); execution status not confirmed in available sources — verify whether this proceeded.
|
||||
|
||||
---
|
||||
|
||||
@@ -165,7 +197,8 @@ No open Syncro tickets as of 2026-06-02.
|
||||
| 2026-03-23 | Lonestar MDM issue investigated — identified ManageEngine self-enrollment as the cause of joser's personal-phone prompt; fix initially blocked by a broken Zoho portal page |
|
||||
| 2026-03-24 | MDM issue RESOLVED — disabled ManageEngine self-enrollment AND removed ManageEngine as GWS third-party EMM. joser's phone stopped prompting immediately |
|
||||
| 2026-05-04 | Win11 upgrades on LS-1 and LS-2 (#32244) |
|
||||
| 2026-05-05 | iPhone field setup (#32251) |
|
||||
| 2026-05-05 | iPhone field setup (#32251) — first Apple device for this client, triggered the Apple MDM enrollment plan |
|
||||
| 2026-05-27 | Apple MDM enrollment plan compiled for ManageEngine (APNs cert + targeted invite enrollment), pending execution |
|
||||
| 2026-05-28/29 | Sophos removal on LS-1/LS-2 begun: enrolled in GuruRMM, removed Datto startup conflict (LS-2), registered Safe Mode agents, removed user-mode Sophos; blocked by `SophosED.sys` kernel driver — WinRE offline removal staged (Ventoy USB), completion pending |
|
||||
| 2026-06-01 | Recovered the (previously unlogged) Sophos removal context, reconstructed it into a session log, and handed the WinRE completion procedure to Howard via coordinator (msg `689cfb7c`) |
|
||||
| 2026-06-02 | Unraid server USB flash drive failed (recurring bzfirmware checksum error); migrated to new stick (Unraid 7.1.4 via USB Creator), copied old config/, re-registered license to new GUID |
|
||||
@@ -173,12 +206,18 @@ No open Syncro tickets as of 2026-06-02.
|
||||
| 2026-06-02 | Sophos removal COMPLETED on LS-1 and LS-2 — offline tamper-disable (SED Start=4 + SEDEnabled=0) + SophosZap two-pass via GuruRMM; LS-2 hit Automatic Repair after boot-critical SophosEL.sys was renamed (recovered by restoring the file from PE, then relying on already-correct offline edits + SophosZap to remove it safely); Windows Defender active on both |
|
||||
| 2026-06-02 | Syncro #32347 (Sophos removal, 2.0h in-shop) and #32372 (Unraid USB replacement, 1.5h in-shop) created, billed, and closed against prepaid block — 17.0 -> 13.5 hrs remaining |
|
||||
| 2026-06-02 | Old failed Unraid USB stick retired (new one registered/stable). Remaining Unraid items (root pw vault, server docs, array verify, LimeTech/Unraid API skill) handed to Mike (todo `de75eec6`), deferred until he posts a note. LS-1/LS-2 are at the ACG office for repair; returning onsite week of 2026-06-08 |
|
||||
| 2026-06-10 | LONESTAR-VM (Windows VM, Warren site, GuruRMM agent) RDP enabled in firewall via GuruRMM: Remote Desktop rules confirmed enabled and tightened from `Any` to Domain+Private (removed Public exposure); active net Private, RDP listening already on. VM documented for the first time — IP 192.168.120.197/24, Warren LAN 192.168.120.0/24. |
|
||||
| 2026-06-03/04 | GuruRMM agent Docker-deployed on the Unraid server (Tower) via the Gitea container registry (`gururmm-agent`, agent ID `e827f798-...`); Warren site (WARM-FALCON-5891) API key regenerated and vaulted; root credentials vaulted. LAN IP at the time was `172.16.1.188` (ACG office, temporary — box had not yet returned to the Warren site) |
|
||||
| 2026-06-06 | LONESTAR-VM went offline (later understood to be an early symptom of the same libvirt boot-order issue root-caused 2026-07-06) |
|
||||
| 2026-06-10 | LONESTAR-VM RDP enabled/tightened in the firewall via GuruRMM: Remote Desktop rules confirmed enabled and scoped from `Any` to Domain+Private (removed Public exposure); VM documented for the first time — IP 192.168.120.197/24, Warren LAN 192.168.120.0/24 |
|
||||
| 2026-06-10 | LONESTAR-VM restored after being offline since 2026-06-06: diagnosed missing `/etc/libvirt/qemu.conf` and a stale `/var/lib/libvirt/swtpm` permissions issue; a comprehensive `/boot/config/go` libvirt fix attempt broke Unraid boot entirely and was reverted to a minimal `go`; VM Manager toggle rediscovered and restored the VM. **A "LibVirt Fix (final 2026-06-10)" hand-hack was left in `/boot/config/go` during this session — this hack was later identified (2026-07-06) as the actual root cause of the box's recurring VM-disappearance problem.** |
|
||||
| 2026-07-06 | Root cause found and fixed for the recurring "Unraid `/VMs` page empty / VM disappears" problem: the 2026-06-10 `/boot/config/go` hand-hack was hijacking libvirt startup ahead of emhttpd's mount of `libvirt.img`. Removed the hijack (backup `go.bak-2026-07-06`), pinned `domain.cfg` `IMAGE_FILE` to `/mnt/cache/...` (backup `domain.cfg.bak-2026-07-06`), recovered VM "Windows 11" (LONESTAR-VM) with autostart. Clean-reboot validation scheduled that night. Confirmed the vaulted Unraid IP (`172.16.1.188`) is stale; live IP is `192.168.120.177`. Noted no active UPS on the box and a crash-looping `gururmm-agent` Docker container (host RMM agent offline, handled separately) |
|
||||
| 2026-07-06 | Syncro prepaid hour block confirmed depleted (0 hrs remaining) and no open tickets — contract is effectively break-fix pending a client conversation about replenishing the block |
|
||||
|
||||
---
|
||||
|
||||
## Compilation Notes
|
||||
|
||||
- **Full rebuild 2026-07-06 (GURU-5070/claude-main):** re-synthesized the entire article from the template. Corrected the Unraid server's live IP throughout (`192.168.120.177`, Warren-site LAN) and flagged the vaulted `172.16.1.188` as stale (only valid while the box was temporarily at the ACG office). Added the 2026-07-06 root-cause finding for the recurring "Unraid `/VMs` empty / VM disappears" issue as a new, prominent Pattern and multiple History rows: a leftover `/boot/config/go` hand-hack from a 2026-06-10 repair session was hijacking libvirt startup ahead of emhttpd's mount, which is why every prior fix (manual mounts, qemu.conf, TPM dir, VM Manager toggles) reverted on the next reboot. Folded in the previously-undocumented 2026-06-03/04 GuruRMM Docker deployment on Tower, the 2026-06-06 to 2026-06-10 VM outage/restore, and the 2026-06-10 RDP firewall tightening. Updated billing to Syncro live data: prepaid block now at 0 hrs (from 13.5 hrs recorded 2026-06-02) and no open tickets; added Amanda B as a newly-seen Syncro contact alongside the existing Robin Eneix. Added a "Servers & Services" table and a Network placeholder to match the current wiki client template. Noted the Unraid host has no active UPS and its sole Docker container (`gururmm-agent`) was crash-looping as of 2026-07-06. Preserved all pre-existing Patterns & Known Issues and History Highlights rows verbatim (Sophos removal procedure, dual-EMM MDM trap, bzfirmware USB failure pattern), adding to rather than replacing them, per rebuild instructions.
|
||||
- Surgical update 2026-06-02 ~18:10 PT (Howard-Home/claude-main): retired the old Unraid USB (new registered/stable); handed remaining Unraid items (root pw vault, server docs, array verify, LimeTech/Unraid API skill) to Mike via todo `de75eec6`, deferred until Mike posts a note; recorded that LS-1/LS-2 are at the ACG office for repair (explains the 172.16.0.0/22 ARP entries) and return onsite week of 2026-06-08.
|
||||
- Refreshed 2026-06-02 ~17:45 PT (recompile by Howard-Home/claude-main) to absorb the "17:39 PT — Sophos removal COMPLETE" update section of the 2026-06-02 session log: marked Sophos removal COMPLETE on both LS-1/LS-2 in Active Work and Infrastructure; updated hours remaining to 13.5 (Syncro #32347 2.0h + #32372 1.5h billed/closed); expanded Patterns with the proven full procedure including the critical two-driver distinction (SophosEL ELAM boot-critical — never rename/delete; SophosED tamper driver — disable via Start=4+SEDEnabled=0); added LS-1/LS-2 GuruRMM agent IDs; added two new History Highlights rows (PE+SophosZap completion, billing).
|
||||
- Refreshed 2026-06-02 22:10 PT (recompile by HOWARD-HOME/claude-main) to absorb the 22:10 PT update section of the 2026-06-02 session log: updated Active Work Sophos bullet to reflect execution-in-progress on LS-1 (BitLocker confirmed off, SophosZap staged, awaiting drive-letter check before PE delete); updated Patterns wording from "in progress 2026-05-28/29" to "execution started 2026-06-02"; added History Highlights row for the LS-1 PE execution start.
|
||||
@@ -187,7 +226,7 @@ No open Syncro tickets as of 2026-06-02.
|
||||
- Seeded 2026-05-26 from two March session logs + credentials.md + vault entry + temp provisioning scripts, enriched with live Syncro data (customer 33809612).
|
||||
- **Vault slug is `lonestar-electrical`** (matches `clients/lonestar-electrical/` in the vault), though session logs and temp scripts use the un-hyphenated `lonestar`.
|
||||
- Lonestar work now lives in both `clients/lonestar-electrical/` (docs + session-logs) and root session logs / `temp/` scripts.
|
||||
- Flagged `[verify]`: billing rate; exact roles/names for James, Kyla, Russ; full workstation inventory; Unraid server hostname/IP/license type/root credentials.
|
||||
- Flagged `[verify]`: billing rate; exact roles/names for James, Kyla, Russ, and Amanda B vs. Robin Eneix; full workstation inventory and current onsite status of LS-1/LS-2; Unraid license type and array/disk layout; whether the Apple MDM enrollment plan was executed; what consumed the prepaid block down to 0 hrs.
|
||||
|
||||
## Backlinks
|
||||
|
||||
|
||||
@@ -46,7 +46,7 @@ Run `/wiki-lint` to check for stale entries and broken backlinks.
|
||||
| [Western Tire](clients/western-tire.md) | Tire retail (jackfurriers.com brand); Mike Furrier owner (Syncro ID 391491); email migrated from websvr to IX 2026-04-22; 30 mailboxes; SSL cert expires 2026-05-30 | 2026-05-24 |
|
||||
| [Kittle Design & Construction LLC](clients/kittle.md) | **Canonical Kittle article.** GC Tucson AZ; Syncro 32460233; M365 kittlearizona.com (tenant 3d073ebe); **major June 2026 BEC/ACH-fraud incident** — Ken+marco+Accounting compromised, fraudulent bank-change to City of Tucson + Town of Marana ($130K+ exposure, PREVENTED, no loss), IC3 filed; root cause = April credential theft + incomplete remediation (password never reset → ~2mo persistence); CA hardened + Entra P2 added 6/9; HPE MicroServer WS2025 EVAL, no backups/firewall | 2026-06-09 |
|
||||
| [Khalsa (two-site)](clients/khalsa.md) | Khalsa Montessori School (Syncro 9456554); two-site (Camden + River); domain khalsa.local on 10.11.12.0/22. **Server inventory captured 2026-06-19** (Syncro+ScreenConnect): TROUT (Camden, 10.11.12.254) = sole live DC+DNS+file(K:)+QuickBooks on aging Dell T110 II, ~490GB used, C: only 10% free; SALMON (River, 10.11.14.76) = standalone file/QB server; KHALSADC2 = intended 2nd DC OFFLINE since 2025-08 (no AD redundancy). **TROUT can't be removed as-is** (Schoolmaster gone but it now runs DC/DNS/QB) — needs role migration first. No server backups captured; VPN to ACG broken (manage via ScreenConnect). M365/contract terms still pending | 2026-06-19 |
|
||||
| [Lone Star Electrical Systems](clients/lonestar-electrical.md) | Electrical contractor Tucson AZ; Syncro 33809612, prepaid block 13.5 hrs; Google Workspace (not M365); ManageEngine MDM (Zoho); Unraid server (7.1.4, USB migrated 2026-06-02); LS-1/LS-2 Sophos removal COMPLETE (2026-06-02); Defender active on both; field/mobile-first | 2026-06-02 |
|
||||
| [Lone Star Electrical Systems](clients/lonestar-electrical.md) | Electrical contractor Tucson AZ; Syncro 33809612, prepaid block 13.5 hrs; Google Workspace (not M365); ManageEngine MDM (Zoho); Unraid server (7.1.4, USB migrated 2026-06-02); LS-1/LS-2 Sophos removal COMPLETE (2026-06-02); Defender active on both; field/mobile-first | 2026-07-06 |
|
||||
| [Anaise](clients/anaise.md) | Single workstation client; contact David (anaisedavid.office@gmail.com); DESKTOP-O8GF4SD; creds in vault at clients/anaise/desktop-o8gf4sd.sops.yaml; onboarding incomplete; M365 enrollment unconfirmed | 2026-05-24 |
|
||||
| [ACG Website (azcomputerguru.com)](clients/azcomputerguru.com.md) | Public website redesign (Astro); score 33/40; placeholder testimonials + no-backend form are pre-launch blockers; OKLCH token design system; see internal-infrastructure.md for ACG servers | 2026-05-24 |
|
||||
| [Quantum WMS](clients/quantumwms.md) | WMS company; quantumwms.com tenant (ddf3d2c9); GoDaddy decoupling + M365 migration; 2x Business Premium + Exchange Online Plan 1; deadline 2026-06-03; Tenant Admin consented 2026-05-26 | 2026-05-26 |
|
||||
|
||||
Reference in New Issue
Block a user