sync: auto-sync from HOWARD-HOME at 2026-06-02 17:51:53

Author: Howard Enos
Machine: HOWARD-HOME
Timestamp: 2026-06-02 17:51:53

clients/lonestar-electrical/session-logs/2026-06-02-session.md

@@ -115,3 +115,76 @@ Separately, a `/sync` exposed a fleet repo-coordination problem: the `.claude/sk
 - Mike's deletion commit: `c759f04` "chore(memory): re-apply consolidation deletions + lift additive-only constraint".
 - HEAD after sync: `dd414c4`.
 - Full LS-1/LS-2 offline procedure: `clients/lonestar-electrical/session-logs/2026-05-29-sophos-removal.md`.
+
+---
+
+## Update: 17:39 PT — Sophos removal COMPLETE (LS-1 + LS-2) + Unraid ticket
+
+## User
+- **User:** Howard Enos (howard)
+- **Machine:** Howard-Home
+- **Role:** tech
+
+### Session Summary
+
+Completed the long-pending Sophos Endpoint removal on both Lone Star Norris workstations (LS-1 and LS-2), then created/closed the Syncro ticket for the earlier Unraid boot-USB replacement. Both removal jobs were driven remotely through GuruRMM once each machine was back in Windows.
+
+LS-1 was resumed from the offline-PE prep done earlier (driver renamed, BitLocker confirmed off). The blocker turned out to be more than the kernel driver: SophosZap refuses to run while the registry flag `HKLM\SYSTEM\CurrentControlSet\services\Sophos Endpoint Defense\TamperProtection\Config\SEDEnabled = 1`. Because the SophosED tamper driver was not loaded this boot (renamed offline), the flag could be cleared live as SYSTEM (`SEDEnabled=0`). SophosZap v1.9.158.0 then ran two passes (with a reboot between) via RMM and reported clean — no Sophos services, drivers, folders, or Add/Remove entries; Windows Defender real-time protection active.
+
+LS-2 was done via the manual offline (WinRE) procedure since it was offline in RMM at the start. Howard loaded the offline SYSTEM hive and set the SED service `Start=4` + `SEDEnabled=0`, then renamed the Sophos driver files. On reboot the machine dropped into Automatic Repair. The `SrtTrail.txt` root cause was explicit: "Boot critical file C:\WINDOWS\system32\DRIVERS\SophosEL.sys is corrupt" — i.e. missing because it was renamed. `SophosEL.sys` is the **Sophos ELAM** (Early Launch Anti-Malware) driver: `Start=0` (Boot), `ErrorControl=3` (Critical), so its absence aborts boot. Recovery: booted back to PE and renamed `SophosEL.sys.old` back to `SophosEL.sys`; the machine then booted.
+
+Once LS-2 was back in Windows, an RMM read of the service config showed the earlier offline edits had actually landed correctly: `Select\Current = 0x1` (ControlSet001 IS active), the SED tamper driver service (`Sophos Endpoint Defense`, SophosED.sys) was already `Start=4`, and `SEDEnabled` was 0 — so tamper protection was already neutralized. SophosZap then ran two passes via RMM (with a verified-safe reboot between — `SophosEL.sys` confirmed present on disk and no pending Sophos file-renames before rebooting) and reported clean. Defender active on LS-2.
+
+Billing: created/closed two Syncro tickets against the Lone Star prepaid block (customer 33809612). #32347 (Sophos removal LS-1+LS-2): 2.0h in-shop, invoiced $0.00, block 17.0 -> 15.0, Closed. #32372 (Unraid boot-USB replacement, documenting the earlier 2026-06-02 server fix): 1.5h in-shop, invoiced $0.00, block 15.0 -> 13.5, Closed.
+
+### Key Decisions
+
+- Cleared `SEDEnabled=0` (the SophosZap tamper gate) rather than only relying on the driver rename — the registry flag, not the driver presence, is what SophosZap checks.
+- LS-2: after the boot failure, did NOT re-rename the boot-critical `SophosEL.sys`. Restored it and relied on the (already-correct) SED service `Start=4` + `SEDEnabled=0` to neutralize tamper, letting SophosZap remove the ELAM driver itself the boot-safe way.
+- Verified `SophosEL.sys` present + no pending Sophos file-renames BEFORE the pass-2 reboot on LS-2, to avoid repeating the boot failure.
+- Drove both machines via GuruRMM (read service config, set registry, run SophosZap, reboot) rather than hands-on once each was in Windows.
+
+### Problems Encountered
+
+- **LS-2 boot failure (Automatic Repair).** Root cause (SrtTrail.txt): boot-critical `SophosEL.sys` (Sophos ELAM, Start=0/ErrorControl=3) was renamed and thus "corrupt"/missing. Resolved by booting to PE and renaming `SophosEL.sys.old` back to `SophosEL.sys`.
+- **SophosZap blocked by tamper flag, not driver.** First LS-1 run errored "SophosZap does not run with tamper protection on" with the driver already renamed — the `SEDEnabled=1` registry flag was the gate. Resolved by setting `SEDEnabled=0`.
+- **Offline ControlSet correctness.** The offline edit used `ControlSet001`; this only worked because `Select\Current=0x1`. Documented that the active control set must be read from `HKLM\OFFSYS\Select\Current` before editing; `CurrentControlSet` does not exist in an offline hive.
+- **PE PowerShell script closed on error.** The first-draft `Remove-Sophos-Offline-PE.ps1` exited (window closed) on an unhandled error. Hardened with a top-level try/catch + guaranteed `Read-Host` pause; abandoned in favor of the manual walkthrough for this job.
+
+### Configuration Changes
+
+- LS-1, LS-2: Sophos Endpoint Protection fully removed (services, drivers, `C:\Program Files\Sophos`, `C:\Program Files (x86)\Sophos`, `C:\ProgramData\Sophos`, Add/Remove entries, catalogs, certs). Windows Defender now the active AV on both.
+- LS-2 registry (offline, ControlSet001): `Sophos Endpoint Defense` service `Start=4`; `...\TamperProtection\Config\SEDEnabled=0`, `IgnoreSAV=0`.
+- Created `clients/lonestar-electrical/scripts/Remove-Sophos-Offline-PE.ps1` (offline PE removal helper; hardened error handling).
+
+### Credentials & Secrets
+
+- None created or changed. (Lone Star Unraid root password still not vaulted — pre-existing TODO.)
+
+### Infrastructure & Servers
+
+- **LS-1** GuruRMM agent id `6b9617fa-5c77-40e1-8b64-a1545e730895` (windows).
+- **LS-2** GuruRMM agent id `97fe5582-aa3d-4132-94a6-f4c8582bca31` (windows).
+- Sophos drivers (LS-2): `SophosED.sys` (2,561,552 B) = "Sophos Endpoint Defense" tamper driver, Type 2, ended at Start=4; `SophosEL.sys` (28,616 B) = "Sophos ELAM", Type 1, Start=0/ErrorControl=3 (BOOT-CRITICAL).
+- SophosZap: v1.9.158.0; log at `C:\WINDOWS\SystemTemp\SophosZap log.txt`; staged to `C:\Windows\Temp\SophosZap.exe` for pass 2.
+
+### Commands & Outputs
+
+- Tamper gate (per SophosZap log): `Value 'SEDEnabled' ... is set to 1. Tamper-protected by SED. ERROR: SophosZap does not run with tamper protection on`.
+- Clear it (live, SYSTEM): `reg add "HKLM\SYSTEM\CurrentControlSet\services\Sophos Endpoint Defense\TamperProtection\Config" /v SEDEnabled /t REG_DWORD /d 0 /f`.
+- Offline (PE): `reg load HKLM\OFFSYS X:\Windows\System32\config\SYSTEM` -> edit under `HKLM\OFFSYS\ControlSet001\Services\...` -> `reg unload HKLM\OFFSYS`. Active set from `reg query HKLM\OFFSYS\Select /v Current`.
+- LS-2 boot root cause: `Boot critical file C:\WINDOWS\system32\DRIVERS\SophosEL.sys is corrupt` (SrtTrail.txt). Fix: `ren X:\Windows\System32\drivers\SophosEL.sys.old SophosEL.sys`.
+- Removal run: `SophosZap.exe --confirm` x2 (reboot between); final outcome `error flag: 0`, services/drivers/folders NONE, Defender RTP True.
+
+### Pending / Incomplete Tasks
+
+- Vault the Lone Star Unraid root password + document the server (hostname, IP, Unraid 7.1.4, license type) in the wiki — still open.
+- Keep the old failing Unraid USB stick as backup until the new stick is confirmed stable, then retire.
+- Optional: delete leftover `SophosEL.sys.old` on LS-2 if any remained (cleanup attempted in pass 2).
+
+### Reference Information
+
+- Syncro: #32347 (Sophos removal, id 111423954, invoice 1650552617) and #32372 (Unraid USB, id 112022651, invoice 1650552739) — both Closed, prepaid, customer 33809612. Block now 13.5 hrs.
+- RMM API base `http://172.16.3.30:3001`.
+- PE removal script: `clients/lonestar-electrical/scripts/Remove-Sophos-Offline-PE.ps1`.
+- Offline procedure reference: `clients/lonestar-electrical/session-logs/2026-05-29-sophos-removal.md`.