# CyndyOffice Intermittent Freeze/Lockup Diagnosis — Syncro #32397

## User
- **User:** Howard Enos (howard)
- **Machine:** Howard-Home
- **Role:** tech

## Session Summary

Investigated the CyndyOffice workstation (Universal Minerals International Inc, Syncro
ticket #32397, "Onsite - Computer intermittently freezing and shutting down") via the
GuruRMM agent fleet. The presenting symptom was the machine locking up hard and requiring
a forced power-off to recover. The first finding corrected a premise: despite the RMM site
name "Howard-VM", CyndyOffice is a physical HP Pavilion Desktop TP01-2xxx (AMD, 16 logical
CPUs, single 16 GB Kingston DIMM, 1 TB WD SN530 NVMe, Win 11 Home build 26200), not a VM.

A multi-pass read-only diagnostic established the lockup signature: ~20 hard lockups over
six weeks, each a Kernel-Power 41 event with bugcheck code 0 and no crash dump, paired with
a 6008 dirty-shutdown. Crash dumps were confirmed enabled (CrashDumpEnabled=3, AutoReboot=1),
so the absence of any minidump/MEMORY.DMP and any WHEA-Logger hardware error is meaningful —
these are true hardware/firmware freezes, not Windows BSODs or software crashes. The event
log goes silent at each freeze and resumes only at the next boot. Storage was ruled out (SSD
healthy, 0% wear, 37C), as were sleep/wake (no auto-sleep on AC), AV conflict (only Defender
registered), and idle thermal (~30C).

Remediation applied: Howard had already updated the BIOS via HP Support Assistant, confirmed
in the diagnostic as F.36 (Oct 2024) -> F.38 (Jul 2025). Fast Startup was disabled
(HiberbootEnabled 1->0). A Windows Memory Diagnostic was armed via bcdedit /bootsequence
{memdiag} and run on reboot — result: PASSED, no errors (Standard 2-pass; an extended
MemTest86 USB run remains available if exhaustive certainty is wanted). An orphaned
mbamchameleon (Malwarebytes leftover) driver service was deleted via sc.exe, clearing boot
Event 7000. The remaining boot error (TPM-WMI 1796, Secure Boot SBAT update 0x800700c1) was
identified as benign Microsoft noise and left alone.

Separately, QuickBooksMessaging.exe was found crash-looping (~15/min; .NET
System.ObjectDisposedException on NotifyIcon system-tray update), unrelated to the freezes.
QuickBooks is Enterprise 22.0 (2022 edition, past Intuit support). Howard ran the QuickBooks
Tool Hub repair; after repair + reboot no new messaging crashes were logged (to be fully
confirmed once the company file is in active use). The Syncro ticket was updated with three
public comments (findings, memtest passed, QB repair + contingency plan). A contingency was
documented: if freezing recurs after the BIOS/Fast Startup fixes, the next step is a full
hardware diagnostic (extended memory + drive/PSU testing) plus a backup and clean Windows
reinstall, requiring ~1-2 days of machine downtime. PSU is the prime remaining hardware
suspect.

## Key Decisions

- Treated the absence of crash dumps + WHEA errors (with dumps confirmed enabled) as positive
  evidence of a hardware/firmware hang rather than a software/driver BSOD — drove the
  hardware-first diagnostic path.
- Re-resolved the agent UUID live on every dispatch: this agent re-enrolls with a new UUID on
  reboot, and an early queued command was dropped ("Command not found") when the UUID changed.
- Chose Windows Memory Diagnostic (remote, bcdedit-armed) over MemTest86 for the first RAM
  pass since MemTest86 requires physical USB boot media; flagged MemTest86 as the exhaustive
  follow-up if Standard passed but freezes continued.
- Recommended a monitoring window as the primary path after BIOS + Fast Startup fixes (freezes
  were every 1-3 days), rather than immediately swapping hardware.
- Posted ticket notes public (per Howard) with do_not_email=true — visible in the customer
  portal without emailing dense technical detail.
- Deleted only the confirmed-orphaned mbamchameleon service (missing .sys file) after verifying
  via registry + sc.exe query; left the benign SBAT/Secure Boot error untouched.

## Problems Encountered

- First queued diagnostic command vanished ("Command not found") because the agent re-enrolled
  with a new UUID (1ae77bda... -> 28708e66...) on reconnect. Resolved by re-resolving the UUID
  live and re-dispatching.
- mbamchameleon service was not enumerated by Win32_Service/Win32_SystemDriver (CIM skips
  drivers whose file is missing), so the first delete attempt reported "not present." Resolved
  by reading the SCM registry key directly and deleting via sc.exe (which operates on the
  registry regardless of file presence) — DeleteService SUCCESS, key confirmed gone.
- Initial memtest arming went pending (agent offline mid-reboot); confirmed on next reconnect
  that the bootsequence flag was consumed and results logged, verifying the test actually ran.

## Configuration Changes

Remote changes on CyndyOffice (via GuruRMM, no local repo files changed except memory/log):
- BIOS F.36 -> F.38 (applied by Howard via HP Support Assistant; confirmed in diagnostic).
- Fast Startup disabled: `HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager\Power`
  HiberbootEnabled 1 -> 0.
- Windows Memory Diagnostic armed (`bcdedit /bootsequence {memdiag}`) and run — passed.
- Orphaned service removed: `sc.exe delete mbamchameleon` (registry key
  `HKLM\SYSTEM\CurrentControlSet\Services\mbamchameleon` deleted).
- QuickBooks Tool Hub repair run by Howard.

Repo files:
- Updated `.claude/memory/cyndyoffice-physical-hp-lockups.md` (full diagnosis + outcome).
- Added index line in `.claude/memory/MEMORY.md` (Project section).
- This session log.

## Credentials & Secrets

None discovered or created this session. RMM and Syncro auth via existing vault paths
(`infrastructure/gururmm-server.sops.yaml`, Syncro per-user key for howard).

## Infrastructure & Servers

- GuruRMM API: http://172.16.3.30:3001
- CyndyOffice: HP Pavilion Desktop TP01-2xxx, Win 11 Home build 26200, agent v0.6.57.
  - Product # (SKU): 318G6AA#ABA
  - Serial / Service Tag: 2MO21549RB
  - Motherboard: HP 8906; BIOS: F.38 (07/30/2025)
  - RMM client "AZ Computer Guru", site "Howard-VM" (misleading — physical machine).
  - Agent UUID at session end: 28708e66-342f-4130-b192-e308b582f00b (re-enrolls/changes on reboot).
- Syncro customer: Universal Minerals International Inc (customer_id 34844920).

## Commands & Outputs

- Memtest result (System log, MemoryDiagnostics-Results): Event 1201/1101 "tested the
  computer's memory and detected no errors."
- Lockup signature: Kernel-Power 41, BugcheckCode=0, no minidump, no WHEA, paired 6008 dirty
  shutdown; ~20 occurrences Apr 26 - Jun 9 2026.
- QuickBooks crash: `Faulting application QuickBooksMessaging.exe v1.0.9186.39137`,
  faulting_module KERNELBASE.dll, exc 0xE0434352 (.NET); inner
  System.ObjectDisposedException at System.Windows.Forms.NotifyIcon.UpdateIcon.
- Boot errors at last boot: SCM 7000 (mbamchameleon, file not found — now deleted); TPM-WMI
  1796 (Secure Boot SBAT 0x800700c1 — benign).
- `sc.exe delete mbamchameleon` -> `[SC] DeleteService SUCCESS`.

## Pending / Incomplete Tasks

- Monitor for any new Kernel-Power 41 events over ~1 week to confirm BIOS + Fast Startup fixed
  the freezing. (Howard declined an automated daily RMM watch — manual check on request.)
- Confirm QuickBooks messaging crash is gone once the company file is opened and in active use.
- If freezing recurs: PSU is prime hardware suspect; full hardware diagnostic (extended
  MemTest86 + drive/PSU) + backup and clean Windows reinstall (~1-2 days downtime) — documented
  publicly on the ticket.
- Optional cleanups not done: pagefile is a fixed 1 GB on C: (recommend system-managed);
  consider QuickBooks upgrade to a supported year (22.0 past Intuit support).

## Reference Information

- Syncro ticket: #32397 (id 112445840) — https://computerguru.syncromsp.com/tickets/112445840
- Public comments posted: 418383840 (findings), 418387764 (memtest passed),
  418395504 (QB repair + contingency).
- Memory file: .claude/memory/cyndyoffice-physical-hp-lockups.md
- RMM commands this session: diagnostics + remediation dispatched to agent
  28708e66-342f-4130-b192-e308b582f00b.

## Update: 13:15 PT — Billing, RMM removal, client wiki

Billed and closed out the diagnostic engagement, removed the temporary RMM agent, and
seeded the client wiki article.

**Billing (Syncro #32397):** 1.0h onsite (product 26118, $175.00). Posted a public
resolution comment WITH `do_not_email: false` so the client was emailed the work summary
plus the verbatim contingency note (if freezing recurs -> full hardware diagnostic +
backup/clean Windows reinstall, ~1-2 days downtime). Line item 42807858; invoice **#67810**
(id 1650637398), total $175.00. Ticket left **In Progress** (monitoring), not marked
Invoiced. Invoice PDF email is a Syncro GUI step — no verified API endpoint, did not probe.

**RMM removal (client is break-fix / no-RMM):** Confirmed CyndyOffice belongs to Universal
Minerals (customer 34844920, prepay 0.0) — the agent was a temporary diagnostic tool. The
Windows agent exposes built-in verbs (`install/uninstall/start/stop/status/watchdog`).
Because running `uninstall` through the agent would kill the command mid-execution, wrote a
removal script to `C:\Windows\Temp\rmm-remove.ps1` and ran it via a **detached one-time
scheduled task** (`GuruRMM-Removal`, SYSTEM) that fires ~8s after the agent acks: runs
`gururmm-agent.exe uninstall`, `taskkill /F`, `sc delete GuruRMMWatchdog/GuruRMMAgent`,
removes `C:\Program Files\GuruRMM` + `C:\ProgramData\GuruRMM`, self-deletes the task. Agent
went offline immediately (service gone). Server-side record cleaned up:
`DELETE /api/agents/28708e66-...` -> HTTP 204, confirmed removed from the dashboard.
Consequence: the freeze monitoring window is now manual / customer-reported, not via RMM.

**Client wiki seeded:** Created `wiki/clients/universal-minerals.md` (break-fix profile,
CyndyOffice hardware/specs, freeze + QuickBooks patterns, #32397 active work, history) and
added the client to `wiki/index.md` (Clients table + Cross-Reference). To be expanded as
more of the environment is learned, per the standard client-wiki pattern.

**Config changes (this update):** updated `.claude/memory/cyndyoffice-physical-hp-lockups.md`
(billing + RMM-removal + general Windows-agent removal method); created
`wiki/clients/universal-minerals.md`; edited `wiki/index.md`.

**Reference:** invoice #67810 (id 1650637398); Syncro comments 418397979 (billing, emailed);
removal command `a25eaab4-...`; removed agent UUID `28708e66-342f-4130-b192-e308b582f00b`.

## Update: 13:25 PT — Full wiki recompile (/wiki-compile client:universal-minerals --full)

Ran the formal full recompile through the staged pipeline to put the client article on the
same footing as the rest of the wiki.

**Pipeline:** Claimed per-article coord lock (`wiki/clients/universal-minerals`, id
`584904d4-...`, TTL 1h). Pulled live Syncro enrichment (customer 34844920: prepay 0.0 =
break-fix; asset count 0; invoice history #100079 $75 2025-12-17, #67060 $75 2026-02-25,
#67810 $175 2026-06-10). Delegated synthesis to a **Sonnet subagent** (general-purpose,
model sonnet) which read the session log + memory + existing article and returned the
recompiled markdown. Wrote it to `.claude/wiki_staging/client-universal-minerals.md`,
reviewed the `diff -u` against the live article (clean enrichment, Patterns/History
preserved, billing Syncro-authoritative, backlink intact, 8 H2 sections, no corruption),
then applied to the live tree. Committed `5eee825` "wiki: compile universal-minerals
(full)", rebased on origin/main, pushed. Released the lock and cleared staging.

**Enrichment over the seed:** added Syncro billing history (establishes a long-standing
break-fix relationship with small prior jobs); **corrected ticket #32397 status to
Invoiced** — Syncro auto-moved it from In Progress when the invoice was created (the earlier
"leaving it In Progress" was silently overridden by Syncro's billing flow); added deferred
items (1 GB fixed pagefile, QB 22.0 EOL); refined fault-code detail in Patterns.

**Lock-release gotcha:** `coord.py lock release` takes the lock **id** as a positional arg
(`lock release <id>`), NOT `release <project> <resource>` like the claim. The Phase 6 example
in the wiki-compile skill (`lock release claudetools "wiki/<type>/<slug>"`) is wrong — it
errors `unrecognized arguments`. Released correctly with the UUID from the claim. Also note
`lock list` does not accept a project positional. [verify / fix the skill's Phase 6 example]

**Config changes (this update):** `wiki/clients/universal-minerals.md` (full recompile,
committed 5eee825); `wiki/index.md` (already carried the client row + Cross-Reference from
the 13:15 update). No memory change this step.

**Reference:** wiki commit `5eee825`; coord lock id `584904d4-d590-47b5-a6f7-4c5a255e8668`
(claimed + released); ticket #32397 now status Invoiced.
