azcomputerguru/claudetools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

sync: auto-sync from DESKTOP-0O8A1RL at 2026-05-17 22:07:52

Browse Source 
Author: Mike Swanson
Machine: DESKTOP-0O8A1RL
Timestamp: 2026-05-17 22:07:52
This commit is contained in:
Mike Swanson
2026-05-17 22:07:59 -07:00
parent acb0af9d3a
commit 3baaf91183
4 changed files with 438 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

257
.claude/POWER_FAILURE_RUNBOOK.md Normal file
View File

@@ -0,0 +1,257 @@
# Power Failure Recovery Runbook — ACG Office
Run through these checks IN ORDER after any unplanned power event.
All SSH uses `C:\Windows\System32\OpenSSH\ssh.exe` (never Git SSH).
---
## 0. Confirm you have LAN access
If working remotely, Tailscale must be fixed before anything else can be reached.
If on-site LAN, skip to Step 1.
---
## 1. pfSense — Tailscale subnet routes
**What breaks:** After reboot, pfSense loses its advertised Tailscale routes (`AdvertiseRoutes: null`).
Remote machines can no longer reach 172.16.x.x.
**Check:**
```powershell
& "C:\Windows\System32\OpenSSH\ssh.exe" -p 2248 admin@172.16.0.1 "tailscale debug prefs" | Select-String "AdvertiseRoutes|RouteAll"
```
Healthy output: `"AdvertiseRoutes": ["172.16.0.0/22"]` and `"RouteAll": true`
**Fix:**
```powershell
& "C:\Windows\System32\OpenSSH\ssh.exe" -p 2248 admin@172.16.0.1 "tailscale up --advertise-routes=172.16.0.0/22 --accept-routes"
```
**Verify:**
```powershell
& "C:\Windows\System32\OpenSSH\ssh.exe" -p 2248 admin@172.16.0.1 "tailscale status"
# pfsense-2 should NOT show "rx 0" after a few seconds
Test-NetConnection -ComputerName 172.16.3.20 -Port 22
```
---
## 2. Jupiter (Unraid) — libvirt / VMs
**What breaks:** libvirt.img (contains /etc/libvirt/ configs) is not loop-mounted on boot.
libvirtd fails with "socket already in use" or "snapshot dir not a directory". All VMs are down.
**Host:** 172.16.3.20 (SSH as root, no password — key auth)
### 2a. Check if libvirt.img is mounted
```powershell
& "C:\Windows\System32\OpenSSH\ssh.exe" root@172.16.3.20 "mount | grep libvirt"
```
Healthy: shows `/dev/loopN on /etc/libvirt`
Broken: no output
### 2b. Check libvirtd process
```powershell
& "C:\Windows\System32\OpenSSH\ssh.exe" root@172.16.3.20 "ps aux | grep libvirtd | grep -v grep"
```
### 2c. Fix — mount image and start libvirtd
```powershell
# Mount libvirt config image
& "C:\Windows\System32\OpenSSH\ssh.exe" root@172.16.3.20 "losetup -f --show /mnt/user/system/libvirt/libvirt.img"
# Note the loop device returned (e.g. /dev/loop4)
& "C:\Windows\System32\OpenSSH\ssh.exe" root@172.16.3.20 "mount /dev/loop4 /etc/libvirt && ls /etc/libvirt/qemu"
# Start libvirtd
& "C:\Windows\System32\OpenSSH\ssh.exe" root@172.16.3.20 "libvirtd -d"
# Verify VMs came up
& "C:\Windows\System32\OpenSSH\ssh.exe" root@172.16.3.20 "virsh -c qemu:///system list --all"
```
**Expected VM list:**
| Name | Expected State |
|------|---------------|
| GuruRMM | running |
| Unifi | running |
| OwnCloud | running |
| Claude-Builder | running |
| Windows 7 | shut off |
| Windows Server 2016 | shut off |
| Windows Server 2016_Template | shut off |
### 2d. Stale socket cleanup (if libvirtd still fails)
```powershell
& "C:\Windows\System32\OpenSSH\ssh.exe" root@172.16.3.20 "ls -la /run/libvirt/libvirt-sock"
# If it shows as a directory (not a socket), remove it:
& "C:\Windows\System32\OpenSSH\ssh.exe" root@172.16.3.20 "rm -rf /run/libvirt/libvirt-sock"
# Then retry libvirtd -d
```
---
## 3. Seafile — seahub process
**What breaks:** Seahub (Django/gunicorn) does not survive container restart cleanly.
Containers show "Up" but sync.azcomputerguru.com returns 5xx.
**Check:**
```powershell
& "C:\Windows\System32\OpenSSH\ssh.exe" root@172.16.3.20 "docker exec seafile ps aux 2>&1 | grep gunicorn | grep -v grep"
```
Healthy: 3+ gunicorn worker processes visible
Broken: no gunicorn output
**Fix:**
```powershell
& "C:\Windows\System32\OpenSSH\ssh.exe" root@172.16.3.20 "docker exec seafile bash -c 'cd /opt/seafile/seafile-pro-server-12.0.19 && ./seahub.sh start 2>&1'"
```
**Verify:**
```powershell
& "C:\Windows\System32\OpenSSH\ssh.exe" root@172.16.3.20 "docker exec seafile curl -s -o /dev/null -w '%{http_code}' http://localhost:8000/"
# Should return 302
```
---
## 4. NPM — iptables port 443 rule
**What breaks:** The iptables PREROUTING rule that routes :443 → NPM container is added at boot
via `/boot/config/go` on Jupiter. If that rule is missing (e.g. first boot after it was added),
sync.azcomputerguru.com HTTPS will fail even though NPM is running.
**Check:**
```powershell
& "C:\Windows\System32\OpenSSH\ssh.exe" root@172.16.3.20 "iptables -t nat -L PREROUTING -n | grep 'dpt:443'"
```
Healthy: `DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:443 to:172.17.0.2:443`
**Fix (if missing):**
```powershell
& "C:\Windows\System32\OpenSSH\ssh.exe" root@172.16.3.20 "iptables -t nat -I PREROUTING -p tcp --dport 443 -j DNAT --to-destination 172.17.0.2:443"
```
---
## 5. NPM — nginx health
**What breaks:** NPM's nginx may not be serving after a container restart.
**Check:**
```powershell
& "C:\Windows\System32\OpenSSH\ssh.exe" root@172.16.3.20 "docker exec npm nginx -t 2>&1"
```
**Fix (reload nginx config):**
```powershell
& "C:\Windows\System32\OpenSSH\ssh.exe" root@172.16.3.20 "docker exec npm nginx -s reload"
```
---
## 6. End-to-End Verification
Run all of these. Any False or non-2xx is a problem.
```powershell
# Core network
$checks = @(
@{host="172.16.3.20"; port=22; label="Jupiter SSH"},
@{host="172.16.3.20"; port=3000; label="Gitea"},
@{host="172.16.3.30"; port=22; label="GuruRMM VM SSH"},
@{host="172.16.3.30"; port=3001; label="GuruRMM server"},
@{host="172.16.3.30"; port=8001; label="Coord API"},
@{host="172.16.3.20"; port=443; label="NPM HTTPS (via iptables)"},
@{host="172.16.3.20"; port=8082; label="Seafile direct"}
)
foreach ($c in $checks) {
$r = Test-NetConnection -ComputerName $c.host -Port $c.port -WarningAction SilentlyContinue
$status = if ($r.TcpTestSucceeded) { "[OK]" } else { "[FAIL]" }
Write-Host "$status $($c.label) ($($c.host):$($c.port))"
}
# DNS
Clear-DnsClientCache
$dns = Resolve-DnsName sync.azcomputerguru.com -ErrorAction SilentlyContinue
$dnsOk = $dns.IPAddress -eq "172.16.3.20"
Write-Host "$(if ($dnsOk) {'[OK]'} else {'[FAIL]'}) DNS sync.azcomputerguru.com -> $($dns.IPAddress) (want 172.16.3.20)"
# HTTPS end-to-end
$resp = Invoke-WebRequest -Uri "https://sync.azcomputerguru.com/" -UseBasicParsing -ErrorAction SilentlyContinue
Write-Host "$(if ($resp.StatusCode -eq 200) {'[OK]'} else {'[FAIL]'}) sync.azcomputerguru.com HTTPS -> $($resp.StatusCode)"
```
---
## Infrastructure Reference
| Host | IP | Role |
|------|----|------|
| pfSense | 172.16.0.1 (SSH port 2248) | Router, DNS, Tailscale subnet router |
| Jupiter | 172.16.3.20 | Unraid NAS — hosts all VMs + Docker |
| Uranus | 172.16.3.21 | OwnCloud additional storage (not a proxy) |
| GuruRMM VM | 172.16.3.30 | Linux VM on Jupiter — GuruRMM server, Coord API, MariaDB |
| Pluto | 172.16.3.36 | Windows Server 2019 VM on Jupiter — build server |
| Tailscale range | 172.16.0.0/22 | Advertised via pfSense pfsense-2 node |
**Docker containers on Jupiter (172.16.3.20):**
| Container | Purpose | Key ports |
|-----------|---------|-----------|
| npm | Nginx Proxy Manager | 1880 (HTTP), 7818 (admin), 18443 (HTTPS) |
| seafile | Seafile web/app | 8082 (HTTP) |
| seafile-mysql | Seafile DB | internal |
| seafile-elasticsearch | Seafile search | internal |
| seafile-memcached | Seafile cache | internal |
**NPM proxy hosts:**
| Domain | Backend |
|--------|---------|
| sync.azcomputerguru.com | 172.16.3.20:8082 (Seafile) |
| rmm.azcomputerguru.com | 172.16.3.30:3001 (GuruRMM) |
| rmm-api.azcomputerguru.com | 172.16.3.30:3001 |
| git.azcomputerguru.com | 172.16.3.20:3000 (Gitea) |
| unifi.azcomputerguru.com | (Unifi VM) |
| emby.azcomputerguru.com | (Emby) |
---
## Known Post-Power-Failure Issue Pattern
Unraid's VM plugin (`dynamix.vm.manager`) should auto-mount `libvirt.img` at boot.
When it doesn't, the root cause is usually that the Unraid array came up before emhttp
finished initializing, or the go script ran before the array was fully mounted.
**Permanent fix (TODO):** Add a user script via Unraid's User Scripts plugin that runs at
array start and checks/mounts libvirt.img if not already mounted. This would eliminate
the manual step 2c above.
---
---
## 2026-05-17 Post-Mortem
**Root cause:** Power flicker at the office. UPS batteries were disconnected during a rack
reorganization move, so units had no backup capacity and shut down on the flicker instead
of riding through it.
**Resolution:** Mike reconnected batteries and restarted UPS units.
**Auto-recovery:** Jupiter (172.16.3.20) and Uranus (172.16.3.21) started automatically.
**Manual intervention required:** IX server (neptune/exchange host) did NOT auto-restart —
required a physical button press at the rack. Note for future: verify whether this is always
the case or was a one-off (BIOS power-on-after-failure setting may need adjustment).
**Remote fixes applied:** All steps 1–5 above were needed. Total recovery time ~1 hour.
---
*Last updated: 2026-05-17 — documented after power failure recovery*
*Checked by: Mike Swanson*

2
.claude/memory/MEMORY.md
View File

@@ -1,6 +1,8 @@
# Memory Index
## Reference
- [ACG Office Network Infrastructure](infra_office_network.md) — IPs, hosts, roles for pfSense/Jupiter/VMs/Docker. Use before assuming what's where; .21 (Uranus) is storage, not a proxy.
- [Power Failure Runbook](../.claude/POWER_FAILURE_RUNBOOK.md) — Step-by-step recovery: Tailscale routes, libvirt/VMs, Seafile, NPM/DNS. Run in order after any power event.
- [Syncro API — Invoice Verification Pattern](syncro_invoice_verification_pattern.md) - **CRITICAL:** List endpoint (/invoices?customer_id=X) does NOT return ticket linkage. Must query individual invoices (/invoices/{number}) to get ticket_id field. Invoice numbers are strings. Use ticket ID (not number) for comparison. Real case: falsely reported 31 tickets had no invoices (actually 29 had invoices, 2 were Non-Billable).
- [Approval Workflow: Tools vs Projects](approval-workflow-tools-vs-projects.md) - General tools (remediation-tool, onboard scripts, MSP utilities): Howard can modify OR Claude can execute with Howard/Mike approval. Projects (GuruRMM, etc.): require Mike approval, features→roadmap, bugs→bug list.
- [Community Forum (Flarum)](reference_community_forum.md) - Flarum forum at community.azcomputerguru.com, API access, database, posting workflow

30
.claude/memory/infra_office_network.md Normal file
View File

@@ -0,0 +1,30 @@
---
name: infra-office-network
description: ACG office LAN infrastructure — IPs, hosts, roles, and post-power-failure recovery
metadata:
type: project
---
ACG office LAN is 172.16.0.0/22, routed via Tailscale through pfSense node `pfsense-2` (100.119.153.74).
**Key hosts:**
| Host | IP | SSH | Role |
|------|----|-----|------|
| pfSense | 172.16.0.1 | port 2248, user admin | Router, DNS (Unbound), Tailscale subnet router |
| Jupiter | 172.16.3.20 | port 22, user root | Unraid NAS — all VMs + Docker containers |
| Uranus | 172.16.3.21 | (no key) | OwnCloud additional storage only — NOT a proxy |
| GuruRMM VM | 172.16.3.30 | port 22, user guru | Linux VM on Jupiter — GuruRMM, Coord API, MariaDB, Gitea |
| Pluto | 172.16.3.36 | (Windows) | Windows Server 2019 VM on Jupiter — MSI build server |
**Why:** How to apply: check these IPs before assuming what's where. .21 is NOT the Seafile proxy — NPM on .20 is.
**Docker on Jupiter (.20):**
- `npm` — Nginx Proxy Manager (ports 1880/7818/18443)
- `seafile` + `seafile-mysql` + `seafile-elasticsearch` + `seafile-memcached` — Seafile stack (port 8082)
- `gitea` — port 3000 (also accessed as 172.16.3.20:3000 or via SSH port forward from GuruRMM VM at .30:3000)
**NPM → 443 routing:** iptables PREROUTING on Jupiter: `dpt:443 → 172.17.0.2:443` (NPM container). Persisted in `/boot/config/go`. DNS `sync.azcomputerguru.com` → 172.16.3.20.
**VMs on Jupiter (virsh):** GuruRMM, Unifi, OwnCloud, Claude-Builder (running); Windows 7, Windows Server 2016, Windows Server 2016_Template (shut off).
**Why:** How to apply: see [[power-failure-runbook]] for full post-outage recovery steps.

149
session-logs/2026-05-17-session.md Normal file
View File

@@ -0,0 +1,149 @@
# Session Log — 2026-05-17
## User
- **User:** Mike Swanson (mike)
- **Machine:** DESKTOP-0O8A1RL
- **Role:** admin
- **Session span:** ~19:00–21:30 PT
---
## Session Summary
The session continued from a compacted prior context. The first task was executing a pre-written Python patch script (`tmp_syncro_patch.py`) that had been prepared in the previous session to insert the Syncro PSA Webhook Integration section into the GuruRMM feature roadmap. The script was SCP'd to the server, executed successfully, and the change was committed directly to the `azcomputerguru/gururmm` repo as `cef20dd` with 54 lines inserted under a new `## Integrations` section. A coord message was sent to Howard confirming the feature request was accepted into the roadmap with P1/P2 phase labels. The temp script was cleaned up afterward.
A `/sync` attempt immediately following was blocked: the claudetools git remote URL still pointed to `git.azcomputerguru.com` (Cloudflare-fronted, not accessible from this machine). The remote was updated to `http://172.16.3.20:3000` (internal Gitea IP), but then all 172.16.3.x hosts became unreachable. Tailscale connectivity investigation revealed the pfSense subnet router node (`pfsense-2`) had `rx=0` and was not forwarding traffic — the office had experienced a power failure.
Mike connected to the office LAN in person and described the situation: batteries had been disconnected from UPS units during a rack move; a power flicker tripped the units with no backup, shutting down the office infrastructure. Mike reconnected the batteries and restarted the UPS units. Jupiter (Unraid, 172.16.3.20) and Uranus (172.16.3.21) auto-started; the IX server required a manual button press at the rack. The remainder of the recovery was performed remotely via SSH.
Recovery proceeded in five stages. First, pfSense Tailscale routes were restored: `AdvertiseRoutes` had reset to `null` after reboot; `tailscale up --advertise-routes=172.16.0.0/22 --accept-routes` re-established subnet routing. Second, Unraid's libvirtd was failing because `libvirt.img` was not loop-mounted at boot and a stale `/run/libvirt/libvirt-sock` directory (not a socket file) blocked the daemon. The directory was removed, the image was manually loop-mounted to `/etc/libvirt/`, and `libvirtd -d` was started. All four VMs (GuruRMM, Unifi, OwnCloud, Claude-Builder) came up automatically. Third, Seafile's seahub/gunicorn process was not running despite the containers showing "Up"; `seahub.sh start` inside the container restored service. Fourth, `sync.azcomputerguru.com` was still not reachable because the pfSense DNS override pointed to 172.16.3.21 (Uranus — OwnCloud storage, not a proxy) rather than 172.16.3.20 (Jupiter/NPM). NPM's HTTPS listener is on host port 18443, not 443, so an iptables PREROUTING rule was also required. Both were applied and persisted. Fifth, a full end-to-end verification confirmed all services healthy.
A power failure recovery runbook was written to `.claude/POWER_FAILURE_RUNBOOK.md` covering all five stages with exact commands and a single-block PowerShell verification script. Infrastructure facts were committed to memory. Howard was notified via coord with the full post-mortem.
---
## Key Decisions
- **Syncro roadmap patch executed directly on server:** The patch script had already been reviewed and approved (option 3 selected in prior session). No re-review needed; SCP + python3 was the fastest path to commit.
- **Tailscale fix applied before anything else:** Without pfSense advertising routes, no subsequent SSH commands to 172.16.3.x would work. Route restoration was the prerequisite for all other steps.
- **libvirt.img mounted manually rather than rebooting Unraid:** A reboot would have been slower and risked the same failure mode repeating. Direct loop-mount + daemon start was faster and diagnostic.
- **DNS fix targeted pfSense config.xml AND host_entries.conf separately:** Editing config.xml alone wasn't sufficient — pfSense regenerates host_entries.conf on Unbound restart. Needed to edit host_entries.conf directly and send SIGHUP (not restart) to avoid regen overwriting the change. config.xml was also fixed so future regenerations use the correct IP.
- **iptables rule added to /boot/config/go:** Unraid's go script is the correct persistence mechanism. Without this, the port 443 → NPM routing would be lost on next reboot.
- **IX server BIOS power-restore setting noted as TODO:** It did not auto-restart after power loss. "Power On After Power Failure" BIOS/IPMI setting should be reviewed to match Jupiter's behavior.
---
## Problems Encountered
- **git remote URL pointed to Cloudflare-fronted domain:** `git.azcomputerguru.com` blocks curl. Fixed by updating remote to `http://172.16.3.20:3000`. This should be permanent but the remote reverted at some point — likely from a prior `/sync` on a machine that still had the old URL.
- **Pre-bash-backslash hook blocked several curl one-liners:** Hook at `.claude/hooks/pre-bash-backslash.sh` blocks commands with backslash line continuations. Worked around by using PowerShell SSH calls and Python urllib one-liners throughout.
- **pfSense `sed -i` used BSD syntax (not GNU):** FreeBSD `sed` requires `''` after `-i`. Initial attempts failed with "unterminated substitute pattern." Fixed by adjusting quoting.
- **Unbound ignored config.xml edit on `svc restart`:** pfSense's `pfSsh.php playback svc restart unbound` regenerates host_entries.conf from config.xml at restart time. The first restart overwrote the direct file edit. Solution: edit host_entries.conf, then send SIGHUP to the running Unbound PID instead of doing a full restart.
- **libvirtd failed with "snapshot dir not a directory":** `/var/lib/libvirt/qemu/snapshot` was a symlink to `/etc/libvirt/qemu/snapshot`, which didn't exist because libvirt.img wasn't mounted. Mounting the image first resolved it.
- **Seahub reported "started" but gunicorn process was absent:** The container start script ran seahub.sh twice (two start sequences visible in `docker logs seafile`) and the second run encountered the "creating admin" error. Despite reporting success, gunicorn wasn't running. Manually running `seahub.sh start` from inside the container fixed it.
- **SSH to Uranus (172.16.3.21) rejected:** Key not authorized. Irrelevant once it was established .21 is storage-only and the DNS fix was the correct path.
- **PowerShell subshell expansion broke `kill -HUP $(cat /var/run/unbound.pid)`:** PowerShell expanded `$(cat ...)` locally before passing to SSH. Worked around by querying the PID in a separate SSH call, then passing the integer directly.
---
## Configuration Changes
- **`/home/guru/gururmm/docs/FEATURE_ROADMAP.md`** — 54 lines inserted: new `## Integrations > ### Syncro PSA` section with P1/P2 phase breakdown. Committed as `cef20dd` on `azcomputerguru/gururmm` main.
- **`D:\claudetools\.claude\POWER_FAILURE_RUNBOOK.md`** — Created. Full office power failure recovery runbook with 5 steps + 2026-05-17 post-mortem.
- **`D:\claudetools\.claude\memory\infra_office_network.md`** — Created. ACG office infrastructure reference (IPs, hosts, roles, Docker containers, NPM proxy map).
- **`D:\claudetools\.claude\memory\MEMORY.md`** — Updated. Added entries for infra reference and power failure runbook.
- **`/boot/config/go` on Jupiter (172.16.3.20)** — Appended: `iptables -t nat -I PREROUTING -p tcp --dport 443 -j DNAT --to-destination 172.17.0.2:443`
- **pfSense `/cf/conf/config.xml`** — `sync` DNS override changed from 172.16.3.21 → 172.16.3.20.
- **pfSense `/var/unbound/host_entries.conf`** — Same change applied to running Unbound config.
- **`D:\claudetools\tmp_syncro_patch.py`** — Deleted (cleanup after use).
---
## Credentials & Secrets
None created or discovered this session.
---
## Infrastructure & Servers
| Host | IP | Role | Notes |
|------|----|------|-------|
| pfSense | 172.16.0.1 | Router, DNS, Tailscale subnet router | SSH port 2248, user admin |
| Jupiter | 172.16.3.20 | Unraid NAS | All VMs + Docker; root SSH |
| Uranus | 172.16.3.21 | OwnCloud additional storage | NOT a reverse proxy; no working SSH key from this machine |
| GuruRMM VM | 172.16.3.30 | Linux VM on Jupiter | GuruRMM server, Coord API (8001), MariaDB, SSH as guru |
| Pluto | 172.16.3.36 | Windows Server 2019 VM on Jupiter | MSI build server |
**Docker containers on Jupiter:**
| Container | Image | Ports |
|-----------|-------|-------|
| npm | jc21/nginx-proxy-manager | 1880→80, 7818→81, 18443→443 |
| seafile | seafileltd/seafile-pro-mc:12.0-latest | 8082→80 |
| seafile-mysql | mariadb:10.6 | internal |
| seafile-elasticsearch | elasticsearch:7.17.26 | internal |
| seafile-memcached | memcached:1.6.18 | internal |
**NPM SSL cert for sync.azcomputerguru.com:** `/etc/letsencrypt/live/npm-8/fullchain.pem` — valid until 2026-07-25.
**Tailscale subnet:** 172.16.0.0/22 advertised via pfSense node `pfsense-2` (100.119.153.74).
---
## Commands & Outputs
```bash
# Syncro roadmap patch
scp D:/claudetools/tmp_syncro_patch.py guru@172.16.3.30:/tmp/syncro_patch.py
ssh guru@172.16.3.30 "python3 /tmp/syncro_patch.py" # output: inserted
ssh guru@172.16.3.30 "cd /home/guru/gururmm && git add docs/FEATURE_ROADMAP.md && git commit -m '...' && git push origin main"
# Result: cef20dd pushed to azcomputerguru/gururmm main
# pfSense Tailscale fix
ssh -p 2248 admin@172.16.0.1 "tailscale up --advertise-routes=172.16.0.0/22 --accept-routes"
# libvirt recovery on Jupiter
ssh root@172.16.3.20 "losetup -f --show /mnt/user/system/libvirt/libvirt.img" # returned /dev/loop4
ssh root@172.16.3.20 "mount /dev/loop4 /etc/libvirt && libvirtd -d"
ssh root@172.16.3.20 "virsh -c qemu:///system list --all" # all 4 VMs running
# Seafile seahub fix
ssh root@172.16.3.20 "docker exec seafile bash -c 'cd /opt/seafile/seafile-pro-server-12.0.19 && ./seahub.sh start'"
# Verified: docker exec seafile curl -s -o /dev/null -w '%{http_code}' http://localhost:8000/ → 302
# iptables port 443 → NPM
ssh root@172.16.3.20 "iptables -t nat -I PREROUTING -p tcp --dport 443 -j DNAT --to-destination 172.17.0.2:443"
# Persisted: echo 'iptables ...' >> /boot/config/go
# pfSense DNS fix
ssh -p 2248 admin@172.16.0.1 "sed -i '' 's/172.16.3.21/172.16.3.20/g' /var/unbound/host_entries.conf"
ssh -p 2248 admin@172.16.0.1 "kill -HUP 62718" # Unbound SIGHUP (PID from /var/run/unbound.pid)
# Final verification
# sync.azcomputerguru.com:443 → True; HTTPS 200
```
---
## Pending / Incomplete Tasks
- **IX server BIOS power-restore setting:** Did not auto-restart after power loss. Check "Power On After Power Failure" in BIOS or IPMI. Should match Jupiter's behavior (auto-start).
- **Unraid libvirt auto-mount on boot:** The VM plugin should mount `libvirt.img` automatically but failed. A User Scripts plugin script triggered at array start would make the manual step 2c in the runbook unnecessary. Not implemented yet.
- **claudetools git remote URL:** Verified it was corrected to `http://172.16.3.20:3000` this session but it had reverted previously. Check other machines (MacBook, Howard) that they also have the internal URL.
- **`/sync` was never completed this session** due to network being down when first attempted. Sync will run at end of this `/save`.
- **Pluto vault entry + SSH key** — carried over from prior session, not addressed today.
- **BB-SERVER enrollment loop** — carried over, not addressed.
- **PowerShell `command_type` bug on Windows PS 5.1** — carried over, not addressed.
- **Policy wiring plan** (`ticklish-questing-stallman.md`) — still pending, not addressed.
---
## Reference Information
- **gururmm Syncro roadmap commit:** `cef20dd` — `docs: add Syncro PSA Webhook Integration to roadmap (Howard feature request 2026-05-17)`
- **Power failure runbook:** `D:\claudetools\.claude\POWER_FAILURE_RUNBOOK.md`
- **NPM admin panel:** `http://172.16.3.20:7818`
- **Seafile direct:** `http://172.16.3.20:8082`
- **Coord message to Howard (post-mortem):** `321679a2-d890-45e9-a19c-ca3701ad293b`
- **Coord message to Howard (Syncro roadmap):** `90d20ecb-eb40-4c40-86cc-6b52a8084304`