dataforth/dsca33-45: recover lost specs from Hoffman API (56/58 models)
The DSCA33/DSCA45 main spec files lost in the cryptolocker wipe are recoverable: the original software published correct certs to the Hoffman product API before the wipe and our null-skipping renderer never overwrote them. Mine per-model Final-Test templates (names + specs + verbatim accuracy headers) straight from those originals instead of requesting spec files from Dataforth/John. - dsca33-45-templates.json: 56 models (DSCA33 34/35, DSCA45 22/23); only DSCA33-1948 + DSCA45-1746 (24 units) lack an original. - mine-hoffman-dsca.py: the re-runnable miner. - DSCA33-45-HOFFMAN-RECOVERY handoff for the AD2 session (incl. the gate: validate each render vs its Hoffman original before enabling live rendering). - memories: Hoffman recovery (supersedes the spec-gap "need John" note) and the AD2 SSH MTU-blackhole root cause/fix; errorlog entries (syncro jq, ssh correction). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -32,6 +32,8 @@
|
||||
- [AAD Connect msDS-KeyCredentialLink writeback](reference_aadconnect_keycredlink_writeback.md) — "completed-export-errors" + 8344 INSUFF_ACCESS_RIGHTS on a protected admin account = WHfB key writeback blocked by AdminSDHolder. Diagnose with csexport /f:x; fix with dsacls WP;msDS-KeyCredentialLink on AdminSDHolder + SDProp.
|
||||
- [UniFi Site Manager cloud API](reference_unifi_site_manager_api.md) — `api.ui.com` + `X-API-KEY` (vault `services/unifi-site-manager`) = remote access to the WHOLE ACG UniFi fleet (~36 consoles) outside UOS. Tier1 `/v1/hosts|sites|devices|isp-metrics` = inventory+health+WAN. Tier2 CONNECTOR `/v1/connector/consoles/{id}/proxy/network/api/s/default/stat/{device,sta}` = **full UOS parity** (per-radio cu_total airtime + per-client RSSI) for ANY console, remote. Backend `unifi-wifi/scripts/gw-sitemanager.sh` (`fleet|devices|sites|isp|net`). Standalone UDM WAN SSH usually firewalled; per-console SSH pw at `clients/<slug>/udm-ssh`.
|
||||
- [reference_sqlx_migrations_immutable](reference_sqlx_migrations_immutable.md) -- NEVER edit an already-applied sqlx migration file — even a comment. sqlx::migrate! checksums each file at compile time and validates against _sqlx_migrations at startup; a changed checksum crash-loops the server with "migration N was previously applied but has been modified". Code review MUST flag any edit to an applied migration.
|
||||
- [AD2 SSH MTU blackhole](ad2-ssh-mtu-blackhole.md) — AD2 SSH "lockouts"/mid-session read-errors over the Dataforth OpenVPN were a PMTU blackhole (tunnel PMTU ~1424 vs adapter MTU 1500), NOT a ban/account-lockout/flaky tunnel. Fix: pin the OpenVPN adapter MTU to 1400 (done on GURU-5070 via its SYSTEM RMM agent); permanent = `mssfix 1360` on the OpenVPN server. Diagnose over RMM, not SSH.
|
||||
- [DSCA33/45 resolved via Hoffman](project_dsca33_45_resolved_via_hoffman.md) — The "lost" DSCA33/45 spec files are recoverable from the Hoffman API (original certs survived the wipe); do NOT ask John. 56/58 models mined into projects/dataforth-dos/dsca33-45-templates.json; only DSCA33-1948 + DSCA45-1746 (24 units) lack an original. AD2 handoff: DSCA33-45-HOFFMAN-RECOVERY-2026-06-18.md.
|
||||
|
||||
## Users
|
||||
- [Howard Enos](user_howard.md) — Mike's brother, technician, full access. Machines: ACG-TECH03L, Howard-Home (authoritative in users.json).
|
||||
|
||||
40
.claude/memory/ad2-ssh-mtu-blackhole.md
Normal file
40
.claude/memory/ad2-ssh-mtu-blackhole.md
Normal file
@@ -0,0 +1,40 @@
|
||||
---
|
||||
name: ad2-ssh-mtu-blackhole
|
||||
description: AD2 SSH "lockouts"/mid-session timeouts over the Dataforth OpenVPN were an MTU/PMTU blackhole, not a ban/account-lockout/flaky tunnel; fix = pin the tunnel adapter MTU to 1400
|
||||
metadata:
|
||||
type: project
|
||||
---
|
||||
|
||||
AD2 (Dataforth, `192.168.0.6`) SSH from the fleet over OpenVPN (client subnet `192.168.6.x`)
|
||||
intermittently looked "locked out": sessions **authenticated fine**, then died mid-session with
|
||||
`Read error from remote host 192.168.6.2 ... Unknown error [postauth]` and
|
||||
`ssh_dispatch_run_fatal: Connection from authenticating user sysadmin ... Connection timed out [preauth]`.
|
||||
Small/interactive commands often worked; bulk reads + `scp` stalled.
|
||||
|
||||
**Root cause (diagnosed 2026-06-18 via RMM — SSH itself was the failing channel, so don't diagnose it over SSH):**
|
||||
- NOT account lockout — Windows lockout threshold is 5/30min but **zero 4740 events**; `sysadmin` never locked.
|
||||
- NOT an IP ban — **no IPBan/wail2ban/RdpGuard**, **0 inbound firewall block rules**.
|
||||
- NOT auth — **every** `Accepted publickey for sysadmin` succeeded.
|
||||
- NOT load — AD2 was CPU ~11%, 11.7 GB RAM free.
|
||||
- It was a **PMTU blackhole.** OpenVPN tunnel path MTU is **~1424** (DF ping: wire 1424 passes,
|
||||
1428 drops). But GURU-5070's OpenVPN adapter (`Local Area Connection`, ifIndex 12, IP
|
||||
`192.168.6.2`) was set to **MTU 1500** → TCP negotiated MSS 1460 → full-size bulk/scp segments
|
||||
exceeded the tunnel and were **silently dropped (DF set)**, while sub-MTU interactive packets
|
||||
passed. That is why it presented as random "lockouts" that got worse with bulk transfer.
|
||||
|
||||
**Fix applied (2026-06-18):** `Set-NetIPInterface -InterfaceIndex 12 -AddressFamily IPv4 -NlMtuBytes 1400`
|
||||
run via **GURU-5070's own RMM agent** (`819df0c8...`, runs as `nt authority\system` = elevated; the
|
||||
elevated lever on the local box when you can't self-elevate from the Claude shell). Validated: a
|
||||
**1.41 MB single-session SSH transfer to AD2 completed in 9s, no read error** (previously blackholed).
|
||||
`~/.ssh/config` `ad2` block annotated + tightened keepalives (`ServerAliveInterval 15`,
|
||||
`ServerAliveCountMax 4`, `ConnectTimeout 20`).
|
||||
|
||||
**Durability / permanent fix:** `Set-NetIPInterface` is registry-persistent, but **OpenVPN Connect may
|
||||
reset the adapter MTU to 1500 on reconnect** — re-apply if SSH bulk transfers start stalling again
|
||||
(check `Get-NetIPInterface -InterfaceIndex 12`). The real permanent fix is **server-side on the
|
||||
Dataforth OpenVPN server: `mssfix 1360` (or `push "tun-mtu 1400"`)** so every fleet client clamps
|
||||
automatically — `192.168.6.4` showed the identical symptom, so this is fleet-wide, not 5070-only.
|
||||
|
||||
Corrects the earlier wrong attribution ("flaky VPN tunnel" / "my rapid scp+ssh bursts triggering a
|
||||
ban") — the tunnel is up and stable for small packets; only over-MSS segments were dropped. See
|
||||
[[prefer-ssh-over-rmm]] (RMM-as-fallback guidance still holds; the *reason* was MTU, not a flaky VPN).
|
||||
27
.claude/memory/project_dsca33_45_resolved_via_hoffman.md
Normal file
27
.claude/memory/project_dsca33_45_resolved_via_hoffman.md
Normal file
@@ -0,0 +1,27 @@
|
||||
---
|
||||
name: project_dsca33_45_resolved_via_hoffman
|
||||
description: DSCA33/45 "lost spec files" are recoverable from the Hoffman API (original certs survived the wipe) — do NOT request spec files from John; mine templates from Hoffman instead
|
||||
metadata:
|
||||
type: project
|
||||
---
|
||||
|
||||
The DSCA33/DSCA45 main spec files lost in Dataforth's cryptolocker wipe (which blocked rendering
|
||||
~8,763 certs and prompted "ask John for the spec files") are **recoverable** — the original software
|
||||
published correct DSCA33/45 certs to the **Hoffman API** before the wipe, and our null-skipping
|
||||
pipeline never overwrote them. **Do not ask John for spec files.** Supersedes the FIX2-5 handoff's
|
||||
TODO 2 and the `ad2`-branch memory `project_dsca33_45_spec_gap` (which says "blocked, need John").
|
||||
|
||||
Mined **56 of 58 models** straight from Hoffman into `projects/dataforth-dos/dsca33-45-templates.json`
|
||||
(per model: `accOut`, verbatim 2-line `accHeader`, Final-Test `rows` of name+spec, and a known-good
|
||||
`_srcSerial`). Only 2 niche models have no original anywhere: **DSCA33-1948 (16u)**, **DSCA45-1746 (8u)**.
|
||||
Coverage: ~7,157 units already correct + live on Hoffman (no action); ~1,580 not-yet-uploaded units
|
||||
need rendering from the mined templates + AD2's already-derived slotMaps.
|
||||
|
||||
AD2 handoff + the critical gate: `projects/dataforth-dos/DSCA33-45-HOFFMAN-RECOVERY-2026-06-18.md`.
|
||||
**Critical:** validate each model's render byte-for-byte against its Hoffman original BEFORE enabling
|
||||
live DSCA33/45 rendering — once the renderer returns non-null, the pipeline stops skipping these and
|
||||
will re-push/UPDATE the 7,157 good originals on the next cycle (safe only if the render matches).
|
||||
|
||||
Hoffman read API: `GET https://www.dataforth.com/api/v1/TestReportDataFiles/{serial}` (returns
|
||||
`{SerialNumber,Content,CreatedAtUtc,UpdatedAtUtc}`); creds vault `clients/dataforth/hoffman-product-api`.
|
||||
Miner: `projects/dataforth-dos/tools/mine-hoffman-dsca.py`. AD2 access notes: [[ad2-ssh-mtu-blackhole]].
|
||||
Reference in New Issue
Block a user