# 2026-06-18 — Dataforth testdatadb render recovery + render UI + new security-assessment app

## User
- **User:** Mike Swanson (mike)
- **Machine:** GURU-5070
- **Role:** admin

## Session Summary

Continued the Dataforth test-datasheet effort and added a new MSP tool. Started by syncing AD2's
notes (DSCA Fix-2 STAGE 2/3 complete, 92 DSCA models published) and discovering the DSCA33/45 spec
gap is **recoverable from the Hoffman API**: the original software published correct certs there
before the cryptolocker wipe, and our null-skipping renderer never overwrote them. Mined per-model
templates from Hoffman (`GET /api/v1/TestReportDataFiles/{serial}`) for 56/58 DSCA33/45 models and
handed them to AD2; AD2 later wired them, solved the frequency/AAC accuracy reverse-engineering
(54/56 validated), and published 1,452 new DSCA33/45 certs (0 overwrites).

Diagnosed the recurring AD2 "SSH lockouts": NOT a ban/account-lockout/flaky tunnel — a **PMTU
blackhole**. The OpenVPN tunnel path MTU is ~1424 but GURU-5070's tunnel adapter was MTU 1500, so
bulk/scp SSH segments were silently dropped (DF set) while small commands passed. Fixed by pinning
the adapter MTU to 1400 via GURU-5070's own SYSTEM RMM agent; validated a 1.41 MB SSH transfer.

Did substantial work on the testdatadb web UI (the multi-AI redesign): fixed the cert-fit bug
(replaced flaky CSS `zoom` with `transform: scale()`), added publish-state chips, built a same-origin
preview proxy, added `/api/search` sort/dir + wired the UI presets and Push-to-Web/Re-push buttons,
deployed the redesign as the live UI on AD2, then fixed the inspector not collapsing (results now
fill full screen) and replaced silent push feedback with explanatory toasts.

Investigated why ~9,624 records are unpublished: it is a **render-coverage gap**, not a backlog —
~98% render null and are correctly skipped. Root-caused the 8B/5B/SCM null renders to (1) a general
`parseRawData` bug (a PASS/FAIL line consumed as the step-response line for families that omit the
`"0","0",v` line) and (2) missing per-model Final-Test templates (8B45 == DSCA45 frequency-input).
Mined 136 8B/5B/SCM templates from Hoffman, built a stage+verify harness (content-normalized vs
Hoffman), and handed the remainder to AD2 since completion needs AD2's now-built machinery
(slotmaps / `Math.fround` / freq-AAC accuracy). Learned AD2 is coord-API-isolated → comms via sync only.

Built and deployed a new tool: **security.azcomputerguru.com** — a single-assessor client security
assessment intake app (PHP + MySQL on the IX cPanel host, gated by Cloudflare Access to only Mike).
Wizard identifies the client by Syncro phone, prefills, walks a risk-ordered questionnaire, and
generates live 365/Google read-only consent links so the audit scan runs after the consult.

## Key Decisions

- **Recover DSCA33/45 (and 8B/5B/SCM) specs from Hoffman, not from John/Dataforth.** The original
  certs survived on the Hoffman API; mining them is self-service and exact (validation oracle).
- **Diagnose AD2 SSH over RMM, not SSH** (the failing channel) — and fix MTU via GURU-5070's own
  SYSTEM RMM agent (the elevated lever when the Claude shell can't self-elevate).
- **Publish customer certs only after byte/content validation vs Hoffman.** Stage+verify before any
  publish; non-templated models stay null so the pipeline can't auto-publish wrong content.
- **Hand 8B/5B/SCM completion to AD2 rather than solo-reimplement** its slotmap/rounding/accuracy
  machinery in the file it actively edits. Comms to AD2 = git sync only (it has no coord API access).
- **security.azcomputerguru.com:** Cloudflare Access (zero app-auth, only Mike) + PHP/MySQL on the
  existing IX cPanel host (matches the Flarum/forum stack) + live in-app consent links.
- **Publish workflow correctness:** the pipeline *skips* null renders by design; the UI must say so
  (toasts), not silently report unpublished.

## Problems Encountered

- **AD2 SSH timeouts** → PMTU blackhole (tunnel MTU ~1424 vs adapter 1500). Fixed: adapter MTU 1400.
  Recurs if OpenVPN resets MTU on reconnect; durable fix = server-side `mssfix 1360`.
- **Cert-fit looked broken in preview** → the iframe was cross-origin (app on localhost, cert from
  AD2), so `fitCert` couldn't read `contentDocument`. Built `preview-proxy.py` (same-origin) to test.
- **`/syncro` comment POST piped to jq** returned an unparseable response and left it ambiguous
  whether the note posted (it hadn't). GET-verified, re-posted with `--data-binary @file`. Logged.
- **Coord messages/lock to AD2 were no-ops** — AD2 is network-isolated from the coord API. Use sync.
- **security app 403 "unable to read htaccess"** — cPanel created `.htaccess` after my chown; and
  local tests hit 127.0.0.1 (wrong vhost) instead of the account's bound IP (172.16.3.10). Fixed
  perms/ownership + tested via `--resolve ...:172.16.3.10`.
- **Push button gave no visible feedback** — it set "skipped" then `search()` re-rendered and wiped
  it. Replaced with persistent toasts.
- **Inspector reserved ~500px when empty** → results never filled the screen. Collapsed the
  inspector column until a record is selected; Esc closes it.
- **ff.py (Firefox preview driver) repeatedly 500s** — Playwright daemon won't start; screenshots
  skipped, preview served locally instead.

## Configuration Changes

Created:
- `projects/dataforth-dos/dsca33-45-templates.json` (56 models), `8b5bscm-templates.json` (136)
- `projects/dataforth-dos/tools/mine-hoffman-dsca.py`, `tools/preview-proxy.py`
- `projects/dataforth-dos/DSCA33-45-HOFFMAN-RECOVERY-2026-06-18.md`, `8B5BSCM-RENDER-VERIFY-2026-06-18.md`
- `projects/msp-tools/security-assessment/` — `app/{index.php,api.php,questions.json,schema.sql,config.sample.php,.gitignore}`, `DEPLOY.md`, `README.md`
- `.claude/memory/`: `ad2-ssh-mtu-blackhole.md`, `project_dsca33_45_resolved_via_hoffman.md`, `ad2-comms-via-sync-only.md`
- Session-log notes: `projects/dataforth-dos/session-logs/2026-06/2026-06-18-mike-8b5bscm-render-handoff.md`

Modified:
- `projects/dataforth-dos/testdatadb-fix/public/index.redesign.html` — cert-fit (transform-scale),
  publish chips, presets + Push-to-Web/Re-push wiring, inspector-collapse (full-screen), push toasts
- `projects/dataforth-dos/testdatadb-fix/routes/api.js` — added `/api/search` sort/dir (synced from live AD2)

Deployed (AD2, on the box, not git): `routes/api.js` (sort) + service restart; `public/index.html`
(the redesign, replacing the old basic UI). Backups: `index.html.bak-pre-redesign-20260618`,
`routes/api.js.bak-<ts>`.

Deployed (IX cPanel acct `azcomputerguru`): subdomain `security.azcomputerguru.com`
(docroot `/home/azcomputerguru/public_html/security`), MySQL DB `azcomputerguru_secassess` +
user `azcomputerguru_secapp`, app files, `config.php`, PHP 8.1. Cloudflare: DNS `security` A
72.194.62.5 proxied; Access app `8ce5f31c-4f4e-4883-bae1-f7606e5b06c0` + policy `Only Mike`.

## Credentials & Secrets

- **Dataforth Hoffman/Product API** (discovered plaintext in AD2 `C:\ProgramData\dataforth-uploader\credentials.json`):
  OAuth2 client-credentials. Vaulted `clients/dataforth/hoffman-product-api`.
  token_url `https://login.dataforth.com/connect/token`, api_base `https://www.dataforth.com`,
  client_id `dataforth.onprem.sync`, secret `Trxvwee2234-Awer8723-2`, scope `dataforth.web`.
  (Same file also held a Dataforth Graph app — vaulted alongside.)
- **security-assessment DB**: vaulted `msp-tools/security-assessment-db` (user `azcomputerguru_secapp`,
  db `azcomputerguru_secassess`, host localhost). App also uses Mike's Syncro key (baked in config).
- **Cloudflare** (used, already vaulted `services/cloudflare`): zone_id `1beb9917c22b54be32e5215df2c227ce`,
  account_id `44594c346617d918bd3302a00b07e122`, full-account token `cfat_...`, dns token.
- testdatadb Postgres (vaulted earlier `clients/dataforth/testdatadb-postgres`): app `testdatadb_app`/`DfTestDB2026!`.

## Infrastructure & Servers

- **AD2** (Dataforth, `192.168.0.6`, RMM agent `cfa93bb6-0cdc-4d4e-a29e-1609cda6f047`): testdatadb
  Node+PG18 web app on :3000; renderer `templates/datasheet-exact.js`; uploader pushes to Hoffman.
  Reached via SSH `sysadmin@192.168.0.6` (key auth; MTU-sensitive) or RMM (reliable, SYSTEM).
- **GURU-5070**: OpenVPN tunnel adapter "Local Area Connection" (ifIndex 12, IP 192.168.6.2) — MTU
  pinned 1400. RMM agent `819df0c8-...` (SYSTEM). Tunnel PMTU ~1424.
- **IX** (`172.16.3.10`, cPanel 134/CloudLinux): account `azcomputerguru` hosts the ACG subdomains
  (PHP 8.1, MySQL). Root SSH key auth from GURU-5070 works. vhosts bind to 172.16.3.10 (not 127.0.0.1).
- **Hoffman/Dataforth API**: `https://www.dataforth.com/api/v1/TestReportDataFiles/{bulk|{serial}|stats}`;
  token at `https://login.dataforth.com/connect/token`.
- **security.azcomputerguru.com**: Cloudflare-proxied → IX origin 72.194.62.5; Access-gated to Mike.

## Commands & Outputs

- MTU fix (via GURU-5070 RMM SYSTEM): `Set-NetIPInterface -InterfaceIndex 12 -AddressFamily IPv4 -NlMtuBytes 1400` → validated 1.41 MB SSH transfer, no read-error.
- DF ping MTU probe: wire 1424 passes, 1428 drops → PMTU ~1424.
- Publish test 180300-1 (5B39-03): `POST /api/upload {serialNumbers:["180300-1"]}` → `{created:0,updated:0,unchanged:0,errors:0,skipped:1}` (renders null → correctly skipped).
- DSCA mine: `python projects/dataforth-dos/tools/mine-hoffman-dsca.py <map> <out>` → 56 / 136 models.
- 8B/5B/SCM verify (template-gated only): 15 content-perfect, 70 NULL (need slotmaps), rest precision/accuracy.
- security app provision: `uapi --user=azcomputerguru SubDomain addsubdomain ... / Mysql create_database|create_user|set_privileges_on_database` (all status 1).
- Cloudflare: created DNS A record + Access app + "Only Mike" policy; unauth request → `403 Cf-Mitigated: challenge`.

## Pending / Incomplete Tasks

- **8B/5B/SCM render completion (open decision):** wire the 136 mined templates into the live
  renderer (slotmaps + QB rounding + freq/AAC accuracy — AD2's now-proven machinery). 5B39 (e.g.
  180300-1) won't publish until done. Choice: route to AD2 (clean, sync handoff already on its
  branch) vs drive it directly (risk: editing AD2's active file). Awaiting Mike's call.
- **AD2 8B38 (0.78) + 7B family** need family-specific render work.
- **AD2 SSH MTU**: re-pin if OpenVPN resets it; recommend server-side `mssfix 1360` (Mike/Dataforth).
- **security.azcomputerguru.com**: Mike to test the Cloudflare Access login (one-time PIN). Google
  read-only OAuth client still to be created (365 consent works now). Register the consent redirect
  URI on the 365 Investigator app.
- DSCA33/45: 2 niche models (DSCA33-1948, DSCA45-1746) + 8B/5B equivalents have no Hoffman original.

## Reference Information

- Commits (main): `419d6e58` DSCA33/45 recovery · `72a2fbe6`/`760c2264` testdatadb UI+API · `82ae3499`/`0745f5d0` 8B5BSCM mine+verify · `03527db0`/`b2a8d0c1` security app · `271dd8e1` inspector-collapse · `15f0941a` push toasts.
- AD2 branch: `3a7ac35d` wire DSCA33/45 · `b5bc0409` accuracy-RE 54/56 · `4615877f` publish 1,452 certs.
- Syncro ticket `#32441` (Dataforth datasheet); hidden notes posted `419648732`, `419661734`.
- Cloudflare Access app id `8ce5f31c-4f4e-4883-bae1-f7606e5b06c0`.
- Memories: `ad2-ssh-mtu-blackhole`, `project_dsca33_45_resolved_via_hoffman`, `ad2-comms-via-sync-only`.
