azcomputerguru/claudetools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
c2e5f4faeb68588e939b7c73cfaa00a795aa6d7d
claudetools/session-logs/2026-06/2026-06-18-mike-testdatadb-render-and-security-app.md
Mike Swanson da3fce0c47 sync: auto-sync from GURU-5070 at 2026-06-18 17:06:16 
Author: Mike Swanson
Machine: GURU-5070
Timestamp: 2026-06-18 17:06:16
2026-06-18 17:48:02 -07:00

11 KiB
Raw Blame History

2026-06-18 — Dataforth testdatadb render recovery + render UI + new security-assessment app

User

  • User: Mike Swanson (mike)
  • Machine: GURU-5070
  • Role: admin

Session Summary

Continued the Dataforth test-datasheet effort and added a new MSP tool. Started by syncing AD2's notes (DSCA Fix-2 STAGE 2/3 complete, 92 DSCA models published) and discovering the DSCA33/45 spec gap is recoverable from the Hoffman API: the original software published correct certs there before the cryptolocker wipe, and our null-skipping renderer never overwrote them. Mined per-model templates from Hoffman (GET /api/v1/TestReportDataFiles/{serial}) for 56/58 DSCA33/45 models and handed them to AD2; AD2 later wired them, solved the frequency/AAC accuracy reverse-engineering (54/56 validated), and published 1,452 new DSCA33/45 certs (0 overwrites).

Diagnosed the recurring AD2 "SSH lockouts": NOT a ban/account-lockout/flaky tunnel — a PMTU blackhole. The OpenVPN tunnel path MTU is ~1424 but GURU-5070's tunnel adapter was MTU 1500, so bulk/scp SSH segments were silently dropped (DF set) while small commands passed. Fixed by pinning the adapter MTU to 1400 via GURU-5070's own SYSTEM RMM agent; validated a 1.41 MB SSH transfer.

Did substantial work on the testdatadb web UI (the multi-AI redesign): fixed the cert-fit bug (replaced flaky CSS zoom with transform: scale()), added publish-state chips, built a same-origin preview proxy, added /api/search sort/dir + wired the UI presets and Push-to-Web/Re-push buttons, deployed the redesign as the live UI on AD2, then fixed the inspector not collapsing (results now fill full screen) and replaced silent push feedback with explanatory toasts.

Investigated why ~9,624 records are unpublished: it is a render-coverage gap, not a backlog — ~98% render null and are correctly skipped. Root-caused the 8B/5B/SCM null renders to (1) a general parseRawData bug (a PASS/FAIL line consumed as the step-response line for families that omit the "0","0",v line) and (2) missing per-model Final-Test templates (8B45 == DSCA45 frequency-input). Mined 136 8B/5B/SCM templates from Hoffman, built a stage+verify harness (content-normalized vs Hoffman), and handed the remainder to AD2 since completion needs AD2's now-built machinery (slotmaps / Math.fround / freq-AAC accuracy). Learned AD2 is coord-API-isolated → comms via sync only.

Built and deployed a new tool: security.azcomputerguru.com — a single-assessor client security assessment intake app (PHP + MySQL on the IX cPanel host, gated by Cloudflare Access to only Mike). Wizard identifies the client by Syncro phone, prefills, walks a risk-ordered questionnaire, and generates live 365/Google read-only consent links so the audit scan runs after the consult.

Key Decisions

  • Recover DSCA33/45 (and 8B/5B/SCM) specs from Hoffman, not from John/Dataforth. The original certs survived on the Hoffman API; mining them is self-service and exact (validation oracle).
  • Diagnose AD2 SSH over RMM, not SSH (the failing channel) — and fix MTU via GURU-5070's own SYSTEM RMM agent (the elevated lever when the Claude shell can't self-elevate).
  • Publish customer certs only after byte/content validation vs Hoffman. Stage+verify before any publish; non-templated models stay null so the pipeline can't auto-publish wrong content.
  • Hand 8B/5B/SCM completion to AD2 rather than solo-reimplement its slotmap/rounding/accuracy machinery in the file it actively edits. Comms to AD2 = git sync only (it has no coord API access).
  • security.azcomputerguru.com: Cloudflare Access (zero app-auth, only Mike) + PHP/MySQL on the existing IX cPanel host (matches the Flarum/forum stack) + live in-app consent links.
  • Publish workflow correctness: the pipeline skips null renders by design; the UI must say so (toasts), not silently report unpublished.

Problems Encountered

  • AD2 SSH timeouts → PMTU blackhole (tunnel MTU ~1424 vs adapter 1500). Fixed: adapter MTU 1400. Recurs if OpenVPN resets MTU on reconnect; durable fix = server-side mssfix 1360.
  • Cert-fit looked broken in preview → the iframe was cross-origin (app on localhost, cert from AD2), so fitCert couldn't read contentDocument. Built preview-proxy.py (same-origin) to test.
  • /syncro comment POST piped to jq returned an unparseable response and left it ambiguous whether the note posted (it hadn't). GET-verified, re-posted with --data-binary @file. Logged.
  • Coord messages/lock to AD2 were no-ops — AD2 is network-isolated from the coord API. Use sync.
  • security app 403 "unable to read htaccess" — cPanel created .htaccess after my chown; and local tests hit 127.0.0.1 (wrong vhost) instead of the account's bound IP (172.16.3.10). Fixed perms/ownership + tested via --resolve ...:172.16.3.10.
  • Push button gave no visible feedback — it set "skipped" then search() re-rendered and wiped it. Replaced with persistent toasts.
  • Inspector reserved ~500px when empty → results never filled the screen. Collapsed the inspector column until a record is selected; Esc closes it.
  • ff.py (Firefox preview driver) repeatedly 500s — Playwright daemon won't start; screenshots skipped, preview served locally instead.

Configuration Changes

Created:

  • projects/dataforth-dos/dsca33-45-templates.json (56 models), 8b5bscm-templates.json (136)
  • projects/dataforth-dos/tools/mine-hoffman-dsca.py, tools/preview-proxy.py
  • projects/dataforth-dos/DSCA33-45-HOFFMAN-RECOVERY-2026-06-18.md, 8B5BSCM-RENDER-VERIFY-2026-06-18.md
  • projects/msp-tools/security-assessment/ — app/{index.php,api.php,questions.json,schema.sql,config.sample.php,.gitignore}, DEPLOY.md, README.md
  • .claude/memory/: ad2-ssh-mtu-blackhole.md, project_dsca33_45_resolved_via_hoffman.md, ad2-comms-via-sync-only.md
  • Session-log notes: projects/dataforth-dos/session-logs/2026-06/2026-06-18-mike-8b5bscm-render-handoff.md

Modified:

  • projects/dataforth-dos/testdatadb-fix/public/index.redesign.html — cert-fit (transform-scale), publish chips, presets + Push-to-Web/Re-push wiring, inspector-collapse (full-screen), push toasts
  • projects/dataforth-dos/testdatadb-fix/routes/api.js — added /api/search sort/dir (synced from live AD2)

Deployed (AD2, on the box, not git): routes/api.js (sort) + service restart; public/index.html (the redesign, replacing the old basic UI). Backups: index.html.bak-pre-redesign-20260618, routes/api.js.bak-<ts>.

Deployed (IX cPanel acct azcomputerguru): subdomain security.azcomputerguru.com (docroot /home/azcomputerguru/public_html/security), MySQL DB azcomputerguru_secassess + user azcomputerguru_secapp, app files, config.php, PHP 8.1. Cloudflare: DNS security A 72.194.62.5 proxied; Access app 8ce5f31c-4f4e-4883-bae1-f7606e5b06c0 + policy Only Mike.

Credentials & Secrets

  • Dataforth Hoffman/Product API (discovered plaintext in AD2 C:\ProgramData\dataforth-uploader\credentials.json): OAuth2 client-credentials. Vaulted clients/dataforth/hoffman-product-api. token_url https://login.dataforth.com/connect/token, api_base https://www.dataforth.com, client_id dataforth.onprem.sync, secret Trxvwee2234-Awer8723-2, scope dataforth.web. (Same file also held a Dataforth Graph app — vaulted alongside.)
  • security-assessment DB: vaulted msp-tools/security-assessment-db (user azcomputerguru_secapp, db azcomputerguru_secassess, host localhost). App also uses Mike's Syncro key (baked in config).
  • Cloudflare (used, already vaulted services/cloudflare): zone_id 1beb9917c22b54be32e5215df2c227ce, account_id 44594c346617d918bd3302a00b07e122, full-account token cfat_..., dns token.
  • testdatadb Postgres (vaulted earlier clients/dataforth/testdatadb-postgres): app testdatadb_app/DfTestDB2026!.

Infrastructure & Servers

  • AD2 (Dataforth, 192.168.0.6, RMM agent cfa93bb6-0cdc-4d4e-a29e-1609cda6f047): testdatadb Node+PG18 web app on :3000; renderer templates/datasheet-exact.js; uploader pushes to Hoffman. Reached via SSH sysadmin@192.168.0.6 (key auth; MTU-sensitive) or RMM (reliable, SYSTEM).
  • GURU-5070: OpenVPN tunnel adapter "Local Area Connection" (ifIndex 12, IP 192.168.6.2) — MTU pinned 1400. RMM agent 819df0c8-... (SYSTEM). Tunnel PMTU ~1424.
  • IX (172.16.3.10, cPanel 134/CloudLinux): account azcomputerguru hosts the ACG subdomains (PHP 8.1, MySQL). Root SSH key auth from GURU-5070 works. vhosts bind to 172.16.3.10 (not 127.0.0.1).
  • Hoffman/Dataforth API: https://www.dataforth.com/api/v1/TestReportDataFiles/{bulk|{serial}|stats}; token at https://login.dataforth.com/connect/token.
  • security.azcomputerguru.com: Cloudflare-proxied → IX origin 72.194.62.5; Access-gated to Mike.

Commands & Outputs

  • MTU fix (via GURU-5070 RMM SYSTEM): Set-NetIPInterface -InterfaceIndex 12 -AddressFamily IPv4 -NlMtuBytes 1400 → validated 1.41 MB SSH transfer, no read-error.
  • DF ping MTU probe: wire 1424 passes, 1428 drops → PMTU ~1424.
  • Publish test 180300-1 (5B39-03): POST /api/upload {serialNumbers:["180300-1"]} → {created:0,updated:0,unchanged:0,errors:0,skipped:1} (renders null → correctly skipped).
  • DSCA mine: python projects/dataforth-dos/tools/mine-hoffman-dsca.py <map> <out> → 56 / 136 models.
  • 8B/5B/SCM verify (template-gated only): 15 content-perfect, 70 NULL (need slotmaps), rest precision/accuracy.
  • security app provision: uapi --user=azcomputerguru SubDomain addsubdomain ... / Mysql create_database|create_user|set_privileges_on_database (all status 1).
  • Cloudflare: created DNS A record + Access app + "Only Mike" policy; unauth request → 403 Cf-Mitigated: challenge.

Pending / Incomplete Tasks

  • 8B/5B/SCM render completion (open decision): wire the 136 mined templates into the live renderer (slotmaps + QB rounding + freq/AAC accuracy — AD2's now-proven machinery). 5B39 (e.g. 180300-1) won't publish until done. Choice: route to AD2 (clean, sync handoff already on its branch) vs drive it directly (risk: editing AD2's active file). Awaiting Mike's call.
  • AD2 8B38 (0.78) + 7B family need family-specific render work.
  • AD2 SSH MTU: re-pin if OpenVPN resets it; recommend server-side mssfix 1360 (Mike/Dataforth).
  • security.azcomputerguru.com: Mike to test the Cloudflare Access login (one-time PIN). Google read-only OAuth client still to be created (365 consent works now). Register the consent redirect URI on the 365 Investigator app.
  • DSCA33/45: 2 niche models (DSCA33-1948, DSCA45-1746) + 8B/5B equivalents have no Hoffman original.

Reference Information

  • Commits (main): 419d6e58 DSCA33/45 recovery · 72a2fbe6/760c2264 testdatadb UI+API · 82ae3499/0745f5d0 8B5BSCM mine+verify · 03527db0/b2a8d0c1 security app · 271dd8e1 inspector-collapse · 15f0941a push toasts.
  • AD2 branch: 3a7ac35d wire DSCA33/45 · b5bc0409 accuracy-RE 54/56 · 4615877f publish 1,452 certs.
  • Syncro ticket #32441 (Dataforth datasheet); hidden notes posted 419648732, 419661734.
  • Cloudflare Access app id 8ce5f31c-4f4e-4883-bae1-f7606e5b06c0.
  • Memories: ad2-ssh-mtu-blackhole, project_dsca33_45_resolved_via_hoffman, ad2-comms-via-sync-only.
Reference in New Issue View Git Blame Copy Permalink