Some DSCA subtypes' raw_data STATUS groups carry more (or fewer) value-bearing
entries than the template's spec-bearing rows (the test program measures slots the
printed sheet omits, e.g. DSCA49's 5mA load pair), so the in-order zip misaligned
values and those models were skipped by the count-guard.
New tool derive-dsca-slotmaps.js derives a per-model slotMap (absolute statusEntries
index per spec-bearing row) by greedily matching a staged original's printed values
to the DB raw_data STATUS entries (same fround formatting), then picking the
candidate map that validates against the most units. Models are grouped by identical
row-name signature and one map is derived per group from all sibling units — this
disambiguates duplicate values (e.g. a unit where 5mA != 50mA linearity forces the
correct slot; DSCA49-04 alone has only 2 staged units that can't, but its siblings'
25 units do). Stored as `slotMap` in dsca-templates.json.
Renderer: consults slotMap only when the sequential zip fails (value count !=
spec-row count), so the 88 already-clean models keep their path (no regression) and
ambiguous ones pull the right value via the map.
STAGE 3 re-validation: FINAL-TEST CLEAN 88 -> 92; 134 more certs now render
(null 450 -> 316); matches 2278 -> 2412. Same 6 retest-vintage dirty models, no
new mismatches. DSCA49 family + DSCA40-03 group now clean and validated.
Still blocked (separate gap, NOT layout ambiguity): DSCA45-* and most DSCA33-*
render null because they have NO spec-reader entries (render-datasheet bails before
rendering). Their slotMaps are derived and ready; they need spec coverage. One DSCA33
group (DSCA33-02/03/03A/04/05/1948) did not reach the slotMap validation threshold
(best 19/35 units) and stays skipped pending more/cleaner staged samples.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause of the DSCA39 footer mismatch: the "Standard output load for test is
250 ohms." line is a footer note, not a parameter, but the STAGE 1 extractor
captured it as a (column-truncated) row "Standard output load for te". And the
renderer's OUTSIGTYPE==='CURRENT' emission was wrong on both ends — it printed the
note (after the underline, invisible to the validator gate) for many -C current
models whose staged originals never had it, and never placed it correctly for the
models that do.
Fix is data-driven, matching the rest of the template approach:
- derive-dsca-templates.js: detect the "Standard output load..." line, capture it
as a per-model `loadNote` property, and exclude it from rows. Regenerated
dsca-templates.json — surgically clean: only the 5 DSCA39 models changed (lost
the truncated row, gained loadNote); all 121 others byte-identical.
- datasheet-exact.js: emit `dscaTpl.loadNote` (blank line + note) before the footer
underline, only for models that have it; removed the OUTSIGTYPE-based emission.
STAGE 3 re-validation: FINAL-TEST CLEAN 85 -> 88, mismatches 9 -> 6, matches
2206 -> 2278. DSCA39-01/02/07 now fully clean; DSCA39-01 byte-content-verified.
No regression — the -C current models stayed clean and no longer carry the
spurious after-underline note.
The 6 remaining dirty models (DSCA38-05/-1793/-19C/-19E, DSCA39-05, DSCA39-1950)
are ALL retest data-vintage: the staged .TXT is an older test run than the DB
latest-wins record (Supply Current / Linearity differ by more than rounding).
Not render bugs — cannot be reconciled against an older sheet.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Restarted testdatadb service (new template live), canary-pushed 1 cert (updated,
0 errors), then re-pushed all PASS certs for the 68 Final-Test-content-clean
models via uploadBySerialNumbers from a fresh node process.
Result over 30,423 PASS certs: updated=26022 unchanged=2738 created=0 errors=0
skipped=1663. The 26,022 updates replace the old defective DSCA renders on the
live site with the rebuilt, byte-content-validated ones. The 1,663 skips are the
count-guard correctly refusing any individual cert whose value count != its
template row count (ambiguous) — never publishing misaligned data.
Artifacts: push-clean68.js (driver), dsca-clean68-models.json (the published
set). NOT yet published: the 26 last-digit-diff models, the ~32 ambiguous/null
layout families, and the 231 untemplated models.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
validate-dsca-stage3.js: read-only harness that, for every staged DSCA original
we have ground truth for (2806 across 126 models), looks up the DB record,
renders it through the live path, and content-compares. GATE = the FINAL TEST
RESULTS section (rule lines canonicalized, whitespace collapsed — so the deferred
column-spacing cosmetic doesn't register); accuracy-section diffs reported
separately as informational.
First run verdict (report attached):
- 68 models FINAL-TEST CONTENT-CLEAN (0 mismatch over compared certs).
- 2123/2316 certs match the Final-Test content exactly (91.7%).
- 26 models show measured-value last-digit diffs only — structure (names, specs,
row alignment, statuses) is correct. Two root causes, neither structural:
* rounding-mode: JS double toFixed vs QB single-precision half-up
(e.g. raw 9.9995 code3 -> "9.999" here, "10.000" in golden). Fixable but
float-precision-sensitive; risks regressing currently-clean values.
* data vintage: staged .TXT is a different test run than the DB latest-wins
record (Fix 3) — e.g. Supply Current 19.6 vs 20.3, 0.7 apart. Not a render
bug; can't be reconciled against an older staged sheet.
- ~32 models render null (count-guard): DSCA33-*, DSCA45-*, DSCA49-* families
whose raw_data carries load points the template omits -> need per-subtype slot
mapping (the canonical-slot approach) before they can render.
Still NOT published: service not restarted, nothing re-pushed to Hoffman.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Backend (deployed live on AD2, service restarted, + repo copy resynced — it was
far behind the deployed server):
- /api/search: add whitelisted sort/dir (NULLS LAST) so sortable headers and the
"Latest uploads" preset work. web_status filter and POST /api/upload already
existed on the server; the stale repo copy now matches live.
Frontend (redesign prototype):
- "Latest uploads" preset (web_status=on + sort=api_uploaded_at desc) and
"Not yet published" (web_status=off) are now active presets.
- Push to Web (inspector) + Re-push (multi-select) wired to POST /api/upload
behind a confirm() gate; refresh WEB status after. Validated idempotently on a
published record (unchanged:1, errors:0).
- "Retested units" stays disabled — needs a retest flag in the pipeline (next).
tools/preview-proxy.py: forward POST so the publish buttons work in same-origin preview.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- fitCert: replace the flaky CSS `zoom` (Firefox support is recent/inconsistent)
with transform:scale() measured against the widest line (+ right margin and
font-load retries) so the cert always scales to fit the inspector with no
horizontal clip. Validated live on a narrow 5B cert (0.74x) and a wide DSCA45
cert (0.55x) against the real AD2 dataset.
- inspector Web field -> Published (green) / Not published (amber) chips.
- widen default inspector 480 -> 500px.
- tools/preview-proxy.py: serve the prototype AND reverse-proxy /api to the live
AD2 server so the cert iframe is same-origin during preview — styleCert/fitCert
read iframe.contentDocument, which silently no-ops when the iframe is loaded
cross-origin straight from AD2 (why the fit looked broken in earlier previews).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The DSCA33/DSCA45 main spec files lost in the cryptolocker wipe are recoverable:
the original software published correct certs to the Hoffman product API before
the wipe and our null-skipping renderer never overwrote them. Mine per-model
Final-Test templates (names + specs + verbatim accuracy headers) straight from
those originals instead of requesting spec files from Dataforth/John.
- dsca33-45-templates.json: 56 models (DSCA33 34/35, DSCA45 22/23); only
DSCA33-1948 + DSCA45-1746 (24 units) lack an original.
- mine-hoffman-dsca.py: the re-runnable miner.
- DSCA33-45-HOFFMAN-RECOVERY handoff for the AD2 session (incl. the gate:
validate each render vs its Hoffman original before enabling live rendering).
- memories: Hoffman recovery (supersedes the spec-gap "need John" note) and the
AD2 SSH MTU-blackhole root cause/fix; errorlog entries (syncro jq, ssh correction).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remote SSH/VPN to AD2 keeps flapping; hand the remaining datasheet fixes to the
local AD2 session. Includes the per-subtype approach (DSCA_TEMPLATES from staged
originals — STAGE 1 done, dsca-templates.json on AD2 = 126 models), the render-wiring
+ per-subtype byte-validation gate, Fix 5 (379 backfill via legacy_cert_text), the
discipline (backup/save-state/validate-before-publish), and the derive-dsca-templates
tool. Ref ticket #32441.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
