Some DSCA subtypes' raw_data STATUS groups carry more (or fewer) value-bearing
entries than the template's spec-bearing rows (the test program measures slots the
printed sheet omits, e.g. DSCA49's 5mA load pair), so the in-order zip misaligned
values and those models were skipped by the count-guard.
New tool derive-dsca-slotmaps.js derives a per-model slotMap (absolute statusEntries
index per spec-bearing row) by greedily matching a staged original's printed values
to the DB raw_data STATUS entries (same fround formatting), then picking the
candidate map that validates against the most units. Models are grouped by identical
row-name signature and one map is derived per group from all sibling units — this
disambiguates duplicate values (e.g. a unit where 5mA != 50mA linearity forces the
correct slot; DSCA49-04 alone has only 2 staged units that can't, but its siblings'
25 units do). Stored as `slotMap` in dsca-templates.json.
Renderer: consults slotMap only when the sequential zip fails (value count !=
spec-row count), so the 88 already-clean models keep their path (no regression) and
ambiguous ones pull the right value via the map.
STAGE 3 re-validation: FINAL-TEST CLEAN 88 -> 92; 134 more certs now render
(null 450 -> 316); matches 2278 -> 2412. Same 6 retest-vintage dirty models, no
new mismatches. DSCA49 family + DSCA40-03 group now clean and validated.
Still blocked (separate gap, NOT layout ambiguity): DSCA45-* and most DSCA33-*
render null because they have NO spec-reader entries (render-datasheet bails before
rendering). Their slotMaps are derived and ready; they need spec coverage. One DSCA33
group (DSCA33-02/03/03A/04/05/1948) did not reach the slotMap validation threshold
(best 19/35 units) and stays skipped pending more/cleaner staged samples.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause of the DSCA39 footer mismatch: the "Standard output load for test is
250 ohms." line is a footer note, not a parameter, but the STAGE 1 extractor
captured it as a (column-truncated) row "Standard output load for te". And the
renderer's OUTSIGTYPE==='CURRENT' emission was wrong on both ends — it printed the
note (after the underline, invisible to the validator gate) for many -C current
models whose staged originals never had it, and never placed it correctly for the
models that do.
Fix is data-driven, matching the rest of the template approach:
- derive-dsca-templates.js: detect the "Standard output load..." line, capture it
as a per-model `loadNote` property, and exclude it from rows. Regenerated
dsca-templates.json — surgically clean: only the 5 DSCA39 models changed (lost
the truncated row, gained loadNote); all 121 others byte-identical.
- datasheet-exact.js: emit `dscaTpl.loadNote` (blank line + note) before the footer
underline, only for models that have it; removed the OUTSIGTYPE-based emission.
STAGE 3 re-validation: FINAL-TEST CLEAN 85 -> 88, mismatches 9 -> 6, matches
2206 -> 2278. DSCA39-01/02/07 now fully clean; DSCA39-01 byte-content-verified.
No regression — the -C current models stayed clean and no longer carry the
spurious after-underline note.
The 6 remaining dirty models (DSCA38-05/-1793/-19C/-19E, DSCA39-05, DSCA39-1950)
are ALL retest data-vintage: the staged .TXT is an older test run than the DB
latest-wins record (Supply Current / Linearity differ by more than rounding).
Not render bugs — cannot be reconciled against an older sheet.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
formatMeasuredExact now applies Math.fround to the parsed value before toFixed.
The DOS QuickBASIC computed/stored these as single-precision floats, so the
last-digit rounding at the .5 boundary follows single, not double, precision.
Without it, double-precision toFixed flipped boundaries (9.9995 -> "9.999" vs
golden "10.000"; 46.85 -> "46.9" vs "46.8"; .45 -> "0.5" vs "0.4"; 3.3325 ->
"3.333" vs "3.332"). Verified each against the staged golden.
STAGE 3 re-validation: FINAL-TEST CLEAN models 68 -> 85 (+17), mismatches
26 -> 9, cert matches 2123 -> 2206. Zero regression — every remaining dirty
model was already dirty pre-fix; no previously-clean model flipped.
The 9 remaining are NOT rounding:
- 4 models (DSCA38-05/-1793/-19C/-19E): Supply Current retest data-vintage — the
staged .TXT is an older test run than the DB latest-wins record; not a render
bug, can't reconcile against an older sheet.
- 5 models (DSCA39-01/02/05/07/1950): STAGE 1 template artifact — the footer note
"Standard output load for test is 250 ohms." was mis-captured as a truncated
parameter row. A renderer-side fix was attempted but emitted the note for all
current-output models (regressed 24 clean -C models), so it was reverted; needs
a targeted STAGE 1 extractor fix instead.
Renamed dsca-clean68-models.json -> dsca-clean-models.json (now 85 models).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Restarted testdatadb service (new template live), canary-pushed 1 cert (updated,
0 errors), then re-pushed all PASS certs for the 68 Final-Test-content-clean
models via uploadBySerialNumbers from a fresh node process.
Result over 30,423 PASS certs: updated=26022 unchanged=2738 created=0 errors=0
skipped=1663. The 26,022 updates replace the old defective DSCA renders on the
live site with the rebuilt, byte-content-validated ones. The 1,663 skips are the
count-guard correctly refusing any individual cert whose value count != its
template row count (ambiguous) — never publishing misaligned data.
Artifacts: push-clean68.js (driver), dsca-clean68-models.json (the published
set). NOT yet published: the 26 last-digit-diff models, the ~32 ambiguous/null
layout families, and the 231 untemplated models.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
validate-dsca-stage3.js: read-only harness that, for every staged DSCA original
we have ground truth for (2806 across 126 models), looks up the DB record,
renders it through the live path, and content-compares. GATE = the FINAL TEST
RESULTS section (rule lines canonicalized, whitespace collapsed — so the deferred
column-spacing cosmetic doesn't register); accuracy-section diffs reported
separately as informational.
First run verdict (report attached):
- 68 models FINAL-TEST CONTENT-CLEAN (0 mismatch over compared certs).
- 2123/2316 certs match the Final-Test content exactly (91.7%).
- 26 models show measured-value last-digit diffs only — structure (names, specs,
row alignment, statuses) is correct. Two root causes, neither structural:
* rounding-mode: JS double toFixed vs QB single-precision half-up
(e.g. raw 9.9995 code3 -> "9.999" here, "10.000" in golden). Fixable but
float-precision-sensitive; risks regressing currently-clean values.
* data vintage: staged .TXT is a different test run than the DB latest-wins
record (Fix 3) — e.g. Supply Current 19.6 vs 20.3, 0.7 apart. Not a render
bug; can't be reconciled against an older staged sheet.
- ~32 models render null (count-guard): DSCA33-*, DSCA45-*, DSCA49-* families
whose raw_data carries load points the template omits -> need per-subtype slot
mapping (the canonical-slot approach) before they can render.
Still NOT published: service not restarted, nothing re-pushed to Hoffman.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Deployed file is C:\Shares\testdatadb\templates\datasheet-exact.js; this
reconciles the repo copy + adds dsca-templates.json (STAGE 1 output).
What changed in generateExactDatasheet (DSCA family only; 5B/8B/7B/DSCT/SCMVAS
paths byte-unchanged):
- Load dsca-templates.json once at module top (126 per-model layouts).
- DSCA Final-Test now renders names + specs from the staged template rows, not
the single hardcoded DATA_LINES['DSCA'] + buildTSpecs DSCA branch.
- Value-bearing raw_data STATUS groups map positionally onto the spec-bearing
template rows; empty-spec rows (240VAC Withstand / Hi-Pot) render blank+PASS.
Removed the duplicate hardcoded 240VAC/Hi-Pot footer for DSCA (now rows).
- ACCURACY header uses the template accOut ("Output (V)"/"Output (mA)") with '-'
rule separators instead of "Vout (V)" + '='.
- Header/columns match the staged originals (Measured Value*, 25/15/19/6 rule).
Two real bugs fixed (both are the handoff's "lines drop" / wrong-value defect):
- formatMeasuredExact reads the value from index 4 so negative signs survive
("PASS-4.24..." -> "-4", not "4"); also decimal-code N -> toFixed(N) exactly
(DSCA differs from 5B/8B where code 2 means 1 decimal).
- parseRawData no longer consumes the first DSCA STATUS group as a bare
step-response line when that line is absent (dropped 3 rows on e.g. DSCA39-01).
Safety: when value count != spec-row count the positional zip is ambiguous
(subtype measures load points the template omits, e.g. DSCA49 5mA pair), so the
cert is SKIPPED (null) and left for STAGE 3 per-subtype mapping rather than
emitting misaligned data.
Validation: DSCA38-05 (SN 180224-1) Final-Test block byte-identical to its
staged original. 92/126 templated models render cleanly; 7 ambiguous + 27
no-spec skip. Remaining ACCURACY-block spacing diffs are the deferred cosmetic
gap. NOT YET LIVE: testdatadb service not restarted, nothing re-pushed to
Hoffman (STAGE 3 gate).
Coverage gap to resolve before publish: only 126/357 DSCA models in the DB have
a staged template (56,074 certs, 70.1%); 231 models / 23,866 certs have none and
now render null — needs a STAGE 1 extension (more staged originals).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Plain-language summary for John (EE, non-developer) covering all 4 problems
(RTD label, DSCA table, same-day run, missing units), framed as ACG-owned fixes.
Cross-links the four supporting technical docs. Draft - not yet sent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per Mike: import runs every 15 min, so routine timing isn't the cause. The 379
absent units are confined to 2025-10..2026-01 (stop after Jan 2026) on TS-4L/4R/1R
- fingerprint of a one-time overwrite during the incident/recovery (fresh DOS logs
overwrote accumulated appended server-side logs for ~2 weeks). One-time, not
recurring; backfill from the surviving staged .TXT.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
608 staged datasheets absent from DB. Two causes: (1) 229 units with encoded/
non-standard serials the importer's leading-digit regex silently skips - data is
in the .DAT, recoverable; full blind spot is 840 serials / 9,510 records / 141
models dropped fleet-wide. (2) 379 units whose per-model .DAT was overwritten by a
later work order - recoverable only from the staged .TXT or a log backup. Adds
John-facing report, raw data, and the chase-missing-units.js tool.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Whole-source sweep (981,716 records / 406,549 serials): 6,515 same-day multi-run
events; DB holds a NON-latest run for 311 (the strictly-greater-date conflict rule
freezes on an arbitrary same-day run). Corrects the verdict doc to flag same-day
retests as a latest-wins faithfulness violation (not benign). Adds the proposed
>= -with-data-differs conflict-rule fix (diagnose-only) and the sweep tool.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Validated all 11,922 staged original .TXT datasheets against test_records.
0 genuine parse faults across 11,239 comparable records; mismatches all explained
(retests, reused serials, VAS format, legacy out-of-scope units). Adds the
validate-parsing.js tool, raw report, and verdict. Two follow-ups (NOT parse bugs):
608 staged units absent from DB (ingestion completeness), and same-day retests keep
the first run (ON CONFLICT strictly-greater-date).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Repo copy only (review before deploying to C:\Shares\testdatadb). Folds RTD
(sensorNum 7) into the temperature path so the ACCURACY input column shows
'Temp. (C)' with signed temperature values, matching the original DOS-generated
datasheets and thermocouples (3-6). raw_data stimulus is already in deg C; no
conversion. getSensorNum and the i==13 ohm/ohm unit override are untouched.
Verified read-only against deployed env: 8B35 SN 179553-13 now shows Temp (C);
regression over 184 5B/8B renders -> 15 RTD changed (intended), 0 non-RTD changed.
Does NOT address Defect B (DSCA template). See DATASHEET-RTD-BUG-DIAGNOSIS-2026-06-17.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Relocated verbatim from .claude/CLAUDE.md so the ad2 fork stops editing the
shared fleet harness doc. After rebase onto main, .claude/CLAUDE.md = the lean
fleet version; Dataforth ops context lives here. Keeps future ad2 syncs clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause of the ~5,148 unpublished 8B/5B/SCM PASS records (driving the fix):
(1) parseRawData wrongly consumes a PASS/FAIL line as the step-response line for
non-DSCA families that omit the "0","0",v line (8B45/8B49/5B39/SCM5B33...) ->
drops the first Final-Test group -> measurement-count mismatch -> null render.
(2) Even parsed, the renderer has ONE hardcoded DATA_LINES['8B'] (RTD-shaped), so
models like 8B45 (frequency input, == DSCA45 structurally) get wrong param
names/specs. Same class as DSCA -> needs per-model templates.
Mined per-model templates from the Hoffman originals (published siblings) for all
136 mineable models via tools/mine-hoffman-dsca.py (family-agnostic extractor):
8b5bscm-templates.json = {accOut, accHeader, rows[name,spec], _srcSerial}.
Input-type split: 72 voltage / 18 temp / 12 current (accuracy already handled) +
10 frequency (8B45/5B45 — same unsolved freq-accuracy as DSCA45) + 24 7B/other.
Only 3 niche models (17 units) have no Hoffman original.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Backend (deployed live on AD2, service restarted, + repo copy resynced — it was
far behind the deployed server):
- /api/search: add whitelisted sort/dir (NULLS LAST) so sortable headers and the
"Latest uploads" preset work. web_status filter and POST /api/upload already
existed on the server; the stale repo copy now matches live.
Frontend (redesign prototype):
- "Latest uploads" preset (web_status=on + sort=api_uploaded_at desc) and
"Not yet published" (web_status=off) are now active presets.
- Push to Web (inspector) + Re-push (multi-select) wired to POST /api/upload
behind a confirm() gate; refresh WEB status after. Validated idempotently on a
published record (unchanged:1, errors:0).
- "Retested units" stays disabled — needs a retest flag in the pipeline (next).
tools/preview-proxy.py: forward POST so the publish buttons work in same-origin preview.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- fitCert: replace the flaky CSS `zoom` (Firefox support is recent/inconsistent)
with transform:scale() measured against the widest line (+ right margin and
font-load retries) so the cert always scales to fit the inspector with no
horizontal clip. Validated live on a narrow 5B cert (0.74x) and a wide DSCA45
cert (0.55x) against the real AD2 dataset.
- inspector Web field -> Published (green) / Not published (amber) chips.
- widen default inspector 480 -> 500px.
- tools/preview-proxy.py: serve the prototype AND reverse-proxy /api to the live
AD2 server so the cert iframe is same-origin during preview — styleCert/fitCert
read iframe.contentDocument, which silently no-ops when the iframe is loaded
cross-origin straight from AD2 (why the fit looked broken in earlier previews).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The DSCA33/DSCA45 main spec files lost in the cryptolocker wipe are recoverable:
the original software published correct certs to the Hoffman product API before
the wipe and our null-skipping renderer never overwrote them. Mine per-model
Final-Test templates (names + specs + verbatim accuracy headers) straight from
those originals instead of requesting spec files from Dataforth/John.
- dsca33-45-templates.json: 56 models (DSCA33 34/35, DSCA45 22/23); only
DSCA33-1948 + DSCA45-1746 (24 units) lack an original.
- mine-hoffman-dsca.py: the re-runnable miner.
- DSCA33-45-HOFFMAN-RECOVERY handoff for the AD2 session (incl. the gate:
validate each render vs its Hoffman original before enabling live rendering).
- memories: Hoffman recovery (supersedes the spec-gap "need John" note) and the
AD2 SSH MTU-blackhole root cause/fix; errorlog entries (syncro jq, ssh correction).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Vanilla single-file (no build/CDN) command-center redesign of the testdatadb search
UI: omni-search with serial/model/text routing + auto-select fast path, dense
monospace results table with PASS/FAIL pills + web-published indicator, persistent
split-pane datasheet inspector (iframe to /api/datasheet), left filter rail
(result/date/model/station/log), server pagination, CSV export, URL state, keyboard
nav (/ ↑↓ ↵ Esc), clinical light theme. Hits the existing API; deployed to AD2 as
public/index.redesign.html (preview at :3000/index.redesign.html). Synthesized from
Grok + Gemini concepts (both converged on this command-center design).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remote SSH/VPN to AD2 keeps flapping; hand the remaining datasheet fixes to the
local AD2 session. Includes the per-subtype approach (DSCA_TEMPLATES from staged
originals — STAGE 1 done, dsca-templates.json on AD2 = 126 models), the render-wiring
+ per-subtype byte-validation gate, Fix 5 (379 backfill via legacy_cert_text), the
discipline (backup/save-state/validate-before-publish), and the derive-dsca-templates
tool. Ref ticket #32441.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
