Probed each of the 1,578 unuploaded PASS serials across the 54 validated DSCA33/45
models against the Hoffman API (stale inventory -> can't trust api_uploaded_at as
"absent"). 1,452 were absent (404), 126 already live. Pushed ONLY the absent set:
created=1452 updated=0 unchanged=0 errors=0 — zero overwrites of pristine originals,
the handoff's hard requirement.
Tools: publish-dsca3345-gap.js (absent-only Created publish), validate-dsca3345.js.
Net: DSCA33/45 effort complete — 54/56 models live + validated; 2 rounding-boundary
holdouts and 2 no-original models (24 units) remain null.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cracked the DSCA33/45 accuracy-block numeric formatting against the Hoffman originals
(formatAccuracyLineDSCA3345):
- mA-output models store calc (and, for DSCA45, meas) in AMPS -> x1000 to display mA;
DSCA33 stores meas already in display unit (NOT scaled), DSCA45 scales both.
- DSCA33 (AC-RMS): stim/calc/meas UNSIGNED, error signed; stim is AC input to 3 dp.
- DSCA45 (frequency): stim is an UNSIGNED integer Hz; calc/meas/error SIGNED.
- Math.fround on accuracy values (QB single-precision rounding), matching the Final-Test fix.
Final-Test fixes too: leading-zero drop only when the value overflows QB's 6-char field
("-0.0005"->"-.0005", but "-0.750" keeps its zero); spec-less section sub-heads
(Zero-Crossing Input / TTL Input) render with NO status (only Withstand/Hi-Pot get PASS);
DSCA33 prints a "Check List" header after the underline.
slotmap-from-hoffman.js (new): derive slotMaps for the models the staged multi-unit
derivation couldn't (vintage-heavy) by matching the Hoffman _srcSerial original's
Final-Test measured values (at display precision) to the DB STATUS entries. Recovered
all 13 remaining DSCA33 models.
Validation (validate-dsca3345.js, content-normalized byte-compare vs live Hoffman
originals): 54 of 56 models PASS and are marked validated:true (the render gate).
2 holdouts (DSCA33-04A, DSCA33-1891) each have ONE accuracy cert at a rounding boundary
where fround rounds opposite to the original; left UNvalidated -> still render null
(safe). DSCA33-1948 + DSCA45-1746 (24 units) have no Hoffman original.
Gate now OPEN for the 54 validated models (render live); 2 holdouts + the no-template
pair stay null. Publishing the api_uploaded_at IS NULL gap next (never re-pushes the
~7,157 pristine originals).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per the 5070 handoff (DSCA33-45-HOFFMAN-RECOVERY): the lost DSCA33/45 specs are
recoverable from Hoffman, not John. Wired the mined dsca33-45-templates.json (56
models) into the renderer:
- datasheet-exact.js: load DSCA3345_TEMPLATES; for family DSCA, the Hoffman-mined
template takes PRECEDENCE over the stale staged-extraction entry (which shadowed 25
models with accOut "?"/no accHeader). Emit the verbatim 2-line accHeader for these
families (Vin (mVAC)/Iin (AAC)/Frequency (Hz), Output (VDC)/(mADC)). Per-model
`validated` GATE: a DSCA33/45 model renders only after byte-matching its Hoffman
original; until then it returns null (skipped) so an unverified render can never
overwrite a pristine live original. DSCA_VALIDATE_MODE env opens the gate for the
validation harness only. Exposed rendersWithoutSpecs().
- render-datasheet.js: allow a null-specs render for DSCA33/45 (their spec files were
lost; template-driven) instead of bailing on missing specs.
- derive-dsca-slotmaps.js: DSCA_TPL env to target the 3345 templates; derived 43 slot
maps into them (22 models need none, 8 DSCA33 still below threshold).
- validate-dsca3345.js (new): renders each model's _srcSerial, fetches the live
Hoffman original (GET TestReportDataFiles/{serial}, deployed uploader token — no
vault needed), content-normalized compare; --apply marks validated.
STATUS: gate is CLOSED — 0 models validated, all DSCA33/45 still render null, nothing
published, no risk. Final-Test block + accuracy headers now byte-match the Hoffman
originals for all 56 models; the remaining blocker is accuracy-DATA numeric quirks that
must match to pass the gate:
- DSCA33 calc column stored in A but displayed in mADC (x1000); measured stored in
mA (not scaled) — an original-software unit quirk.
- sign conventions differ per layout (DSCA33 stim/calc/meas unsigned, error signed;
DSCA45 stim unsigned, calc/meas/error signed).
- DSCA45 frequency-input stim formatting.
These need per-layout reverse-engineering against the originals (the validation harness
is the oracle). 8 DSCA33 models (DSCA33-02/03/03A/04/04A/05/05A/1642) also lack a slot
map (below threshold). DSCA33-1948 + DSCA45-1746 (24 units) have no Hoffman original.
Cleanups: deleted superseded memory project_dsca33_45_spec_gap; struck the obsolete
"ask John" TODO 2 from the handoff note.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Some DSCA subtypes' raw_data STATUS groups carry more (or fewer) value-bearing
entries than the template's spec-bearing rows (the test program measures slots the
printed sheet omits, e.g. DSCA49's 5mA load pair), so the in-order zip misaligned
values and those models were skipped by the count-guard.
New tool derive-dsca-slotmaps.js derives a per-model slotMap (absolute statusEntries
index per spec-bearing row) by greedily matching a staged original's printed values
to the DB raw_data STATUS entries (same fround formatting), then picking the
candidate map that validates against the most units. Models are grouped by identical
row-name signature and one map is derived per group from all sibling units — this
disambiguates duplicate values (e.g. a unit where 5mA != 50mA linearity forces the
correct slot; DSCA49-04 alone has only 2 staged units that can't, but its siblings'
25 units do). Stored as `slotMap` in dsca-templates.json.
Renderer: consults slotMap only when the sequential zip fails (value count !=
spec-row count), so the 88 already-clean models keep their path (no regression) and
ambiguous ones pull the right value via the map.
STAGE 3 re-validation: FINAL-TEST CLEAN 88 -> 92; 134 more certs now render
(null 450 -> 316); matches 2278 -> 2412. Same 6 retest-vintage dirty models, no
new mismatches. DSCA49 family + DSCA40-03 group now clean and validated.
Still blocked (separate gap, NOT layout ambiguity): DSCA45-* and most DSCA33-*
render null because they have NO spec-reader entries (render-datasheet bails before
rendering). Their slotMaps are derived and ready; they need spec coverage. One DSCA33
group (DSCA33-02/03/03A/04/05/1948) did not reach the slotMap validation threshold
(best 19/35 units) and stays skipped pending more/cleaner staged samples.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause of the DSCA39 footer mismatch: the "Standard output load for test is
250 ohms." line is a footer note, not a parameter, but the STAGE 1 extractor
captured it as a (column-truncated) row "Standard output load for te". And the
renderer's OUTSIGTYPE==='CURRENT' emission was wrong on both ends — it printed the
note (after the underline, invisible to the validator gate) for many -C current
models whose staged originals never had it, and never placed it correctly for the
models that do.
Fix is data-driven, matching the rest of the template approach:
- derive-dsca-templates.js: detect the "Standard output load..." line, capture it
as a per-model `loadNote` property, and exclude it from rows. Regenerated
dsca-templates.json — surgically clean: only the 5 DSCA39 models changed (lost
the truncated row, gained loadNote); all 121 others byte-identical.
- datasheet-exact.js: emit `dscaTpl.loadNote` (blank line + note) before the footer
underline, only for models that have it; removed the OUTSIGTYPE-based emission.
STAGE 3 re-validation: FINAL-TEST CLEAN 85 -> 88, mismatches 9 -> 6, matches
2206 -> 2278. DSCA39-01/02/07 now fully clean; DSCA39-01 byte-content-verified.
No regression — the -C current models stayed clean and no longer carry the
spurious after-underline note.
The 6 remaining dirty models (DSCA38-05/-1793/-19C/-19E, DSCA39-05, DSCA39-1950)
are ALL retest data-vintage: the staged .TXT is an older test run than the DB
latest-wins record (Supply Current / Linearity differ by more than rounding).
Not render bugs — cannot be reconciled against an older sheet.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
formatMeasuredExact now applies Math.fround to the parsed value before toFixed.
The DOS QuickBASIC computed/stored these as single-precision floats, so the
last-digit rounding at the .5 boundary follows single, not double, precision.
Without it, double-precision toFixed flipped boundaries (9.9995 -> "9.999" vs
golden "10.000"; 46.85 -> "46.9" vs "46.8"; .45 -> "0.5" vs "0.4"; 3.3325 ->
"3.333" vs "3.332"). Verified each against the staged golden.
STAGE 3 re-validation: FINAL-TEST CLEAN models 68 -> 85 (+17), mismatches
26 -> 9, cert matches 2123 -> 2206. Zero regression — every remaining dirty
model was already dirty pre-fix; no previously-clean model flipped.
The 9 remaining are NOT rounding:
- 4 models (DSCA38-05/-1793/-19C/-19E): Supply Current retest data-vintage — the
staged .TXT is an older test run than the DB latest-wins record; not a render
bug, can't reconcile against an older sheet.
- 5 models (DSCA39-01/02/05/07/1950): STAGE 1 template artifact — the footer note
"Standard output load for test is 250 ohms." was mis-captured as a truncated
parameter row. A renderer-side fix was attempted but emitted the note for all
current-output models (regressed 24 clean -C models), so it was reverted; needs
a targeted STAGE 1 extractor fix instead.
Renamed dsca-clean68-models.json -> dsca-clean-models.json (now 85 models).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Restarted testdatadb service (new template live), canary-pushed 1 cert (updated,
0 errors), then re-pushed all PASS certs for the 68 Final-Test-content-clean
models via uploadBySerialNumbers from a fresh node process.
Result over 30,423 PASS certs: updated=26022 unchanged=2738 created=0 errors=0
skipped=1663. The 26,022 updates replace the old defective DSCA renders on the
live site with the rebuilt, byte-content-validated ones. The 1,663 skips are the
count-guard correctly refusing any individual cert whose value count != its
template row count (ambiguous) — never publishing misaligned data.
Artifacts: push-clean68.js (driver), dsca-clean68-models.json (the published
set). NOT yet published: the 26 last-digit-diff models, the ~32 ambiguous/null
layout families, and the 231 untemplated models.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
validate-dsca-stage3.js: read-only harness that, for every staged DSCA original
we have ground truth for (2806 across 126 models), looks up the DB record,
renders it through the live path, and content-compares. GATE = the FINAL TEST
RESULTS section (rule lines canonicalized, whitespace collapsed — so the deferred
column-spacing cosmetic doesn't register); accuracy-section diffs reported
separately as informational.
First run verdict (report attached):
- 68 models FINAL-TEST CONTENT-CLEAN (0 mismatch over compared certs).
- 2123/2316 certs match the Final-Test content exactly (91.7%).
- 26 models show measured-value last-digit diffs only — structure (names, specs,
row alignment, statuses) is correct. Two root causes, neither structural:
* rounding-mode: JS double toFixed vs QB single-precision half-up
(e.g. raw 9.9995 code3 -> "9.999" here, "10.000" in golden). Fixable but
float-precision-sensitive; risks regressing currently-clean values.
* data vintage: staged .TXT is a different test run than the DB latest-wins
record (Fix 3) — e.g. Supply Current 19.6 vs 20.3, 0.7 apart. Not a render
bug; can't be reconciled against an older staged sheet.
- ~32 models render null (count-guard): DSCA33-*, DSCA45-*, DSCA49-* families
whose raw_data carries load points the template omits -> need per-subtype slot
mapping (the canonical-slot approach) before they can render.
Still NOT published: service not restarted, nothing re-pushed to Hoffman.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Deployed file is C:\Shares\testdatadb\templates\datasheet-exact.js; this
reconciles the repo copy + adds dsca-templates.json (STAGE 1 output).
What changed in generateExactDatasheet (DSCA family only; 5B/8B/7B/DSCT/SCMVAS
paths byte-unchanged):
- Load dsca-templates.json once at module top (126 per-model layouts).
- DSCA Final-Test now renders names + specs from the staged template rows, not
the single hardcoded DATA_LINES['DSCA'] + buildTSpecs DSCA branch.
- Value-bearing raw_data STATUS groups map positionally onto the spec-bearing
template rows; empty-spec rows (240VAC Withstand / Hi-Pot) render blank+PASS.
Removed the duplicate hardcoded 240VAC/Hi-Pot footer for DSCA (now rows).
- ACCURACY header uses the template accOut ("Output (V)"/"Output (mA)") with '-'
rule separators instead of "Vout (V)" + '='.
- Header/columns match the staged originals (Measured Value*, 25/15/19/6 rule).
Two real bugs fixed (both are the handoff's "lines drop" / wrong-value defect):
- formatMeasuredExact reads the value from index 4 so negative signs survive
("PASS-4.24..." -> "-4", not "4"); also decimal-code N -> toFixed(N) exactly
(DSCA differs from 5B/8B where code 2 means 1 decimal).
- parseRawData no longer consumes the first DSCA STATUS group as a bare
step-response line when that line is absent (dropped 3 rows on e.g. DSCA39-01).
Safety: when value count != spec-row count the positional zip is ambiguous
(subtype measures load points the template omits, e.g. DSCA49 5mA pair), so the
cert is SKIPPED (null) and left for STAGE 3 per-subtype mapping rather than
emitting misaligned data.
Validation: DSCA38-05 (SN 180224-1) Final-Test block byte-identical to its
staged original. 92/126 templated models render cleanly; 7 ambiguous + 27
no-spec skip. Remaining ACCURACY-block spacing diffs are the deferred cosmetic
gap. NOT YET LIVE: testdatadb service not restarted, nothing re-pushed to
Hoffman (STAGE 3 gate).
Coverage gap to resolve before publish: only 126/357 DSCA models in the DB have
a staged template (56,074 certs, 70.1%); 231 models / 23,866 certs have none and
now render null — needs a STAGE 1 extension (more staged originals).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Plain-language summary for John (EE, non-developer) covering all 4 problems
(RTD label, DSCA table, same-day run, missing units), framed as ACG-owned fixes.
Cross-links the four supporting technical docs. Draft - not yet sent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per Mike: import runs every 15 min, so routine timing isn't the cause. The 379
absent units are confined to 2025-10..2026-01 (stop after Jan 2026) on TS-4L/4R/1R
- fingerprint of a one-time overwrite during the incident/recovery (fresh DOS logs
overwrote accumulated appended server-side logs for ~2 weeks). One-time, not
recurring; backfill from the surviving staged .TXT.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
608 staged datasheets absent from DB. Two causes: (1) 229 units with encoded/
non-standard serials the importer's leading-digit regex silently skips - data is
in the .DAT, recoverable; full blind spot is 840 serials / 9,510 records / 141
models dropped fleet-wide. (2) 379 units whose per-model .DAT was overwritten by a
later work order - recoverable only from the staged .TXT or a log backup. Adds
John-facing report, raw data, and the chase-missing-units.js tool.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Whole-source sweep (981,716 records / 406,549 serials): 6,515 same-day multi-run
events; DB holds a NON-latest run for 311 (the strictly-greater-date conflict rule
freezes on an arbitrary same-day run). Corrects the verdict doc to flag same-day
retests as a latest-wins faithfulness violation (not benign). Adds the proposed
>= -with-data-differs conflict-rule fix (diagnose-only) and the sweep tool.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Validated all 11,922 staged original .TXT datasheets against test_records.
0 genuine parse faults across 11,239 comparable records; mismatches all explained
(retests, reused serials, VAS format, legacy out-of-scope units). Adds the
validate-parsing.js tool, raw report, and verdict. Two follow-ups (NOT parse bugs):
608 staged units absent from DB (ingestion completeness), and same-day retests keep
the first run (ON CONFLICT strictly-greater-date).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Repo copy only (review before deploying to C:\Shares\testdatadb). Folds RTD
(sensorNum 7) into the temperature path so the ACCURACY input column shows
'Temp. (C)' with signed temperature values, matching the original DOS-generated
datasheets and thermocouples (3-6). raw_data stimulus is already in deg C; no
conversion. getSensorNum and the i==13 ohm/ohm unit override are untouched.
Verified read-only against deployed env: 8B35 SN 179553-13 now shows Temp (C);
regression over 184 5B/8B renders -> 15 RTD changed (intended), 0 non-RTD changed.
Does NOT address Defect B (DSCA template). See DATASHEET-RTD-BUG-DIAGNOSIS-2026-06-17.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause of the ~5,148 unpublished 8B/5B/SCM PASS records (driving the fix):
(1) parseRawData wrongly consumes a PASS/FAIL line as the step-response line for
non-DSCA families that omit the "0","0",v line (8B45/8B49/5B39/SCM5B33...) ->
drops the first Final-Test group -> measurement-count mismatch -> null render.
(2) Even parsed, the renderer has ONE hardcoded DATA_LINES['8B'] (RTD-shaped), so
models like 8B45 (frequency input, == DSCA45 structurally) get wrong param
names/specs. Same class as DSCA -> needs per-model templates.
Mined per-model templates from the Hoffman originals (published siblings) for all
136 mineable models via tools/mine-hoffman-dsca.py (family-agnostic extractor):
8b5bscm-templates.json = {accOut, accHeader, rows[name,spec], _srcSerial}.
Input-type split: 72 voltage / 18 temp / 12 current (accuracy already handled) +
10 frequency (8B45/5B45 — same unsolved freq-accuracy as DSCA45) + 24 7B/other.
Only 3 niche models (17 units) have no Hoffman original.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Backend (deployed live on AD2, service restarted, + repo copy resynced — it was
far behind the deployed server):
- /api/search: add whitelisted sort/dir (NULLS LAST) so sortable headers and the
"Latest uploads" preset work. web_status filter and POST /api/upload already
existed on the server; the stale repo copy now matches live.
Frontend (redesign prototype):
- "Latest uploads" preset (web_status=on + sort=api_uploaded_at desc) and
"Not yet published" (web_status=off) are now active presets.
- Push to Web (inspector) + Re-push (multi-select) wired to POST /api/upload
behind a confirm() gate; refresh WEB status after. Validated idempotently on a
published record (unchanged:1, errors:0).
- "Retested units" stays disabled — needs a retest flag in the pipeline (next).
tools/preview-proxy.py: forward POST so the publish buttons work in same-origin preview.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- fitCert: replace the flaky CSS `zoom` (Firefox support is recent/inconsistent)
with transform:scale() measured against the widest line (+ right margin and
font-load retries) so the cert always scales to fit the inspector with no
horizontal clip. Validated live on a narrow 5B cert (0.74x) and a wide DSCA45
cert (0.55x) against the real AD2 dataset.
- inspector Web field -> Published (green) / Not published (amber) chips.
- widen default inspector 480 -> 500px.
- tools/preview-proxy.py: serve the prototype AND reverse-proxy /api to the live
AD2 server so the cert iframe is same-origin during preview — styleCert/fitCert
read iframe.contentDocument, which silently no-ops when the iframe is loaded
cross-origin straight from AD2 (why the fit looked broken in earlier previews).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The DSCA33/DSCA45 main spec files lost in the cryptolocker wipe are recoverable:
the original software published correct certs to the Hoffman product API before
the wipe and our null-skipping renderer never overwrote them. Mine per-model
Final-Test templates (names + specs + verbatim accuracy headers) straight from
those originals instead of requesting spec files from Dataforth/John.
- dsca33-45-templates.json: 56 models (DSCA33 34/35, DSCA45 22/23); only
DSCA33-1948 + DSCA45-1746 (24 units) lack an original.
- mine-hoffman-dsca.py: the re-runnable miner.
- DSCA33-45-HOFFMAN-RECOVERY handoff for the AD2 session (incl. the gate:
validate each render vs its Hoffman original before enabling live rendering).
- memories: Hoffman recovery (supersedes the spec-gap "need John" note) and the
AD2 SSH MTU-blackhole root cause/fix; errorlog entries (syncro jq, ssh correction).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Vanilla single-file (no build/CDN) command-center redesign of the testdatadb search
UI: omni-search with serial/model/text routing + auto-select fast path, dense
monospace results table with PASS/FAIL pills + web-published indicator, persistent
split-pane datasheet inspector (iframe to /api/datasheet), left filter rail
(result/date/model/station/log), server pagination, CSV export, URL state, keyboard
nav (/ ↑↓ ↵ Esc), clinical light theme. Hits the existing API; deployed to AD2 as
public/index.redesign.html (preview at :3000/index.redesign.html). Synthesized from
Grok + Gemini concepts (both converged on this command-center design).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remote SSH/VPN to AD2 keeps flapping; hand the remaining datasheet fixes to the
local AD2 session. Includes the per-subtype approach (DSCA_TEMPLATES from staged
originals — STAGE 1 done, dsca-templates.json on AD2 = 126 models), the render-wiring
+ per-subtype byte-validation gate, Fix 5 (379 backfill via legacy_cert_text), the
discipline (backup/save-state/validate-before-publish), and the derive-dsca-templates
tool. Ref ticket #32441.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- UNREGISTERED_MODELS set: 9 model numbers not in Hoffman API catalog; skipped
silently instead of generating errors
- batch-500 fallback: when a bulk batch returns HTTP 500, retry each record
individually so good records get stamped and only truly-bad records count
as errors
- FAIL-parameter filter: records with any FAIL on a parameter line are excluded
from the push before the batch is assembled
- notify.js integration: wired in existing notification module
Files added:
- projects/dataforth-dos/database/upload-to-api.js
- projects/dataforth-dos/database/notify.js
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Establishes inter-session coordination for 29 projects/clients:
- Full lock/component format for active projects (dataforth-dos,
radio-show, cascades-tucson, valleywide, instrumental-music-center,
lens-auto-brokerage, msp-audit-scripts)
- Light format for complete/stalled/planning (msp-pricing, pavon,
wrightstown-*, gururmm-agent, community-forum, glaztech, etc.)
- Onboarding stubs for recently added clients
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- TEST-DATASHEET-PROCESS.md: comprehensive pipeline documentation for
Dataforth engineering (10 sections, data flow, state diagram, FAQ)
- signing-attestation/: domain ownership attestation letter with
in-place signature for Azure Trusted Signing identity validation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Installed C:\ProgramData\dataforth-uploader\ on AD2 with:
- credentials.json (SYSTEM+Administrators ACL only)
- run-pipeline.ps1 (DFWDS-process -> enumerate For_Web -> upload-delta)
- dfwds-process.js + upload-delta.js (copied from prior install dir)
- logs/ with 60-day retention
Scheduled Task 'DataforthTestDatasheetUploader' registered as SYSTEM,
hourly trigger, 30-min execution limit. First SYSTEM-context run verified:
received=7061 unchanged=7061 errors=0 in 8.7s.
Initial registration via inline base64 mangled the backslashes in the -File
argument (resulted in ERROR_DIRECTORY 0x8007010B). Fixed by running the
registration PowerShell from a file rather than an encoded command string.
Also deleted throwaway tmp/list_amtransit.py + tmp/reset_cansley.py which
had hardcoded ACG admin password.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Built the missing piece between the test datasheet pipeline and Dataforth's
new product API. End-to-end:
- Pulled DFWDS (Dataforth Web Datasheet System) VB6 source from
AD1\Engineering\ENGR\ATE\Test Datasheets\DFWDS to local for analysis
- Decoded its filename validation: A-J prefix decodes (A=10..J=19), all-
numeric WO# valid (no leading 0), anything else bad
- Ported the validation + move logic to Node (dfwds-process.js)
- Built bulk uploader (upload-delta.js) for Hoffman's Swagger API
(POST /api/v1/TestReportDataFiles/bulk with OAuth client_credentials)
Sanitized 3 prior reference scripts (fetch-server-inventory, test-scenarios,
test-upload-two) to read CF_* env vars instead of hardcoded creds.
Live drain results:
- 897 files moved Test_Datasheets -> For_Web (all valid, no renames, no
bad), DFWDS port summary in 1.1s
- Pushed entire For_Web (7,061 files) to Hoffman API in 49.7s @ 142/s:
Created=803 Updated=114 Unchanged=6,144 Errors=0
- Server count: 489,579 -> 490,382 (+803 net new)
Also:
- Added clients/dataforth/.gitignore to exclude plaintext Oauth.txt note
- Added clients/instrumental-music-center/docs/2026-04-13-ticket-notes.md
(ticket write-up of 2026-04-11/12/13 IMC1 RDS removal/SQL migration work)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comprehensive record of 2026-04-11/12 work extending the Dataforth Test
Datasheet Pipeline: discovery, implementation, deploy to AD2, full
backfill of 27,937 datasheets, post-deploy regex patch for QB plain-
decimal PASS lines, and repo commit 0dd3d82.
Includes credentials, infrastructure paths, commit reference, open
items (vault hygiene, rsync coverage), and accuracy-extraction
reference logic for future sessions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extends the Test Datasheet Pipeline on AD2:C:\Shares\testdatadb to
generate web-published datasheets for the SCMVAS-Mxxx (obsolete) and
SCMHVAS-Mxxxx (replacement) High Voltage Input Module product lines.
Both are tested either with the existing TESTHV3 software (production
VASLOG .DAT logs) or in Engineering with plain .txt output.
Key changes on AD2 (all deployed 2026-04-12 with dated backups):
- parsers/spec-reader.js: getSpecs() returns a `{_family:'SCMVAS',
_noSpecs:true}` sentinel for SCMVAS/SCMHVAS/VAS-M/HVAS-M model prefixes
so the export pipeline does not silently skip them for missing specs.
- templates/datasheet-exact.js: new Accuracy-only template branch
(generateSCMVASDatasheet + helpers) that mirrors the existing shipped
format byte-for-byte. Extraction regex covers both QuickBASIC STR$()
output formats: scientific-with-trailing-status-digit (98.4% of
records) and plain-decimal (1.6% of records above QB's threshold).
- parsers/vaslog-engtxt.js (new): parses the Engineering-Tested .txt
files in TS-3R\LOGS\VASLOG\VASLOG - Engineering Tested\. Filename SN
regex strips optional trailing 14-digit timestamp; in-file "SN:"
header is the authoritative source when the filename is malformed.
- database/import.js: LOG_TYPES grows a VASLOG_ENG entry with
subfolder + recursive flags. Pre-existing 7 log types keep their
implicit recursive=true behaviour (config.recursive !== false).
importFiles() routes VASLOG_ENG paths before the generic loop so a
VASLOG - Engineering Tested/*.txt path does not mis-dispatch to the
multiline parser.
- database/export-datasheets.js: VASLOG_ENG records are written
verbatim via fs.copyFileSync(source_file, For_Web/<SN>.TXT) for true
byte-level pass-through, with a graceful raw_data fallback when the
source file is no longer on disk.
Deploy outcome:
- 27,503 SCMVAS/SCMHVAS datasheets rendered (27,065 from scientific +
438 from plain-decimal PASS lines, post-patch rerun)
- 434 Engineering-Tested .txt files pass-through-copied to For_Web
- 0 errors across both batches
Repo layout added here:
- scmvas-hvas-research/: discovery artifacts (source .BAS, hvin.dat,
sample .DAT + .txt, binary-format notes, IMPLEMENTATION_PLAN.md)
- implementation/: staged final code + deploy helpers + local test
harness + per-step verification scripts
- backups/pre-deploy-20260412/: independent local snapshot of the 4
AD2 files replaced, pulled byte-for-byte before deploy
All helper scripts fetch the AD2 password at runtime from the SOPS
vault (clients/dataforth/ad2.sops.yaml). None of the committed files
contain the plaintext credential. Known vault-entry hygiene issue
(stale shell-escape backslash before the `!`) is documented in the
fetcher comments and stripped at read-time; flagged separately for
cleanup.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
DOS 6.22 MD fails on existing dirs, then COPY creates a file instead of
copying into a directory. Dirs now pre-created on NAS server side.
Also removed trailing backslash from COPY target path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mixed-case paths caused "invalid directory" on TS-27. All paths now uppercase
to match DOS convention and Samba default case = upper setting.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
