---
name: Test datasheet pipeline rebuilt 2026-03-27
description: New server-side pipeline replaces CTONWTXT/DFWDS/TestDataSheetUploader. Generates exact-match TXT from DAT imports. 1470 model specs loaded.
type: project
---

## What Was Built (2026-03-27)

All files in C:\Shares\testdatadb\:

### New Files
- `parsers/spec-reader.js` — Reads 8 binary spec DAT files, 1470 models across 6 families (SCM5B, 8B, DSCA, DSCT, SCM7B, plus sub-families 5B45, 5B48)
- `templates/datasheet-exact.js` — Exact-match TXT formatter reverse-engineered from QuickBASIC DATASHEETWRITE
- `database/export-datasheets.js` — Standalone/library export script, writes {SN}.TXT to X:\For_Web
- `database/archive-for-web.js` — Archives old For_Web files into year subfolders
- `database/generate-customer-pdfs.js` — One-off script for Quatronix customer order (72/73 generated)
- `specdata/` — Binary spec DAT files (5BMAIN, 8BMAIN, DSCOUT, SCTMAIN, DSCMAIN4, 7BMAIN, 5B45DATA, DB5B48)

### Modified Files
- `database/import.js` — Post-import hook calls export-datasheets after incremental imports
- `database/schema.sql` — Added datasheet_exported_at and forweb_exported_at columns
- `routes/api.js` — Added /api/datasheet/:id/pdf endpoint (pdfkit)
- `public/index.html` — Added ALL filter, auto-search on load, PDF download button, export status in detail view
- `package.json` — Added pdfkit dependency

### Database Changes
- `datasheet_exported_at` — When TXT was generated
- `forweb_exported_at` — When file was confirmed in X:\For_Web
- 1,436,007 records pre-marked as exported (matched existing For_Web files)

### Pipeline Flow
DAT file arrives via NAS sync → import.js parses to DB → export-datasheets generates {SN}.TXT → X:\For_Web

### What's Eliminated
- CTONWTXT.BAT (DOS-side TXT transfer) — no longer needed
- DFWDS.exe (filename decoder/validator) — no longer needed
- TestDataSheetUploader (VB.NET web uploader) — endpoints are 404, needs replacement

### Remaining Gaps
- Website upload: old ASP.NET endpoints dead (404), need new upload mechanism
- SCM5B49 spec file empty (0 bytes) — need John's help
- 7B datasheet formatting not yet added to datasheet-exact.js (7B has 30+ unique parameters)
- Service runs as SYSTEM causing permission issues with SHM/WAL files
- 500K files archived into year subfolders in X:\For_Web

**Why:** Rebuilt to recover from crypto wipe and unblock customer shipments (Quatronix/China blocking on missing datasheets).
**How to apply:** For new test data, the pipeline is automatic via Sync-FromNAS. For the 830K backlog of unexported records, run `node database/export-datasheets.js --limit N`.