---
name: Dataforth Test Datasheet Pipeline - Rebuilt 2026-03-27
description: Full pipeline from DOS test stations to website. New server-side generation replaces DFWDS/Uploader. 72/73 Quatronix datasheets generated. AD2 crypto wipe recovery.
type: project
---

## Background
AD2 (192.168.0.6) was wiped in a crypto/ransomware attack months ago. The test datasheet pipeline was broken. Customer Quatronix (China) blocking shipment of 328 modules (whittled to 54) without datasheets.

## Pipeline (5 stages, rebuilt 2026-03-27)

### Stage 1: DOS Test Stations (64 stations)
- QuickBASIC programs generate test data -> C:\STAGE on each DOS PC
- DAT files (raw test data) + TXT files (formatted datasheets)
- CTONW.BAT copies DAT files to NAS (working)
- CTONWTXT.BAT copies TXT files (NOT called in current AUTOEXEC v4.1 since 2026-03-12)
- TXT files piling up in C:\STAGE since Sept 2025

### Stage 2: NAS <-> AD2 Sync
- Script: C:\Shares\test\scripts\Sync-FromNAS-rsync.ps1 (every 15 min, WORKING)
- Rsync daemon on NAS: port 873, module "test", user rsync / IQ203s32119
- PULL: DAT files from NAS -> AD2, triggers database import
- PUSH: Software updates from AD2 -> NAS for DOS machines

### Stage 3: TestDataDB (Node.js/SQLite, WORKING)
- App: C:\Shares\testdatadb\ (Windows service "testdatadb", auto-start)
- API: http://192.168.0.6:3000
- Database: C:\Shares\testdatadb\database\testdata.db (2.27M records)
- Import: database/import.js (post-import hook calls export)
- **NEW: Spec parser** (parsers/spec-reader.js) - reads binary spec DATs, 1470 models
- **NEW: Exact-match formatter** (templates/datasheet-exact.js) - reverse-engineered from QB
- **NEW: Auto-export** (database/export-datasheets.js) - generates TXT to X:\For_Web

### Stage 4: WebShare (X: = \\ad2\webshare = C:\Shares\webshare)
- X:\Test_Datasheets - incoming (staging for old DFWDS)
- X:\For_Web - validated datasheets (501K+ files, pre-2026 archived to year subfolders)
- X:\For_Web_PDF - PDF versions (4.7K files)
- X:\Bad_Datasheets - invalid files (18K)
- X:\Datasheets_Log - DFWDS logs

### Stage 5: Website Upload (BROKEN)
- Old endpoints: dataforth.com/Services/{Uploader,DirectoryManifest,DeleteFile}.aspx - ALL 404
- Credentials: DataforthWebShare / Data6277
- TestDataSheetUploader (VB.NET, Hoffman) - not running, config pointed to dev paths
- Legacy site: legacy.dataforth.com/TestDataReport_Print.aspx (still works, no auth)
- New site: dataforth.com/TestDataReport (requires OIDC login)

## What Was Eliminated by Rebuild
- CTONWTXT.BAT (DOS TXT transfer) - no longer needed, server generates from DAT data
- DFWDS.exe (VB6 filename decoder) - no longer needed
- TestDataSheetUploader (VB.NET web uploader) - endpoints dead anyway

## Key File Encoding
H-prefix decode: A=10, B=11, C=12, D=13, E=14, F=15, G=16, H=17, I=18, J=19
Example: H8601-6.TXT -> serial 178601-6
New pipeline extracts SN from DAT record data directly, not filenames.

## Open Items
1. Website upload replacement (old ASP.NET endpoints dead)
2. 7B datasheet formatting (specs loaded, needs 7B-specific layout, ~830K records)
3. SCM5B49 spec file empty - need from John Lehman
4. Service permissions (runs as SYSTEM, causes SHM/WAL conflicts)
5. New product lines: MAQ20/PWRM (XLS), 10D (JSON, ~May 2026), DSCMHV

## Key Contacts
- John Lehman (jlehman@dataforth.com) - Engineering, QB code, specs
- Peter Iliya (pIliya@dataforth.com) - Applications Engineer, manual datasheet retrieval
- Ken Hoffman - TestDataSheetUploader author (VB.NET), DFWDS author, unresponsive
- Georg Haubner (ghaubner@dataforth.com) - D: drive has pre-crypto backup of network shares
- Ginger (gy@quatronix-cn.com) - Quatronix China, customer requesting datasheets

**Why:** Critical business issue - customer refusing shipments without datasheets.
**How to apply:** Pipeline is mostly rebuilt. Priority: website upload replacement, then 7B support.