claudetools/clients/dataforth/session-logs/project_test_datasheet_pipeline.md

name, description, type

name	description	type
Test datasheet pipeline map	Full pipeline from DOS test stations through NAS sync, DFWDS processing, to web upload at dataforth.com. Includes current status of each stage.	project

Complete Pipeline (5 stages)

Stage 1: Test Stations (QuickBASIC on DOS)

• 64 test stations (TS-1 through TS-30, plus L/R variants, TS-GURU, TS-TOM)
• QuickBASIC programs generate test data -> local C:\STAGE on each DOS PC
• Data flows to D2TESTNAS (NAS at 192.168.0.9, rsync daemon port 873, module "test" -> /data/test)

Stage 2: NAS <-> AD2 Sync (Sync-FromNAS-rsync.ps1)

• Runs every 15 min via scheduled task on AD2
• PULL: .DAT files + .TXT reports from NAS -> C:\Shares\test\TS-XX\LOGS<type>\ and \Reports\
• Imports .DAT files into TestDataDB via C:\Shares\testdatadb\database\import.js
• PUSH: Software updates (ProdSW, UPDATE.BAT, DEPLOY.BAT) from AD2 -> NAS for DOS machines
• Log types: 5BLOG, 7BLOG, 8BLOG, DSCLOG, SCTLOG, VASLOG, PWRLOG, HVLOG
• Script: C:\Shares\test\scripts\Sync-FromNAS-rsync.ps1
• Log: C:\Shares\test\scripts\sync-from-nas.log (rotated at 10MB, 5 archives)

Stage 3: DFWDS.exe (Dataforth Website Datasheet program, v2015)

• Source exe: \AD1\Engineering\ENGR\ATE\Test Datasheets\DFWDS\DFWDS.exe
• Config: C:\DFWDS\DFWDS_NAMES.TXT (hardcoded path in exe)
• Reads X:\Test_Datasheets (X: = \ad2\webshare)
• Validates files, renames DOS-encoded filenames to serial number format
• Valid -> X:\For_Web (~501K files), Invalid -> X:\Bad_Datasheets (~18K files)
• Logs -> X:\Datasheets_Log (daily logs since 2014)
• Operations: COUNT, LISTALL, LISTBAD, LISTRENAME, INPLACE, WEBMOVE

Stage 4: TestDataSheetUploader (VB.NET, .NET 4.7.2)

• Source: \AD1\Engineering\ENGR\ATE\Test Datasheets\TestDataSheetUploader\
• Syncs files to www.dataforth.com via 3 HTTP services:
  • Uploader.aspx, DirectoryManifest.aspx, DeleteFile.aspx
• Credentials: DataforthWebShare / Data6277
• Only uploads files modified in current year
• Compares local vs server manifest (filename, size, last-write-time)
• Deletes server files that no longer exist locally
• Author: "hoffm" (Hoffman)

Stage 5: PDF generation

• X:\For_Web_PDF has ~4,773 PDFs (separate process, origin unclear)

X: Drive Layout (\ad2\webshare)

• X:\Test_Datasheets — incoming datasheets (staging for DFWDS)
• X:\For_Web — validated/renamed datasheets (501K+ files)
• X:\For_Web_PDF — PDF versions (4.7K files)
• X:\Bad_Datasheets — invalid files (18K files)
• X:\Datasheets_Log — DFWDS operation logs

Datasheet Format

• Plain text, ~50 lines
• Header: Dataforth Corporation address/phone
• Fields: Date, Model (e.g. SCM5B41-03), SN (e.g. 178439-1)
• Accuracy test table (Vin, Calculated Vout, Measured Vout, Error%, Status)
• Final test results (Supply Current, Output Resistance, Linearity, Accuracy, etc.)
• Filename format: {SN}.txt (e.g. 178439-1.txt)

TestDataDB

• Node.js/SQLite app at C:\Shares\testdatadb\
• PM2-managed via start-server.bat
• Scheduled task "TestDataDB Server" runs at system startup (currently DISABLED)
• Provides web UI and API for querying test data
• Import script: C:\Shares\testdatadb\database\import.js

Known Gaps (as of 2026-03-27)

1. MISSING LINK: How do .TXT reports get from C:\Shares\test\TS-XX\Reports\ to X:\Test_Datasheets? May need to build this. Clues may exist in Engineering share.
2. DFWDS config (C:\DFWDS\DFWDS_NAMES.TXT) missing — wiped in crypto attack. Check Haubner machine D: drive for backup of visible shares.
3. TestDataSheetUploader not deployed/scheduled on AD2 — needs production config
4. TestDataDB Server task disabled — should be operational

Why: Reconstructing this pipeline after AD2 crypto wipe. Manufacturing is actively producing test data (files arriving daily) but downstream processing (DFWDS, web upload) is broken.

How to apply: Priority is restoring the full pipeline end-to-end. Check Haubner D: for DFWDS config backup. Look in ENGR share for clues about the Reports->Test_Datasheets transfer mechanism.