claudetools/Test Datasheets/findings-draft.md

Subject: RE: Test Datasheets to Website - Full Pipeline Analysis & Recovery Plan

John, Ken,

Following up on John's email - I've done a thorough trace of the entire test datasheet pipeline from the DOS test stations through to dataforth.com. Here's what I've found and what needs to be fixed.

Current Pipeline (5 Stages)

Stage 1 - QuickBASIC Test Programs (WORKING)

Test programs (e.g., TEST5B2E.BAS) write results two ways:

• LOGIT: binary .DAT files → C:\ATE{5BLOG,8BLOG,...}\
• DATASHEETWRITE: .TXT datasheets → C:\STAGE\

Filename encoding via GETDSFNAME:

• 5-digit WO#: {WO#}-{dash#}.TXT (e.g., 12345-1.TXT)
• 6-digit WO# (10xxxx-19xxxx): alpha-encoded (e.g., 123456 → C3456-1.TXT)
• Invalid WO#: BAD-{dash#}.TXT

Stage 2 - Network Transfer on Boot (BROKEN)

AUTOEXEC.BAT v4.1 (deployed 2026-03-12) calls:

1. STARTNET.BAT → maps T: = \D2TESTNAS\test, X: = \D2TESTNAS\datasheets
2. NWTOC.BAT → software updates (working)
3. CTONW.BAT → copies .DAT files to T:\TS-XX\LOGS\ (working)
4. menux → test menu (working)

PROBLEM: CTONWTXT.BAT is never called. This is the batch file that copies C:\STAGE*.txt to X: (the datasheets share). It exists on the machines (1,504 bytes, dated 03-12-26) but nothing invokes it. The old ARCHBAT.BAT called it, but ARCHBAT is not called in the current AUTOEXEC either.

Result: .TXT datasheets have been accumulating in C:\STAGE on the test stations since approximately September 2025. Peter confirmed finding old files there.

Fix: Add the following line to CTONW.BAT (or AUTOEXEC.BAT), after the existing .DAT transfer: CALL C:\BAT\CTONWTXT.BAT %MACHINE% X:

This can be deployed to all 64 stations via the existing NAS software push (Sync-FromNAS pushes updated ProdSW and BAT files to the NAS, and NWTOC pulls them to each station on boot).

Stage 3 - DFWDS.exe Processing (BROKEN)

DFWDS.exe (Dataforth Website Datasheet program, version 2015-06-08) processes files that arrive in X:\Test_Datasheets:

• Validates filenames and content
• Renames DOS-encoded filenames to serial number format
• Moves valid files → X:\For_Web (currently ~501,000 files)
• Moves invalid files → X:\Bad_Datasheets (~18,800 files)
• Logs to X:\Datasheets_Log

PROBLEM: The config file C:\DFWDS\DFWDS_NAMES.TXT was lost in the crypto wipe. The exe is hardcoded to read from that path. The ENGR share has a copy of the exe and the config template at \AD1\Engineering\ENGR\ATE\Test Datasheets\DFWDS. We also don't have a scheduled task for it on AD2.

DFWDS last ran successfully on 2026-03-11 (processed 978 files, 0 bad). Before that, there's a gap back to September 2025. The X:\Test_Datasheets folder currently has 338 files waiting to be processed.

Fix:

1. Create C:\DFWDS\ on AD2 and restore DFWDS_NAMES.TXT with the correct paths
2. Copy DFWDS.exe from the ENGR share
3. Create a scheduled task to run it (daily? on a schedule that matches production?)

Question for John/Ken: How often was DFWDS.exe previously run, and was it on a schedule or triggered manually?

Stage 4 - TestDataSheetUploader to Website (BROKEN)

Ken's VB.NET app (TestDataSheetUploader) syncs X:\For_Web to dataforth.com via three web services:

• Uploader.aspx - uploads new/changed files
• DirectoryManifest.aspx - gets server file list for comparison
• DeleteFile.aspx - removes obsolete files from server

The app compares local files against the server manifest by filename, size, and last-write-time, and only uploads files modified in the current year.

Source is on the ENGR share at: \AD1\Engineering\ENGR\ATE\Test Datasheets\TestDataSheetUploader
Last built: November 2, 2022

PROBLEM: Not deployed or scheduled on AD2. The app.config still has dev paths (C:\Users\hoffm...) instead of production paths.

Fix:

1. Update app.config TestDataSheetPath to point to X:\For_Web
2. Deploy the built exe to AD2
3. Create a scheduled task

Question for Ken: What was the production config for TestDataSheetPath? Was it X:\For_Web or something else? Was this run on a schedule or manually? Are the web service credentials (DataforthWebShare) still valid?

Stage 5 - PDF Generation (UNKNOWN)

X:\For_Web_PDF contains ~4,773 PDF versions of datasheets. I haven't identified what generates these.

Question: What creates the PDFs? Is this a separate process we need to restore?

What's Currently Working

• Test programs are generating data (files from today in X:\Test_Datasheets)
• NAS sync (Sync-FromNAS-rsync.ps1) runs every 15 min, pulling .DAT files and pushing software updates
• TestDataDB server is running on AD2 (port 3000), importing .DAT files
• X: drive (\AD2\webshare) is accessible with all folders intact
• 501K+ historical datasheets in X:\For_Web are preserved

What's Broken (in pipeline order)

1. CTONWTXT.BAT not called → TXT files stuck on DOS machines
2. DFWDS.exe config missing → files in Test_Datasheets not processed
3. TestDataSheetUploader not deployed → files in For_Web not synced to website

Proposed Fix Order

1. Fix CTONW.BAT to call CTONWTXT.BAT → deploy via NAS push
2. Restore DFWDS on AD2 (config + exe + scheduled task)
3. Deploy TestDataSheetUploader with production config + scheduled task

Let me know your thoughts and answers to the questions above so we can start restoring this.

Mike