ece3222d3a
- Imported AD1 Claude session files to clients/dataforth/session-logs/ - Created memory: project_datasheet_pipeline.md (full pipeline architecture) - Created memory: project_dataforth_incident_2026-03-27.md (security incident + MFA) - Updated MEMORY.md index - Updated session log with AD1 pipeline rebuild findings Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3.8 KiB
3.8 KiB
name, description, type
| name | description | type |
|---|---|---|
| Dataforth Test Datasheet Pipeline - Rebuilt 2026-03-27 | Full pipeline from DOS test stations to website. New server-side generation replaces DFWDS/Uploader. 72/73 Quatronix datasheets generated. AD2 crypto wipe recovery. | project |
Background
AD2 (192.168.0.6) was wiped in a crypto/ransomware attack months ago. The test datasheet pipeline was broken. Customer Quatronix (China) blocking shipment of 328 modules (whittled to 54) without datasheets.
Pipeline (5 stages, rebuilt 2026-03-27)
Stage 1: DOS Test Stations (64 stations)
- QuickBASIC programs generate test data -> C:\STAGE on each DOS PC
- DAT files (raw test data) + TXT files (formatted datasheets)
- CTONW.BAT copies DAT files to NAS (working)
- CTONWTXT.BAT copies TXT files (NOT called in current AUTOEXEC v4.1 since 2026-03-12)
- TXT files piling up in C:\STAGE since Sept 2025
Stage 2: NAS <-> AD2 Sync
- Script: C:\Shares\test\scripts\Sync-FromNAS-rsync.ps1 (every 15 min, WORKING)
- Rsync daemon on NAS: port 873, module "test", user rsync / IQ203s32119
- PULL: DAT files from NAS -> AD2, triggers database import
- PUSH: Software updates from AD2 -> NAS for DOS machines
Stage 3: TestDataDB (Node.js/SQLite, WORKING)
- App: C:\Shares\testdatadb\ (Windows service "testdatadb", auto-start)
- API: http://192.168.0.6:3000
- Database: C:\Shares\testdatadb\database\testdata.db (2.27M records)
- Import: database/import.js (post-import hook calls export)
- NEW: Spec parser (parsers/spec-reader.js) - reads binary spec DATs, 1470 models
- NEW: Exact-match formatter (templates/datasheet-exact.js) - reverse-engineered from QB
- NEW: Auto-export (database/export-datasheets.js) - generates TXT to X:\For_Web
Stage 4: WebShare (X: = \ad2\webshare = C:\Shares\webshare)
- X:\Test_Datasheets - incoming (staging for old DFWDS)
- X:\For_Web - validated datasheets (501K+ files, pre-2026 archived to year subfolders)
- X:\For_Web_PDF - PDF versions (4.7K files)
- X:\Bad_Datasheets - invalid files (18K)
- X:\Datasheets_Log - DFWDS logs
Stage 5: Website Upload (BROKEN)
- Old endpoints: dataforth.com/Services/{Uploader,DirectoryManifest,DeleteFile}.aspx - ALL 404
- Credentials: DataforthWebShare / Data6277
- TestDataSheetUploader (VB.NET, Hoffman) - not running, config pointed to dev paths
- Legacy site: legacy.dataforth.com/TestDataReport_Print.aspx (still works, no auth)
- New site: dataforth.com/TestDataReport (requires OIDC login)
What Was Eliminated by Rebuild
- CTONWTXT.BAT (DOS TXT transfer) - no longer needed, server generates from DAT data
- DFWDS.exe (VB6 filename decoder) - no longer needed
- TestDataSheetUploader (VB.NET web uploader) - endpoints dead anyway
Key File Encoding
H-prefix decode: A=10, B=11, C=12, D=13, E=14, F=15, G=16, H=17, I=18, J=19 Example: H8601-6.TXT -> serial 178601-6 New pipeline extracts SN from DAT record data directly, not filenames.
Open Items
- Website upload replacement (old ASP.NET endpoints dead)
- 7B datasheet formatting (specs loaded, needs 7B-specific layout, ~830K records)
- SCM5B49 spec file empty - need from John Lehman
- Service permissions (runs as SYSTEM, causes SHM/WAL conflicts)
- New product lines: MAQ20/PWRM (XLS), 10D (JSON, ~May 2026), DSCMHV
Key Contacts
- John Lehman (jlehman@dataforth.com) - Engineering, QB code, specs
- Peter Iliya (pIliya@dataforth.com) - Applications Engineer, manual datasheet retrieval
- Ken Hoffman - TestDataSheetUploader author (VB.NET), DFWDS author, unresponsive
- Georg Haubner (ghaubner@dataforth.com) - D: drive has pre-crypto backup of network shares
- Ginger (gy@quatronix-cn.com) - Quatronix China, customer requesting datasheets
Why: Critical business issue - customer refusing shipments without datasheets. How to apply: Pipeline is mostly rebuilt. Priority: website upload replacement, then 7B support.