Add AD1 session data, memory entries for datasheet pipeline and security incident
- Imported AD1 Claude session files to clients/dataforth/session-logs/ - Created memory: project_datasheet_pipeline.md (full pipeline architecture) - Created memory: project_dataforth_incident_2026-03-27.md (security incident + MFA) - Updated MEMORY.md index - Updated session log with AD1 pipeline rebuild findings Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
73
.claude/memory/project_datasheet_pipeline.md
Normal file
73
.claude/memory/project_datasheet_pipeline.md
Normal file
@@ -0,0 +1,73 @@
|
||||
---
|
||||
name: Dataforth Test Datasheet Pipeline - Rebuilt 2026-03-27
|
||||
description: Full pipeline from DOS test stations to website. New server-side generation replaces DFWDS/Uploader. 72/73 Quatronix datasheets generated. AD2 crypto wipe recovery.
|
||||
type: project
|
||||
---
|
||||
|
||||
## Background
|
||||
AD2 (192.168.0.6) was wiped in a crypto/ransomware attack months ago. The test datasheet pipeline was broken. Customer Quatronix (China) blocking shipment of 328 modules (whittled to 54) without datasheets.
|
||||
|
||||
## Pipeline (5 stages, rebuilt 2026-03-27)
|
||||
|
||||
### Stage 1: DOS Test Stations (64 stations)
|
||||
- QuickBASIC programs generate test data -> C:\STAGE on each DOS PC
|
||||
- DAT files (raw test data) + TXT files (formatted datasheets)
|
||||
- CTONW.BAT copies DAT files to NAS (working)
|
||||
- CTONWTXT.BAT copies TXT files (NOT called in current AUTOEXEC v4.1 since 2026-03-12)
|
||||
- TXT files piling up in C:\STAGE since Sept 2025
|
||||
|
||||
### Stage 2: NAS <-> AD2 Sync
|
||||
- Script: C:\Shares\test\scripts\Sync-FromNAS-rsync.ps1 (every 15 min, WORKING)
|
||||
- Rsync daemon on NAS: port 873, module "test", user rsync / IQ203s32119
|
||||
- PULL: DAT files from NAS -> AD2, triggers database import
|
||||
- PUSH: Software updates from AD2 -> NAS for DOS machines
|
||||
|
||||
### Stage 3: TestDataDB (Node.js/SQLite, WORKING)
|
||||
- App: C:\Shares\testdatadb\ (Windows service "testdatadb", auto-start)
|
||||
- API: http://192.168.0.6:3000
|
||||
- Database: C:\Shares\testdatadb\database\testdata.db (2.27M records)
|
||||
- Import: database/import.js (post-import hook calls export)
|
||||
- **NEW: Spec parser** (parsers/spec-reader.js) - reads binary spec DATs, 1470 models
|
||||
- **NEW: Exact-match formatter** (templates/datasheet-exact.js) - reverse-engineered from QB
|
||||
- **NEW: Auto-export** (database/export-datasheets.js) - generates TXT to X:\For_Web
|
||||
|
||||
### Stage 4: WebShare (X: = \\ad2\webshare = C:\Shares\webshare)
|
||||
- X:\Test_Datasheets - incoming (staging for old DFWDS)
|
||||
- X:\For_Web - validated datasheets (501K+ files, pre-2026 archived to year subfolders)
|
||||
- X:\For_Web_PDF - PDF versions (4.7K files)
|
||||
- X:\Bad_Datasheets - invalid files (18K)
|
||||
- X:\Datasheets_Log - DFWDS logs
|
||||
|
||||
### Stage 5: Website Upload (BROKEN)
|
||||
- Old endpoints: dataforth.com/Services/{Uploader,DirectoryManifest,DeleteFile}.aspx - ALL 404
|
||||
- Credentials: DataforthWebShare / Data6277
|
||||
- TestDataSheetUploader (VB.NET, Hoffman) - not running, config pointed to dev paths
|
||||
- Legacy site: legacy.dataforth.com/TestDataReport_Print.aspx (still works, no auth)
|
||||
- New site: dataforth.com/TestDataReport (requires OIDC login)
|
||||
|
||||
## What Was Eliminated by Rebuild
|
||||
- CTONWTXT.BAT (DOS TXT transfer) - no longer needed, server generates from DAT data
|
||||
- DFWDS.exe (VB6 filename decoder) - no longer needed
|
||||
- TestDataSheetUploader (VB.NET web uploader) - endpoints dead anyway
|
||||
|
||||
## Key File Encoding
|
||||
H-prefix decode: A=10, B=11, C=12, D=13, E=14, F=15, G=16, H=17, I=18, J=19
|
||||
Example: H8601-6.TXT -> serial 178601-6
|
||||
New pipeline extracts SN from DAT record data directly, not filenames.
|
||||
|
||||
## Open Items
|
||||
1. Website upload replacement (old ASP.NET endpoints dead)
|
||||
2. 7B datasheet formatting (specs loaded, needs 7B-specific layout, ~830K records)
|
||||
3. SCM5B49 spec file empty - need from John Lehman
|
||||
4. Service permissions (runs as SYSTEM, causes SHM/WAL conflicts)
|
||||
5. New product lines: MAQ20/PWRM (XLS), 10D (JSON, ~May 2026), DSCMHV
|
||||
|
||||
## Key Contacts
|
||||
- John Lehman (jlehman@dataforth.com) - Engineering, QB code, specs
|
||||
- Peter Iliya (pIliya@dataforth.com) - Applications Engineer, manual datasheet retrieval
|
||||
- Ken Hoffman - TestDataSheetUploader author (VB.NET), DFWDS author, unresponsive
|
||||
- Georg Haubner (ghaubner@dataforth.com) - D: drive has pre-crypto backup of network shares
|
||||
- Ginger (gy@quatronix-cn.com) - Quatronix China, customer requesting datasheets
|
||||
|
||||
**Why:** Critical business issue - customer refusing shipments without datasheets.
|
||||
**How to apply:** Pipeline is mostly rebuilt. Priority: website upload replacement, then 7B support.
|
||||
Reference in New Issue
Block a user