diff --git a/Test Datasheets/weekend-update-draft.md b/Test Datasheets/weekend-update-draft.md new file mode 100644 index 0000000..5bf2ae3 --- /dev/null +++ b/Test Datasheets/weekend-update-draft.md @@ -0,0 +1,80 @@ +Subject: Test Datasheets - Weekend Update: All 73 Quatronix Sheets Generated, Work Order Search Live + +John, Ken, + +Quick update on progress since Friday's email. The pipeline is significantly further along. + +## Quatronix Customer Issue - RESOLVED + +All 73 requested datasheets have been generated (TXT + PDF). The last holdout was SCM5B49-05 (SN 177000-15) — the 5B49DATA.DAT spec file was empty, but John pointed us to 5B49_2.DAT which had the data. All 73 files are ready to send to Peter/Ginger. + +## Model Spec Coverage Expanded + +We went from 751 model specs to 1,470+ by loading additional spec databases: + +| Spec File | Family | Models | +|-----------|--------|--------| +| 5BMAIN.DAT | SCM5B | 481 | +| 5B45DATA.DAT | SCM5B (freq/counter) | 56 | +| DB5B48.DAT | SCM5B (multi-bandwidth) | 3 | +| 5B49_2.DAT | SCM5B (sample & hold) | 15 | +| 8BMAIN.DAT | 8B | 148 | +| DSCOUT.DAT | DSCA (output) | 23 | +| DSCMAIN4.DAT | DSCA (input) | 391 | +| SCTMAIN.DAT | DSCT | 103 | +| 7BMAIN.DAT | SCM7B | 276 | + +If there are additional spec files we're missing, let me know the paths and we'll add them. + +## SCM7B Support Added + +The 7B product family is now fully supported in the datasheet generator: +- 31 test parameters (vs 20 for SCM5B) +- Correct header ("SCM" prefix prepended to model name) +- 120VAC Withstand / Hi-Pot (skipped for 7BPT models) +- "Packing Check List" with blank fields (vs pre-marked checkboxes on 5B/8B) +- "Tested by" and "QC" signature lines +- Note: The 7B DAT format (single CSV line) doesn't include individual accuracy test points, so the accuracy table is omitted. Only the Final Test Results section is generated from DAT data. + +## Work Order Search & Linking + +Imported all 33,745 work order status reports from the test station Reports folders: +- 63,263 individual test lines parsed (serial number, model, pass/fail, date/time, station) +- 2.27 million test records linked to their work orders + +In the web app (http://192.168.0.6:3000): +- New "Work Order #" search field — enter a WO number to find all associated test records +- Click the WO number in any record's detail view to see the full work order: + - All serial numbers tested under that WO + - Pass/fail status for each (including retests) + - Test program and version used + - Test station and timestamps +- New work order reports are automatically imported when synced from the NAS + +## Datasheet Formatting Refined + +Compared generated datasheets against originals from the DFWDS archive and fixed column alignment to match the QuickBASIC output: +- TAB positions match exactly (parameter names, measured values, spec limits, pass/fail) +- Number formatting matches QB PRINT USING (right-justified, correct decimal places) +- STR$() behavior replicated (leading space for positive numbers, dropped leading zeros) +- Spec limit formatting matches (e.g., "+/- .03 %" not "+/- 0.03 %") + +## View Button Updated + +The "SHEET" button in the web app now shows a styled HTML page that matches the PDF/TXT layout — white page, monospace font, same column alignment. Includes Print and Download PDF buttons. + +## Infrastructure + +- Created domain service account (INTRANET\svc_testdatadb) for the TestDataDB Windows service — resolves the file permission issues we were hitting +- Added STAGE folder sync to the NAS sync script — TXT datasheets from DOS machines will now be pulled to AD2 automatically +- Work order report import added to sync script — new reports are ingested automatically every 15 minutes + +## Still Open + +1. **Website upload** — The old Uploader.aspx endpoints are dead. Need to determine the new upload mechanism for dataforth.com. +2. **STAGE backlog** — ~8,100 TXT files on the NAS from DOS machines need to be processed (script ready, haven't run it yet). +3. **Pending ForWeb export** — ~845K records in the database don't have TXT files in For_Web yet (mostly 7B and older records). Can batch-export as needed. + +Let me know if you need anything else. + +Mike diff --git a/session-logs/2026-04-03-session-ad2.md b/session-logs/2026-04-03-session-ad2.md new file mode 100644 index 0000000..a5f8e6b --- /dev/null +++ b/session-logs/2026-04-03-session-ad2.md @@ -0,0 +1,177 @@ +# Session Log - AD2 - 2026-04-03 + +## Session Summary + +Continued test datasheet pipeline work from the 03-27/28/29 session. Major focus on: +1. Fixing formatting issues in generated TXT datasheets (column alignment matching QuickBASIC) +2. Adding 7B product family support to the exact-match formatter +3. Loading additional spec files (5B49_2.DAT for the last missing Quatronix datasheet) +4. Work order report ingestion (33K WOs, 63K lines, 2.27M records linked) +5. UI improvements (WO search, WO detail popup, View button styled HTML) +6. Generating full Dataforth audit document +7. Planning and beginning PostgreSQL migration (SQLite hitting 4.4GB, queries timing out) +8. Uninstalling SQL Server Express, installing PostgreSQL 18 + +## Key Decisions +- PostgreSQL over SQL Server Express (no 10GB limit, better FTS, no licensing) +- Install PG locally on AD2 (not SAGE-SQL or new VM) +- Clean cutover (no parallel run with SQLite) +- MSSQL Express uninstalled to free resources + +## Credentials + +### PostgreSQL (newly installed) +- Host: localhost (AD2, 192.168.0.6) +- Port: 5432 +- Superuser: postgres / DfPgSQL2026! +- App role: testdatadb_app / DfTestDB2026! +- Database: testdatadb +- Service: postgresql-18 (runs as INTRANET\sysadmin) +- Install path: C:\Program Files\PostgreSQL\18\ +- Data dir: C:\Program Files\PostgreSQL\18\data\ + +### TestDataDB (existing) +- Service: testdatadb (runs as INTRANET\svc_testdatadb / DfSvcTDB2026!) +- URL: http://192.168.0.6:3000 +- SQLite DB: C:\Shares\testdatadb\database\testdata.db (4.4 GB) + +### Existing (from CLAUDE.md) +- AD Sysadmin: INTRANET\sysadmin / Paper123!@# +- D2TESTNAS SSH: root@192.168.0.9 / Paper123!@#-nas +- Rsync: port 873, user rsync / IQ203s32119 +- M365 Tenant: 7dfa3ce8-c496-4b51-ab8d-bd3dcd78b584 +- Gitea: mike@azcomputerguru.com / Gptf*77ttb123!@#-git +- Gitea API Token: 9b1da4b79a38ef782268341d25a4b6880572063f + +## Work Completed + +### Formatting Fixes (datasheet-exact.js) +- Compared generated TXT against originals from X:\For_Web\2025\ +- Fixed Final Test Results column alignment to match QB TAB positions: + - TAB(5) param name, TAB(31) measured value, TAB(60-speclen) spec, TAB(61) unit, TAB(71) PASS/FAIL +- Added setCol() and padToCol() helpers for exact column positioning +- Fixed STR$() emulation: leading space for positive, drops leading zero (.03 not 0.03) +- Fixed header spacing (Date, Model, SN fields) +- Fixed checklist spacing to match QB TAB(45) +- Fixed separator line length (23 chars not 24) + +### 7B Product Family Support +- Added SCM7B DATA_LINES (31 parameters vs 20 for SCM5B) +- Added SCM7B TSPECS (supply current, linearity, accuracy, excitation, CJC, noise, etc.) +- Added 7B raw_data parser (single CSV line format vs multi-line for other families) +- 7B-specific footer: 120VAC (not 240), "Packing Check List" (not "Check List"), blank checkmarks, Tested by/QC lines +- Accuracy section suppressed for 7B (DAT format doesn't include individual test points) +- 7B model names prepend "SCM" in header + +### Additional Spec File: 5B49_2.DAT +- 5B49DATA.DAT was 0 bytes (empty), John pointed to 5B49_2.DAT +- 15 models, 93 bytes/record, added SCM5B49_FIELDS to spec-reader.js +- Completed final missing Quatronix datasheet (177000-15, SCM5B49-05) +- All 73/73 Quatronix datasheets now generated + +### View Button Enhancement +- /api/datasheet/:id now uses exact-match formatter with styled HTML +- White page on gray background, monospace font, print-optimized +- Includes Print and Download PDF buttons +- Falls back to generic template if exact-match not available + +### PDF Endpoint Fallback +- /api/datasheet/:id/pdf falls back to generic template when exact-match fails +- Prevents 422 errors for unsupported families + +### Work Order Reports +- Created parsers/wo-report.js (parses WO status report TXT format) +- Created database/import-work-orders.js (imports WOs, creates tables, links to test records) +- Created work_orders table (33,745 records) and work_order_lines table (63,263 records) +- Added work_order column to test_records, 2,277,183 records linked via serial number pattern +- Added /api/workorder/:wo and /api/workorder-search endpoints +- Added WO# search field and clickable WO link in UI detail view +- WO detail popup shows all test lines, pass/fail history, program version +- Added auto-import hooks to Sync-FromNAS-rsync.ps1 for Reports and STAGE folders + +### Sync Script Updates (Sync-FromNAS-rsync.ps1) +- Added STAGE folder sync (NAS STAGE -> AD2) +- Added $syncedReportFiles and $syncedStageFiles tracking +- Added WO report auto-import after sync +- Added STAGE TXT auto-import after sync +- Updated status file with WO and STAGE counts + +### Dataforth Audit Document +- Generated C:\Users\sysadmin\Desktop\Dataforth-Audit-2026-04-02.txt +- 22 sections: company info, network, AD computers/users/groups, GPO, DNS, shares, tasks, services, firewall, credentials, M365, security incidents, manufacturing infrastructure, pipeline, applications, backups, known issues, contacts + +### PostgreSQL Migration (IN PROGRESS) +- Plan approved: PostgreSQL 18 on AD2, clean cutover +- PostgreSQL 18.3 installed via Chocolatey +- Database cluster initialized (C:\Program Files\PostgreSQL\18\data\) +- Service registered as postgresql-18 (runs as INTRANET\sysadmin) +- Database created: testdatadb owned by testdatadb_app +- SQL Server Express 2022 uninstalled + +### SQLite Issues Documented +- Database grown to 4.4GB +- Single search query took 5,157 seconds (85 minutes) +- Stats query took 216 seconds +- WAL file grew to 454MB +- ANALYZE ran for 30+ minutes without completing +- SHM/WAL permission conflicts between SYSTEM and sysadmin + +### Service Account (from previous session, still relevant) +- INTRANET\svc_testdatadb created in OU=ServiceAccounts +- testdatadb Windows service runs under this account +- SeServiceLogonRight granted +- Permissions on C:\Shares\testdatadb and C:\Shares\webshare + +## Files Created +- C:\Shares\testdatadb\fix-sysadmin-logon.ps1 +- C:\ClaudeTools\Test Datasheets\weekend-update-draft.md +- C:\Users\sysadmin\Desktop\Dataforth-Audit-2026-04-02.txt +- C:\Users\sysadmin\.claude\plans\generic-petting-lovelace.md (PG migration plan) + +## Files Modified +- C:\Shares\testdatadb\templates\datasheet-exact.js — Column alignment, 7B support, STR$() emulation +- C:\Shares\testdatadb\parsers\spec-reader.js — Added 5B49 TYPE, fuzzy model matching +- C:\Shares\testdatadb\parsers\wo-report.js — Created (WO report parser) +- C:\Shares\testdatadb\database\import-work-orders.js — Created (WO importer) +- C:\Shares\testdatadb\database\schema.sql — Added work_order column +- C:\Shares\testdatadb\routes\api.js — Added WO endpoints, PDF fallback, view button exact-match +- C:\Shares\testdatadb\public\index.html — WO search, WO popup, view button styling +- C:\Shares\test\scripts\Sync-FromNAS-rsync.ps1 — STAGE sync, WO auto-import + +## Database Stats (as of session end) +- Test records: 2,294,877 +- Work orders: 33,745 +- Work order lines: 63,263 +- Records with WO linked: 2,277,183 +- ForWeb exported: ~1,436,000 +- Model specs loaded: 1,470+ + +## Pending / Next Steps +1. **PostgreSQL Migration (Step 2-8):** + - Create PG schema (schema-pg.sql) with tsvector FTS + - Create database abstraction layer (db.js) + - Migrate routes/api.js to async + - Migrate import/export scripts + - Build data migration script + - Migrate 2.28M records + - Test and cutover + +2. **Tune PostgreSQL config:** + - shared_buffers=1GB, work_mem=64MB, maintenance_work_mem=256MB + +3. **MSSQL cleanup:** + - May need reboot to fully remove SQL Server services + +4. **Website upload:** + - Old ASP.NET endpoints still dead (404) + - Need to determine new upload mechanism + +5. **Joel Lohr account:** + - Retired March 31 — account needs to be disabled + +## Infrastructure +- AD2: 192.168.0.6 (Windows Server 2016) +- PostgreSQL 18: localhost:5432 (newly installed, service running) +- TestDataDB: localhost:3000 (SQLite, service running) +- NAS: 192.168.0.9 (rsync port 873) +- STAGE backlog: cleared (0 files remaining)