Session log 2026-04-03: WO import, 7B support, PG migration started
- 33K work orders imported, 2.27M records linked - 7B exact-match formatter added (31 params, 120VAC, Packing Check List) - TXT formatting refined to match QB TAB positions exactly - PostgreSQL 18 installed on AD2, database created - SQL Server Express uninstalled - Full Dataforth audit document generated Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
sysadmin
177
session-logs/2026-04-03-session-ad2.md
Normal file
177
session-logs/2026-04-03-session-ad2.md
Normal file
@@ -0,0 +1,177 @@
|
||||
# Session Log - AD2 - 2026-04-03
|
||||
|
||||
## Session Summary
|
||||
|
||||
Continued test datasheet pipeline work from the 03-27/28/29 session. Major focus on:
|
||||
1. Fixing formatting issues in generated TXT datasheets (column alignment matching QuickBASIC)
|
||||
2. Adding 7B product family support to the exact-match formatter
|
||||
3. Loading additional spec files (5B49_2.DAT for the last missing Quatronix datasheet)
|
||||
4. Work order report ingestion (33K WOs, 63K lines, 2.27M records linked)
|
||||
5. UI improvements (WO search, WO detail popup, View button styled HTML)
|
||||
6. Generating full Dataforth audit document
|
||||
7. Planning and beginning PostgreSQL migration (SQLite hitting 4.4GB, queries timing out)
|
||||
8. Uninstalling SQL Server Express, installing PostgreSQL 18
|
||||
|
||||
## Key Decisions
|
||||
- PostgreSQL over SQL Server Express (no 10GB limit, better FTS, no licensing)
|
||||
- Install PG locally on AD2 (not SAGE-SQL or new VM)
|
||||
- Clean cutover (no parallel run with SQLite)
|
||||
- MSSQL Express uninstalled to free resources
|
||||
|
||||
## Credentials
|
||||
|
||||
### PostgreSQL (newly installed)
|
||||
- Host: localhost (AD2, 192.168.0.6)
|
||||
- Port: 5432
|
||||
- Superuser: postgres / DfPgSQL2026!
|
||||
- App role: testdatadb_app / DfTestDB2026!
|
||||
- Database: testdatadb
|
||||
- Service: postgresql-18 (runs as INTRANET\sysadmin)
|
||||
- Install path: C:\Program Files\PostgreSQL\18\
|
||||
- Data dir: C:\Program Files\PostgreSQL\18\data\
|
||||
|
||||
### TestDataDB (existing)
|
||||
- Service: testdatadb (runs as INTRANET\svc_testdatadb / DfSvcTDB2026!)
|
||||
- URL: http://192.168.0.6:3000
|
||||
- SQLite DB: C:\Shares\testdatadb\database\testdata.db (4.4 GB)
|
||||
|
||||
### Existing (from CLAUDE.md)
|
||||
- AD Sysadmin: INTRANET\sysadmin / Paper123!@#
|
||||
- D2TESTNAS SSH: root@192.168.0.9 / Paper123!@#-nas
|
||||
- Rsync: port 873, user rsync / IQ203s32119
|
||||
- M365 Tenant: 7dfa3ce8-c496-4b51-ab8d-bd3dcd78b584
|
||||
- Gitea: mike@azcomputerguru.com / Gptf*77ttb123!@#-git
|
||||
- Gitea API Token: 9b1da4b79a38ef782268341d25a4b6880572063f
|
||||
|
||||
## Work Completed
|
||||
|
||||
### Formatting Fixes (datasheet-exact.js)
|
||||
- Compared generated TXT against originals from X:\For_Web\2025\
|
||||
- Fixed Final Test Results column alignment to match QB TAB positions:
|
||||
- TAB(5) param name, TAB(31) measured value, TAB(60-speclen) spec, TAB(61) unit, TAB(71) PASS/FAIL
|
||||
- Added setCol() and padToCol() helpers for exact column positioning
|
||||
- Fixed STR$() emulation: leading space for positive, drops leading zero (.03 not 0.03)
|
||||
- Fixed header spacing (Date, Model, SN fields)
|
||||
- Fixed checklist spacing to match QB TAB(45)
|
||||
- Fixed separator line length (23 chars not 24)
|
||||
|
||||
### 7B Product Family Support
|
||||
- Added SCM7B DATA_LINES (31 parameters vs 20 for SCM5B)
|
||||
- Added SCM7B TSPECS (supply current, linearity, accuracy, excitation, CJC, noise, etc.)
|
||||
- Added 7B raw_data parser (single CSV line format vs multi-line for other families)
|
||||
- 7B-specific footer: 120VAC (not 240), "Packing Check List" (not "Check List"), blank checkmarks, Tested by/QC lines
|
||||
- Accuracy section suppressed for 7B (DAT format doesn't include individual test points)
|
||||
- 7B model names prepend "SCM" in header
|
||||
|
||||
### Additional Spec File: 5B49_2.DAT
|
||||
- 5B49DATA.DAT was 0 bytes (empty), John pointed to 5B49_2.DAT
|
||||
- 15 models, 93 bytes/record, added SCM5B49_FIELDS to spec-reader.js
|
||||
- Completed final missing Quatronix datasheet (177000-15, SCM5B49-05)
|
||||
- All 73/73 Quatronix datasheets now generated
|
||||
|
||||
### View Button Enhancement
|
||||
- /api/datasheet/:id now uses exact-match formatter with styled HTML
|
||||
- White page on gray background, monospace font, print-optimized
|
||||
- Includes Print and Download PDF buttons
|
||||
- Falls back to generic template if exact-match not available
|
||||
|
||||
### PDF Endpoint Fallback
|
||||
- /api/datasheet/:id/pdf falls back to generic template when exact-match fails
|
||||
- Prevents 422 errors for unsupported families
|
||||
|
||||
### Work Order Reports
|
||||
- Created parsers/wo-report.js (parses WO status report TXT format)
|
||||
- Created database/import-work-orders.js (imports WOs, creates tables, links to test records)
|
||||
- Created work_orders table (33,745 records) and work_order_lines table (63,263 records)
|
||||
- Added work_order column to test_records, 2,277,183 records linked via serial number pattern
|
||||
- Added /api/workorder/:wo and /api/workorder-search endpoints
|
||||
- Added WO# search field and clickable WO link in UI detail view
|
||||
- WO detail popup shows all test lines, pass/fail history, program version
|
||||
- Added auto-import hooks to Sync-FromNAS-rsync.ps1 for Reports and STAGE folders
|
||||
|
||||
### Sync Script Updates (Sync-FromNAS-rsync.ps1)
|
||||
- Added STAGE folder sync (NAS STAGE -> AD2)
|
||||
- Added $syncedReportFiles and $syncedStageFiles tracking
|
||||
- Added WO report auto-import after sync
|
||||
- Added STAGE TXT auto-import after sync
|
||||
- Updated status file with WO and STAGE counts
|
||||
|
||||
### Dataforth Audit Document
|
||||
- Generated C:\Users\sysadmin\Desktop\Dataforth-Audit-2026-04-02.txt
|
||||
- 22 sections: company info, network, AD computers/users/groups, GPO, DNS, shares, tasks, services, firewall, credentials, M365, security incidents, manufacturing infrastructure, pipeline, applications, backups, known issues, contacts
|
||||
|
||||
### PostgreSQL Migration (IN PROGRESS)
|
||||
- Plan approved: PostgreSQL 18 on AD2, clean cutover
|
||||
- PostgreSQL 18.3 installed via Chocolatey
|
||||
- Database cluster initialized (C:\Program Files\PostgreSQL\18\data\)
|
||||
- Service registered as postgresql-18 (runs as INTRANET\sysadmin)
|
||||
- Database created: testdatadb owned by testdatadb_app
|
||||
- SQL Server Express 2022 uninstalled
|
||||
|
||||
### SQLite Issues Documented
|
||||
- Database grown to 4.4GB
|
||||
- Single search query took 5,157 seconds (85 minutes)
|
||||
- Stats query took 216 seconds
|
||||
- WAL file grew to 454MB
|
||||
- ANALYZE ran for 30+ minutes without completing
|
||||
- SHM/WAL permission conflicts between SYSTEM and sysadmin
|
||||
|
||||
### Service Account (from previous session, still relevant)
|
||||
- INTRANET\svc_testdatadb created in OU=ServiceAccounts
|
||||
- testdatadb Windows service runs under this account
|
||||
- SeServiceLogonRight granted
|
||||
- Permissions on C:\Shares\testdatadb and C:\Shares\webshare
|
||||
|
||||
## Files Created
|
||||
- C:\Shares\testdatadb\fix-sysadmin-logon.ps1
|
||||
- C:\ClaudeTools\Test Datasheets\weekend-update-draft.md
|
||||
- C:\Users\sysadmin\Desktop\Dataforth-Audit-2026-04-02.txt
|
||||
- C:\Users\sysadmin\.claude\plans\generic-petting-lovelace.md (PG migration plan)
|
||||
|
||||
## Files Modified
|
||||
- C:\Shares\testdatadb\templates\datasheet-exact.js — Column alignment, 7B support, STR$() emulation
|
||||
- C:\Shares\testdatadb\parsers\spec-reader.js — Added 5B49 TYPE, fuzzy model matching
|
||||
- C:\Shares\testdatadb\parsers\wo-report.js — Created (WO report parser)
|
||||
- C:\Shares\testdatadb\database\import-work-orders.js — Created (WO importer)
|
||||
- C:\Shares\testdatadb\database\schema.sql — Added work_order column
|
||||
- C:\Shares\testdatadb\routes\api.js — Added WO endpoints, PDF fallback, view button exact-match
|
||||
- C:\Shares\testdatadb\public\index.html — WO search, WO popup, view button styling
|
||||
- C:\Shares\test\scripts\Sync-FromNAS-rsync.ps1 — STAGE sync, WO auto-import
|
||||
|
||||
## Database Stats (as of session end)
|
||||
- Test records: 2,294,877
|
||||
- Work orders: 33,745
|
||||
- Work order lines: 63,263
|
||||
- Records with WO linked: 2,277,183
|
||||
- ForWeb exported: ~1,436,000
|
||||
- Model specs loaded: 1,470+
|
||||
|
||||
## Pending / Next Steps
|
||||
1. **PostgreSQL Migration (Step 2-8):**
|
||||
- Create PG schema (schema-pg.sql) with tsvector FTS
|
||||
- Create database abstraction layer (db.js)
|
||||
- Migrate routes/api.js to async
|
||||
- Migrate import/export scripts
|
||||
- Build data migration script
|
||||
- Migrate 2.28M records
|
||||
- Test and cutover
|
||||
|
||||
2. **Tune PostgreSQL config:**
|
||||
- shared_buffers=1GB, work_mem=64MB, maintenance_work_mem=256MB
|
||||
|
||||
3. **MSSQL cleanup:**
|
||||
- May need reboot to fully remove SQL Server services
|
||||
|
||||
4. **Website upload:**
|
||||
- Old ASP.NET endpoints still dead (404)
|
||||
- Need to determine new upload mechanism
|
||||
|
||||
5. **Joel Lohr account:**
|
||||
- Retired March 31 — account needs to be disabled
|
||||
|
||||
## Infrastructure
|
||||
- AD2: 192.168.0.6 (Windows Server 2016)
|
||||
- PostgreSQL 18: localhost:5432 (newly installed, service running)
|
||||
- TestDataDB: localhost:3000 (SQLite, service running)
|
||||
- NAS: 192.168.0.9 (rsync port 873)
|
||||
- STAGE backlog: cleared (0 files remaining)
|
||||
Reference in New Issue
Block a user