Mike Swanson
b1a588d0db
Author: Mike Swanson Machine: DESKTOP-0O8A1RL Timestamp: 2026-05-12 06:47:00
4.0 KiB
Dataforth DOS — Project State
READ THIS before starting work on this project. UPDATE THIS when you begin work (claim a lock) and when you finish (release lock + log changes). Last updated: 2026-05-12
Active Session Locks
| Session | Working On | Status | Started |
|---|---|---|---|
| (none active) |
How to claim a lock: Add a row before starting work. Remove it when done. Locks older than 2 hours with no update are considered stale.
Current State
Status: ACTIVE / DEPLOYED Last Activity: 2026-05-12
Pipeline is healthy and fully operational. testdatadb service running on AD2 (PostgreSQL backend, 469K records, 458K on website). Daily scheduled task (DataforthTestDatasheetUploader) runs at 02:30 AM and ran clean this morning (16 created, 9 updated, 0 errors). Email notification code deployed — blocked on M365 SMTP AUTH configuration (AJ must enable "Authenticated SMTP" for sysadmin@dataforth.com in Exchange Admin Center, or create an Entra app with Mail.Send). See CONTEXT.md for details.
Infrastructure / Access
| Component | IP/Location | Notes |
|---|---|---|
| AD2 (primary) | 192.168.0.6 | Windows Server 2022, hosts testdatadb service on port 3000 |
| AD1 | 192.168.0.27 | Hosts Engineering share at \AD1\Engineering |
| D2TESTNAS | 192.168.0.9 | SMB1 only — bridge for DOS test stations |
| VPN required | FortiClient | Must be connected to reach 192.168.0.x |
Service: testdatadb on AD2, account INTRANET\svc_testdatadb, working dir C:\Shares\testdatadb, API port 3000.
Database: SQLite at C:\Shares\testdatadb\database/testdata.db (4.1 GB, not in git).
Web output: \\ad2\webshare\For_Web (NOT X:\For_Web in SSH sessions).
Credentials:
# AD2 password (strip backslash escape)
bash D:/vault/scripts/vault.sh get-field clients/dataforth/ad2.sops.yaml credentials.password | sed 's/\\//g'
# AD1 password
bash D:/vault/scripts/vault.sh get-field clients/dataforth/ad1.sops.yaml credentials.password
Anti-patterns: See CONTEXT.md — especially: no X: drive in SSH, no 50+ args on PowerShell, no hardcoded passwords, no SMB1 on AD2.
Pending / Next Up
- BLOCKER (email): AJ enable "Authenticated SMTP" for sysadmin@dataforth.com in Exchange Admin Center (https://admin.exchange.microsoft.com → Mailboxes → sysadmin → Manage mail flow settings → Authenticated SMTP). Once done, re-run test email to confirm, then add jlehman@dataforth.com to TO list in notify.js and run-pipeline.ps1.
- After email confirmed working: add John Lehman (jlehman@dataforth.com) to notify.js
TOconstant and run-pipeline.ps1SendEmail -Tolist. Redeploy both. - Clean vault entry: ad2.sops.yaml has stale backslash in password (remove
\escape) - Clean up diagnostic scripts from C:\Shares\testdatadb\database\ (all underscore-prefixed files from 2026-04-15 session)
Recent Changes
| Date | By | Change | Status |
|---|---|---|---|
| 2026-05-12 | Mike | Email notifications: nodemailer deployed, credentials.json SMTP creds, run-pipeline.ps1 summary email, TEST-DATASHEET-PROCESS.md docs, CONTEXT.md updated | DEPLOYED (blocked on M365 SMTP AUTH) |
| 2026-04-22 | Mike | Modifications to import.js, notify.js, upload-to-api.js (undocumented session) | DEPLOYED |
| 2026-04-15 | Mike | DB dedup (2.89M→469K), FAIL→PASS rule, For_Web eliminated, 170,984 records pushed to Hoffman, dashboard UI | DEPLOYED |
| 2026-04-12 | Mike | SCMVAS/SCMHVAS pipeline: new VASLOG parser, accuracy-only template, 27,503 records backfilled, 434 ENG txt files imported | DEPLOYED |
| 2026-04-11 | Mike | Discovery session: explored VASLOG .DAT format, hvin.dat binary structure | RESEARCH |
How to Update
When starting: Add your session to Active Session Locks (Session = "User/Machine", e.g. "Mike/DESKTOP-0O8A1RL"). When finishing: Remove your lock row, add entries to Recent Changes, update Current State if needed.