Mike Swanson
b1a588d0db
Author: Mike Swanson Machine: DESKTOP-0O8A1RL Timestamp: 2026-05-12 06:47:00
78 lines
4.0 KiB
Markdown
78 lines
4.0 KiB
Markdown
# Dataforth DOS — Project State
|
|
|
|
> READ THIS before starting work on this project.
|
|
> UPDATE THIS when you begin work (claim a lock) and when you finish (release lock + log changes).
|
|
> Last updated: 2026-05-12
|
|
|
|
---
|
|
|
|
## Active Session Locks
|
|
|
|
| Session | Working On | Status | Started |
|
|
|---------|-----------|--------|---------|
|
|
| _(none active)_ | | | |
|
|
|
|
**How to claim a lock:** Add a row before starting work. Remove it when done. Locks older than 2 hours with no update are considered stale.
|
|
|
|
---
|
|
|
|
## Current State
|
|
|
|
**Status:** ACTIVE / DEPLOYED
|
|
**Last Activity:** 2026-05-12
|
|
|
|
Pipeline is healthy and fully operational. testdatadb service running on AD2 (PostgreSQL backend, 469K records, 458K on website). Daily scheduled task (`DataforthTestDatasheetUploader`) runs at 02:30 AM and ran clean this morning (16 created, 9 updated, 0 errors). Email notification code deployed — blocked on M365 SMTP AUTH configuration (AJ must enable "Authenticated SMTP" for sysadmin@dataforth.com in Exchange Admin Center, or create an Entra app with Mail.Send). See CONTEXT.md for details.
|
|
|
|
---
|
|
|
|
## Infrastructure / Access
|
|
|
|
| Component | IP/Location | Notes |
|
|
|-----------|-------------|-------|
|
|
| AD2 (primary) | 192.168.0.6 | Windows Server 2022, hosts testdatadb service on port 3000 |
|
|
| AD1 | 192.168.0.27 | Hosts Engineering share at \\AD1\Engineering |
|
|
| D2TESTNAS | 192.168.0.9 | SMB1 only — bridge for DOS test stations |
|
|
| VPN required | FortiClient | Must be connected to reach 192.168.0.x |
|
|
|
|
**Service:** `testdatadb` on AD2, account `INTRANET\svc_testdatadb`, working dir `C:\Shares\testdatadb`, API port 3000.
|
|
**Database:** SQLite at `C:\Shares\testdatadb\database/testdata.db` (4.1 GB, not in git).
|
|
**Web output:** `\\ad2\webshare\For_Web` (NOT `X:\For_Web` in SSH sessions).
|
|
|
|
**Credentials:**
|
|
```bash
|
|
# AD2 password (strip backslash escape)
|
|
bash D:/vault/scripts/vault.sh get-field clients/dataforth/ad2.sops.yaml credentials.password | sed 's/\\//g'
|
|
# AD1 password
|
|
bash D:/vault/scripts/vault.sh get-field clients/dataforth/ad1.sops.yaml credentials.password
|
|
```
|
|
|
|
**Anti-patterns:** See `CONTEXT.md` — especially: no X: drive in SSH, no 50+ args on PowerShell, no hardcoded passwords, no SMB1 on AD2.
|
|
|
|
---
|
|
|
|
## Pending / Next Up
|
|
|
|
- [ ] **BLOCKER (email):** AJ enable "Authenticated SMTP" for sysadmin@dataforth.com in Exchange Admin Center (https://admin.exchange.microsoft.com → Mailboxes → sysadmin → Manage mail flow settings → Authenticated SMTP). Once done, re-run test email to confirm, then add jlehman@dataforth.com to TO list in notify.js and run-pipeline.ps1.
|
|
- [ ] After email confirmed working: add John Lehman (jlehman@dataforth.com) to notify.js `TO` constant and run-pipeline.ps1 `SendEmail -To` list. Redeploy both.
|
|
- [ ] Clean vault entry: ad2.sops.yaml has stale backslash in password (remove `\` escape)
|
|
- [ ] Clean up diagnostic scripts from C:\Shares\testdatadb\database\ (all underscore-prefixed files from 2026-04-15 session)
|
|
|
|
---
|
|
|
|
## Recent Changes
|
|
|
|
| Date | By | Change | Status |
|
|
|------|-----|--------|--------|
|
|
| 2026-05-12 | Mike | Email notifications: nodemailer deployed, credentials.json SMTP creds, run-pipeline.ps1 summary email, TEST-DATASHEET-PROCESS.md docs, CONTEXT.md updated | DEPLOYED (blocked on M365 SMTP AUTH) |
|
|
| 2026-04-22 | Mike | Modifications to import.js, notify.js, upload-to-api.js (undocumented session) | DEPLOYED |
|
|
| 2026-04-15 | Mike | DB dedup (2.89M→469K), FAIL→PASS rule, For_Web eliminated, 170,984 records pushed to Hoffman, dashboard UI | DEPLOYED |
|
|
| 2026-04-12 | Mike | SCMVAS/SCMHVAS pipeline: new VASLOG parser, accuracy-only template, 27,503 records backfilled, 434 ENG txt files imported | DEPLOYED |
|
|
| 2026-04-11 | Mike | Discovery session: explored VASLOG .DAT format, hvin.dat binary structure | RESEARCH |
|
|
|
|
---
|
|
|
|
## How to Update
|
|
|
|
**When starting:** Add your session to Active Session Locks (Session = "User/Machine", e.g. "Mike/DESKTOP-0O8A1RL").
|
|
**When finishing:** Remove your lock row, add entries to Recent Changes, update Current State if needed.
|