Files
Mike Swanson 45083f4735 Add SCMVAS/SCMHVAS datasheet pipeline extension (Dataforth)
Extends the Test Datasheet Pipeline on AD2:C:\Shares\testdatadb to
generate web-published datasheets for the SCMVAS-Mxxx (obsolete) and
SCMHVAS-Mxxxx (replacement) High Voltage Input Module product lines.
Both are tested either with the existing TESTHV3 software (production
VASLOG .DAT logs) or in Engineering with plain .txt output.

Key changes on AD2 (all deployed 2026-04-12 with dated backups):

- parsers/spec-reader.js: getSpecs() returns a `{_family:'SCMVAS',
  _noSpecs:true}` sentinel for SCMVAS/SCMHVAS/VAS-M/HVAS-M model prefixes
  so the export pipeline does not silently skip them for missing specs.
- templates/datasheet-exact.js: new Accuracy-only template branch
  (generateSCMVASDatasheet + helpers) that mirrors the existing shipped
  format byte-for-byte. Extraction regex covers both QuickBASIC STR$()
  output formats: scientific-with-trailing-status-digit (98.4% of
  records) and plain-decimal (1.6% of records above QB's threshold).
- parsers/vaslog-engtxt.js (new): parses the Engineering-Tested .txt
  files in TS-3R\LOGS\VASLOG\VASLOG - Engineering Tested\. Filename SN
  regex strips optional trailing 14-digit timestamp; in-file "SN:"
  header is the authoritative source when the filename is malformed.
- database/import.js: LOG_TYPES grows a VASLOG_ENG entry with
  subfolder + recursive flags. Pre-existing 7 log types keep their
  implicit recursive=true behaviour (config.recursive !== false).
  importFiles() routes VASLOG_ENG paths before the generic loop so a
  VASLOG - Engineering Tested/*.txt path does not mis-dispatch to the
  multiline parser.
- database/export-datasheets.js: VASLOG_ENG records are written
  verbatim via fs.copyFileSync(source_file, For_Web/<SN>.TXT) for true
  byte-level pass-through, with a graceful raw_data fallback when the
  source file is no longer on disk.

Deploy outcome:
- 27,503 SCMVAS/SCMHVAS datasheets rendered (27,065 from scientific +
  438 from plain-decimal PASS lines, post-patch rerun)
- 434 Engineering-Tested .txt files pass-through-copied to For_Web
- 0 errors across both batches

Repo layout added here:
- scmvas-hvas-research/: discovery artifacts (source .BAS, hvin.dat,
  sample .DAT + .txt, binary-format notes, IMPLEMENTATION_PLAN.md)
- implementation/: staged final code + deploy helpers + local test
  harness + per-step verification scripts
- backups/pre-deploy-20260412/: independent local snapshot of the 4
  AD2 files replaced, pulled byte-for-byte before deploy

All helper scripts fetch the AD2 password at runtime from the SOPS
vault (clients/dataforth/ad2.sops.yaml). None of the committed files
contain the plaintext credential. Known vault-entry hygiene issue
(stale shell-escape backslash before the `!`) is documented in the
fetcher comments and stripped at read-time; flagged separately for
cleanup.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 07:36:45 -07:00

140 lines
4.0 KiB
JavaScript

/**
* PostgreSQL Database Abstraction Layer
*
* Provides a connection pool and helper methods for the TestDataDB app.
* Replaces better-sqlite3 singleton with pg.Pool.
*
* Environment variables (all optional, defaults connect to local PG):
* PGHOST (default: localhost)
* PGPORT (default: 5432)
* PGUSER (default: testdatadb_app)
* PGPASSWORD (default: DfTestDB2026!)
* PGDATABASE (default: testdatadb)
*/
const { Pool } = require('pg');
const pool = new Pool({
host: process.env.PGHOST || 'localhost',
port: parseInt(process.env.PGPORT || '5432', 10),
user: process.env.PGUSER || 'testdatadb_app',
password: process.env.PGPASSWORD || 'DfTestDB2026!',
database: process.env.PGDATABASE || 'testdatadb',
max: 20,
idleTimeoutMillis: 30000,
connectionTimeoutMillis: 5000,
});
pool.on('error', (err) => {
console.error(`[${new Date().toISOString()}] [PG POOL ERROR] ${err.message}`);
});
/**
* Convert SQLite-style ? placeholders to PostgreSQL $1, $2, ... placeholders.
* Skips ? inside single-quoted strings.
*/
function convertPlaceholders(sql) {
let idx = 0;
let inString = false;
let result = '';
for (let i = 0; i < sql.length; i++) {
const ch = sql[i];
if (ch === "'" && (i === 0 || sql[i - 1] !== '\\')) {
inString = !inString;
result += ch;
} else if (ch === '?' && !inString) {
idx++;
result += '$' + idx;
} else {
result += ch;
}
}
return result;
}
/**
* Execute a query, return all rows.
* @param {string} sql - SQL with ? or $N placeholders
* @param {Array} params - Parameter values
* @returns {Promise<Array>} rows
*/
async function query(sql, params = []) {
const pgSql = sql.includes('?') ? convertPlaceholders(sql) : sql;
const result = await pool.query(pgSql, params);
return result.rows;
}
/**
* Execute a query, return the first row or null.
*/
async function queryOne(sql, params = []) {
const rows = await query(sql, params);
return rows[0] || null;
}
/**
* Execute a statement (INSERT/UPDATE/DELETE), return { rowCount }.
*/
async function execute(sql, params = []) {
const pgSql = sql.includes('?') ? convertPlaceholders(sql) : sql;
const result = await pool.query(pgSql, params);
return { rowCount: result.rowCount, rows: result.rows };
}
/**
* Run a function inside a transaction.
* The callback receives a client with query/execute helpers.
* @param {Function} fn - async (client) => result
* @returns {Promise<*>} result of fn
*/
async function transaction(fn) {
const client = await pool.connect();
try {
await client.query('BEGIN');
const txClient = {
async query(sql, params = []) {
const pgSql = sql.includes('?') ? convertPlaceholders(sql) : sql;
const result = await client.query(pgSql, params);
return result.rows;
},
async queryOne(sql, params = []) {
const rows = await txClient.query(sql, params);
return rows[0] || null;
},
async execute(sql, params = []) {
const pgSql = sql.includes('?') ? convertPlaceholders(sql) : sql;
const result = await client.query(pgSql, params);
return { rowCount: result.rowCount, rows: result.rows };
},
// Direct pg client access for COPY or other advanced operations
raw: client,
};
const result = await fn(txClient);
await client.query('COMMIT');
return result;
} catch (err) {
await client.query('ROLLBACK');
throw err;
} finally {
client.release();
}
}
/**
* Close the pool (for graceful shutdown).
*/
async function close() {
await pool.end();
}
/**
* Get the raw pool (for advanced use like COPY).
*/
function getPool() {
return pool;
}
module.exports = { query, queryOne, execute, transaction, close, getPool, convertPlaceholders };