sync: auto-sync from AD2 at 2026-06-18 18:44:28

Author: Mike Swanson Machine: AD2 Timestamp: 2026-06-18 18:44:28
2026-06-18 18:45:26 -07:00
parent fc5017134e
commit 1f0ffdbf9b
6 changed files with 240 additions and 5 deletions
--- a/.claude/memory/MEMORY.md
+++ b/.claude/memory/MEMORY.md
@@ -130,6 +130,7 @@
 - [Audio Processor Architecture](project_audio_processor_architecture.md) — Segment-first pipeline: detect breaks before transcription for complete content capture.
 - [Neptune SBR Email Routing Setup](project_neptune_sbr_email_routing.md) — Full SBR routing chain, config file locations, MailProtector integration, access methods. Treat routing breakage as systemic (devcon, Sorensen/rieussetcorp), not per-client.
 - [Dataforth Test Datasheet Pipeline](project_datasheet_pipeline.md) — Full pipeline rebuilt 2026-03-27. Server-side generation replaces DFWDS/Uploader. Website upload still broken.
+- [8B/5B sign KEEP finding](project_8b5b_sign_keep.md) — 8B/5B Final-Test KEEPS the sign on negatives (verified vs pristine Hoffman originals); the prior "drop sign" diagnosis came from contaminated (our-own-upload) oracles; legacy formatMeasured has a substring(5) sign-drop bug.
 - [Dataforth](project_dataforth.md) — M365 email (Graph API; tenant in vault at clients/dataforth/m365.sops.yaml); neptune.acghosting.com is ACG's, NOT Dataforth's. MFA enforced 2026-04-04 (3 CA policies). AJ needs dataforthgit@ forwarding.
 - [Dataforth history (2026-03-27 incident)](project_dataforth_history.md) — DF-JOEL2 compromise via ScreenConnect social-engineering, attacker C2 IPs + IC3 case + remediation log + MFA rollout origin story + Joel Lohr retirement. RESOLVED 2026-04-04.
 - [Radio show co-host — Tara, not Tom](radio_show_no_cohost_named_tom.md) — Co-host in 2014-s6e19 and 2016-s8e43 is Tara. "Tom" was hallucinated; rename complete.
--- a/.claude/memory/project_8b5b_sign_keep.md
+++ b/.claude/memory/project_8b5b_sign_keep.md
@@ -0,0 +1,40 @@
+---
+name: project_8b5b_sign_keep
+description: 8B/5B Final-Test measured values KEEP the sign; prior "drop sign" diagnosis was from contaminated oracles
+metadata:
+  type: project
+---
+
+Dataforth 8B/SCM5B datasheet Final-Test measured values **KEEP the sign** on negatives
+(e.g. `-346 uV/%`, `-0.172 %`, `-12.3 uA`). Verified 2026-06-18 against PRISTINE Hoffman
+originals (api_uploaded_at NULL — never touched by us) across all eras: 8B33-01 (2026),
+8BP01-24 (2025), SCM5B41-06D (1990), 5B39 variants (2010).
+
+So the mined-template path (`formatMeasuredExact`, keep-sign) is CORRECT. The 69 validated
+mined models are right. The legacy `formatMeasured` path in `templates/datasheet-exact.js`
+has a sign-DROPPING bug: `valueStr = statusStr.substring(5,...)` skips the sign char at
+index 4 — it should be `substring(4)`. That bug mis-renders every negative Final-Test value
+for non-mined 8B/5B models (the bulk of legacy-path production).
+
+**Why the prior session mis-diagnosed "8B/5B drops the sign":** ALL 112 mined-template
+`_srcSerial` validation oracles are OUR OWN prior uploads (zero pristine), many produced by
+the buggy legacy drop-sign renderer. The validator (`validate-mined.js`) was comparing the
+renderer against our own contaminated output — circular. The "sign FAIL" cases (e.g.
+8B34-01 render `-0.2` vs oracle `0.2`) are the renderer being CORRECT and the oracle wrong.
+
+**Status (2026-06-18, applied — UNCOMMITTED in repo working tree, live at C:\Shares\testdatadb):**
+Fixed `datasheet-exact.js`: (1) legacy `formatMeasured` keep-sign (substring 5->4);
+(2) per-model `checkListTitle` capture in 8b5bscm-templates.json (23 "Packing Check List" /
+89 "Check List") + renderer emits it; (3) 5B/8B template path uses new `formatMeasured5B8BTpl`
+= keep-sign + QB decimal CODES (code 2 -> 1dp, not literal). Repointed 9 oracles to pristine;
+10 models pristine-confirmed. Smart-classified 112 models -> 93 publishable (42 clean + 51
+sign/decimal false-fail where renderer is right and the contaminated oracle is wrong);
+19 genuine gaps held back (12 render-null/slotMap, 6 CJC blank, 1 "Output Switch" status,
+2 accuracy-ULP). Published absent-only gap fill: **created=2467 updated=0 (zero overwrites)
+errors=3 skipped=85**.
+
+**Still open:** (a) REPUBLISH the ~14% of already-published 5B/8B certs that have dropped
+signs (sampled assessment: 34/238 live certs dropped a sign) — overwrites our own wrong
+certs, needs explicit go. (b) 19 genuine gaps. (c) commit/sync (held by Mike). Tools live in
+C:\Shares\testdatadb: `_smart_validate.js`, `_revalidate_pristine.js`, `_capture_checklist.js`,
+`_assess_signdamage.js`, `_publish_8b5b_gap.js`. See [[project_pipeline_rebuilt]] [[feedback_sn_from_data]].
--- a/errorlog.md
+++ b/errorlog.md
@@ -19,6 +19,12 @@ Categories (the `[type]` tag): _(none)_ = skill/command execution failure ·

 2026-06-19 | Howard-Home | git/sync-temp-files | [friction] RECURRED (3rd time): .fleet325.dev controller-scratch swept into commit by git add -A; my earlier fix only ignored specific names (.sta/.dev/.q*), missed .fleet*. Real fix: write API scratch OUTSIDE the repo (e.g. $TMPDIR), not CWD-relative dotfiles [ctx: ref=git/sync-temp-files prior entry; recurring]

+2026-06-19 | AD2 | dataforth/publish-mined-gap | 3 of 2555 absent-only 8B/5B gap pushes failed (Dataforth API push error); created=2467 updated=0 errors=3 skipped=85 [ctx: models=93 overwrites=0]
+
+2026-06-19 | AD2 | vault | real vault.sh not found at resolved vault_path; vault read failed [ctx: path=D:/vault/scripts/vault.sh]
+
+2026-06-19 | AD2 | vault | real vault.sh not found at resolved vault_path; vault read failed [ctx: path=D:/vault/scripts/vault.sh]
+
 2026-06-18 | AD2 | dataforth/hoffman-publish | 9 of 2440 DSCA/8B gap pushes errored during 8B/5B/SCM Created-publish [ctx: created=2348 errors=9]

 2026-06-18 | GURU-5070 | agy/search | gemini CLI threw ineligible/projectId setup error (throwIneligibleOrProjectIdError), empty response after 3 attempts [ctx: mode=search host=GURU-5070]
--- a/projects/dataforth-dos/8b5bscm-templates.json
+++ b/projects/dataforth-dos/8b5bscm-templates.json
--- a/projects/dataforth-dos/datasheet-pipeline/implementation/templates/datasheet-exact.js
+++ b/projects/dataforth-dos/datasheet-pipeline/implementation/templates/datasheet-exact.js
@@ -319,7 +319,11 @@ function formatMeasured(statusStr) {

    const passFail = statusStr.substring(0, 4); // "PASS" or "FAIL"
    const decimalDigit = statusStr[statusStr.length - 1];
-    const valueStr = statusStr.substring(5, statusStr.length - 1).trim();
+    // Start at index 4 (the STR$ sign/space char) so the sign survives — the 5B/8B/DSCT
+    // Final-Test KEEPS the sign on negatives, verified against pristine Hoffman originals
+    // (e.g. "-346 uV/%", "-12.3 uA"). Was substring(5), which silently dropped the sign on
+    // every negative measured value (latent bug in the legacy DATA_LINES render path).
+    const valueStr = statusStr.substring(4, statusStr.length - 1).trim();
    const value = parseFloat(valueStr);

    if (isNaN(value)) return { passFail, formatted: valueStr, width: 6 };
@@ -365,6 +369,35 @@ function formatMeasuredExact(statusStr) {
    return { passFail, valStr, value };
 }

+/**
+ * Measured-value formatter for the 5B/8B Hoffman-mined template path. Same as
+ * formatMeasuredExact (KEEPS the sign — verified vs pristine originals) EXCEPT the trailing
+ * STATUS$ digit is a QB PRINT USING format-string CODE, not a literal decimal count:
+ * code 2 means 1 decimal (same field as code 1), unlike DSCA where code N -> toFixed(N).
+ * Confirmed by pristine originals (e.g. SCM5B35 Supply Sensitivity prints "0.3"/"-0.2",
+ * 1dp, where the literal reading would give 2dp "0.28"/"-0.20"). Codes: 0 int, 1/2 -> 1dp,
+ * 3 -> 3dp, 4 -> 4dp.
+ */
+function formatMeasured5B8BTpl(statusStr) {
+    if (!statusStr || statusStr.length <= 4) return null;
+    const passFail = statusStr.substring(0, 4);
+    const decimalDigit = statusStr[statusStr.length - 1];
+    const valueStr = statusStr.substring(4, statusStr.length - 1).trim();
+    const parsed = parseFloat(valueStr);
+    if (isNaN(parsed)) return { passFail, valStr: valueStr, value: NaN };
+    const value = Math.fround(parsed);
+    let valStr;
+    switch (decimalDigit) {
+        case '0': valStr = String(Math.round(value)); break;
+        case '1':
+        case '2': valStr = value.toFixed(1); break;
+        case '3': valStr = value.toFixed(3); break;
+        case '4': valStr = value.toFixed(4); break;
+        default:  valStr = value.toFixed(1); break;
+    }
+    return { passFail, valStr, value };
+}
+
 /**
 * Split a spec string into its value part and unit. The unit is everything after the
 * last number, so multi-word units survive ("< 17 uA RMS" -> unit "uA RMS", not "RMS";
@@ -763,10 +796,16 @@ function generateExactDatasheet(record, specs) {
        // Spec-less pass/fail rows (240VAC Withstand / Hi-Pot) render PASS; spec-less
        // section sub-heads (Zero-Crossing/TTL) render with no status.

+        // Measured-value formatter is family-specific: 5B/8B Final-Test uses QB PRINT USING
+        // decimal CODES (code 2 -> 1dp) while DSCA uses literal code->toFixed(N). Both KEEP
+        // the sign (verified vs pristine originals). 7B never reaches this branch.
+        const fmtMeas = (family === '8B' || family === 'SCM5B')
+            ? formatMeasured5B8BTpl : formatMeasuredExact;
+
        // Value-bearing measurements in source order (drop "PASS"/"" padding entries).
        const measurements = [];
        for (const s of parsed.statusEntries) {
-            const m = formatMeasuredExact(s);
+            const m = fmtMeas(s);
            if (m) measurements.push(m);
        }
        const specRowCount = tpl.rows.filter(r => (r.spec || '').trim()).length;
@@ -804,7 +843,7 @@ function generateExactDatasheet(record, specs) {
            if (spec) {
                const su = splitSpecUnit(spec);
                const m = useSlot
-                    ? formatMeasuredExact(parsed.statusEntries[tpl.slotMap[si++]])
+                    ? fmtMeas(parsed.statusEntries[tpl.slotMap[si++]])
                    : measurements[mi++];
                if (m) {
                    // measured value right-justified ending at col 38, unit at col 40.
@@ -933,7 +972,11 @@ function generateExactDatasheet(record, specs) {
        // DSCA33 originals print just the centered "Check List" header (no items).
        lines.push('                                  Check List');
    } else if (family !== 'DSCA') {
-        lines.push('                                  Check List');
+        // Check-list title varies per model: most print centered "Check List", but several
+        // 8B/5B subtypes (5B39/5B45/5B49/8B45, power modules) print "Packing Check List"
+        // (indented). Hoffman-mined templates capture the verbatim title line as
+        // `checkListTitle`; legacy (untemplated) models default to "Check List".
+        lines.push((tpl && tpl.checkListTitle) || '                                  Check List');
        lines.push('');
        lines.push(setCol(TAB5 + 'Module Appearance:      __X__', 44, 'Mounting Screw:           __X__'));
        lines.push('');
@@ -1175,5 +1218,7 @@ module.exports = {
    parseRawData,
    parse7BRawData,
    rendersWithoutSpecs,
+    formatMeasured,
+    formatMeasuredExact,
    DATA_LINES,
 };
--- a/projects/dataforth-dos/session-logs/2026-06/2026-06-18-mike-8b5b-sign-keep-fix.md
+++ b/projects/dataforth-dos/session-logs/2026-06/2026-06-18-mike-8b5b-sign-keep-fix.md
@@ -0,0 +1,143 @@
+## User
+- **User:** Mike Swanson (mike)
+- **Machine:** AD2
+- **Role:** admin
+
+## Session Summary
+
+Continued the 8B/5B/SCM datasheet grind, tasked with "keep grinding the 8B/5B sign
+formatting." The prior session had left 67/136 mined models failing, with a stated
+diagnosis that "8B/5B Final-Test drops the sign on negative measured values." Implementing
+that drop-sign direction regressed the validator net (69 -> 48), which triggered a deeper
+investigation into ground truth rather than chasing the validator count.
+
+The investigation inverted the task premise. Pulling PRISTINE Hoffman originals (serials
+with api_uploaded_at IS NULL — genuine pre-pipeline certs we never rendered) across all eras
+(8B33-01 2026, 8BP01-24 2025, SCM5B41-06D 1990, 5B39 variants 2010) showed every original
+KEEPS the sign on negatives. The keep-sign renderer (formatMeasuredExact) was already
+correct. The prior drop-sign diagnosis was an artifact of a CIRCULAR validator: all 112
+mined-template _srcSerial oracles are our own uploads (zero pristine), many produced by a
+legacy renderer that drops signs via a substring(5) bug. The validator was comparing the
+renderer against our own contaminated output.
+
+Applied four correct fixes to datasheet-exact.js (verified against pristine, not the
+contaminated oracle): (1) fixed the legacy formatMeasured sign-drop bug; (2) captured the
+per-model "Packing Check List" vs "Check List" footer title (23/89 split) and rendered it
+verbatim; (3) gave the 5B/8B template path a keep-sign + QB-decimal-code formatter
+(formatMeasured5B8BTpl: code 2 -> 1 decimal, unlike DSCA's literal toFixed(N)); (4) repointed
+9 oracles to pristine serials where the corrected renderer matched exactly. Result: 10 models
+pristine-confirmed (exact match vs true originals), independently proving keep-sign correct.
+
+Assessed already-published certs (read-only, sampled): ~14% of live 5B/8B certs have a
+dropped sign (= our wrong pushes, since pristine always keeps signs). The larger diff bucket
+is cosmetic (param-label and spec-spacing differences vs pristine old-system certs; values
+correct). Then smart-classified all 112 models into 42 clean + 51 sign/decimal false-fail
+(renderer correct, oracle wrong) = 93 publishable, holding back 19 genuine gaps (12
+render-null/slotMap, 6 blank CJC Gain, 1 "Output Switch" status, 2 accuracy-ULP). Published
+the absent-only gap fill for the 93 (plus the 24 always-valid 7B): created=2467, updated=0
+(zero overwrites), errors=3, skipped=85.
+
+## Key Decisions
+
+- Reverted the drop-sign change after pristine originals proved keep-sign is correct; did not
+  trust the validator count because the oracle set is self-referential (our own uploads).
+- Used PRISTINE originals (api_uploaded_at IS NULL) as the only trustworthy ground truth;
+  built a pristine re-validation harness instead of relying on validate-mined.
+- Kept the 5B/8B QB decimal-code mapping (code 2 -> 1dp) despite an 8B45 "0.0" vs "0.00"
+  apparent conflict — confirmed the "0.00" was our own April formatMeasuredExact output (not
+  pristine), so 1dp matches the long-standing legacy convention and SCM5B35 pristine.
+- Repointed oracles to pristine only when the corrected renderer matched exactly
+  (non-regressive); did not downgrade models to old-format pristine oracles that render null.
+- Smart-classified models so sign/decimal "failures" (renderer correct, oracle wrong) are
+  treated as publishable, while genuine gaps (render-null, CJC, status) are excluded.
+- Published absent-only (404-probed, Created) so no pristine or existing cert is overwritten.
+- Held the commit/sync of renderer+template changes per Mike's instruction to keep iterating;
+  held the republish of dropped-sign certs (overwrite) pending explicit go.
+
+## Problems Encountered
+
+- Drop-sign implementation regressed the validator (69 -> 48). Resolved by recognizing the
+  oracle contamination and validating against pristine originals instead.
+- validate-mined.js and other tools require ./database/db relative to CWD; running the repo
+  tool from tools/ failed (Cannot find module). Resolved by copying tools into the deployed
+  dir C:\Shares\testdatadb and running there.
+- Heredoc mangled backslash Windows paths in scratch scripts (C:\ProgramData...). Resolved by
+  using forward-slash paths.
+- Legacy-path full-document validation against pristine was dominated by unrelated
+  accuracy-header and date-collision diffs, masking the sign rows. Resolved by unit-testing
+  formatMeasured directly and comparing only the Final-Test region.
+- 3 of 2555 gap pushes failed with Dataforth API push errors (transient; records remain
+  api_uploaded_at NULL so a future run retries). Logged to errorlog.md.
+
+## Configuration Changes
+
+Modified (LIVE at C:\Shares\testdatadb AND repo working tree — UNCOMMITTED per Mike):
+- `projects/dataforth-dos/datasheet-pipeline/implementation/templates/datasheet-exact.js`
+  - `formatMeasured`: substring(5) -> substring(4) (keep sign); comment added
+  - added `formatMeasured5B8BTpl` (keep-sign + QB decimal codes)
+  - template Final-Test branch: family-aware `fmtMeas` selection (5B/8B vs DSCA)
+  - footer: emit per-model `tpl.checkListTitle` when present
+  - exported `formatMeasured`, `formatMeasuredExact` (for unit tests)
+- `projects/dataforth-dos/8b5bscm-templates.json`
+  - added `checkListTitle` to all 112 8B/5B models
+  - repointed `_srcSerial` + `_oraclePristine` for 9 models
+  - recomputed `validated` flags: 93 8B/5B models (42 clean + 51 sign/decimal false-fail)
+
+Memory:
+- created `.claude/memory/project_8b5b_sign_keep.md` + MEMORY.md index line
+
+Scratch tools created in C:\Shares\testdatadb (not in repo): `_validate_mined.js`,
+`_signdiag.js`, `_revalidate_pristine.js`, `_repoint_oracles.js`, `_capture_checklist.js`,
+`_smart_validate.js`, `_assess_signdamage.js`, `_assess_decexamples.js`, `_pristine_*.js`,
+`_publish_8b5b_gap.js`.
+
+## Credentials & Secrets
+
+None discovered or created. Hoffman/CloudFilter API creds read from existing
+`C:\ProgramData\dataforth-uploader\credentials.json` (CF_CLIENT_ID/SECRET/SCOPE/TOKEN_URL/
+API_BASE) — unchanged.
+
+## Infrastructure & Servers
+
+- Deployed pipeline / live working dir: `C:\Shares\testdatadb` (this box, AD2, 192.168.0.6)
+- PostgreSQL test_records DB via `C:\Shares\testdatadb\database\db.js` (pg pool)
+- Dataforth/Hoffman API: CF_API_BASE `/api/v1/TestReportDataFiles/{serial}` (GET probe,
+  POST create); OAuth client_credentials at CF_TOKEN_URL
+- Renderer entry: `database/render-datasheet.js` -> `templates/datasheet-exact.js`;
+  uploader `database/upload-to-api.js` `uploadBySerialNumbers()`
+
+## Commands & Outputs
+
+- Pristine re-validation: `node _revalidate_pristine.js [--apply]` (DSCA_VALIDATE_MODE=1)
+- Smart classify + set validated: `node _smart_validate.js --apply`
+- Footer capture: `node _capture_checklist.js --apply`
+- Sign-damage assessment: `node _assess_signdamage.js` (sampled 238: 34 dropped-sign = 14.3%)
+- Gap publish (dry): `DSCA_TPL=./8b5bscm-templates.json node _publish_8b5b_gap.js`
+  -> PROBE DONE absent=2555 present=193
+- Gap publish (push): `... _publish_8b5b_gap.js --push`
+  -> DONE {"created":2467,"updated":0,"unchanged":0,"errors":3,"skipped":85}
+- formatMeasured unit test confirmed: "PASS-.23168951"->"-0.2", "PASS-12.728490"->"-13",
+  positives unchanged.
+
+## Pending / Incomplete Tasks
+
+- REPUBLISH the ~14% dropped-sign already-published 5B/8B certs (overwrites our own wrong
+  certs; outward-facing, needs explicit go). Scope precisely first.
+- 19 genuine gaps: slotMap mining (12 render-null: 5B45-xx/5B49-xx/8B39-03/8B47K-04/8B49-02/
+  8B51-12), CJC seebeck-polynomial computation (6: SCM5B37K/KD/R/S/37K-1530), "Output Switch"
+  status (SCM5B31-1882), 2 accuracy-ULP (SCM5B32-02, SCM5B38-35).
+- 3 failed gap pushes to retry on next run.
+- Commit/sync renderer + template changes (held by Mike — currently uncommitted).
+- Verify 5B/8B decimal-code (1dp) against a pristine 8B45 original if one can be found (8B45
+  pristine certs render null on the modern template).
+
+## Reference Information
+
+- Memory: `.claude/memory/project_8b5b_sign_keep.md`
+- Prior handoffs: `projects/dataforth-dos/session-logs/2026-06/2026-06-18-mike-8b5bscm-render-handoff.md`,
+  `...-dsca-fix2-stage2-3-publish.md`
+- Templates: `C:\Shares\testdatadb\8b5bscm-templates.json` (+ repo copy
+  `projects/dataforth-dos/8b5bscm-templates.json`)
+- Renderer: `projects/dataforth-dos/datasheet-pipeline/implementation/templates/datasheet-exact.js`
+- Pristine-confirmed models (10): 8B31-1876, 8B33-01, 8B38-1935, 8B38-38, SCM5B33-02,
+  SCM5B33-02D, SCM5B33-04D, SCM5B33-05, SCM5B33-07, SCM5B35-1761