claudetools/projects/dataforth-dos/session-logs/2026-06/2026-06-17-mike-datasheet-rtd-diagnosis.md

2026-06-17 — Dataforth datasheet RTD bug diagnosis + AD2 harness onboarding

User

• User: Mike Swanson (mike)
• Machine: AD2
• Role: admin

Session Summary

Diagnosed a Dataforth test-datasheet defect reported 2026-06-17 by John Lehman / Peter Iliya, triggered by a Wellbore Integrity (Joseph Swinehart) cal-cert audit on 8B35 4-wire RTD certificates. The complaint: column headers wrong and some Final Test lines missing. Worked on AD2 (the testdatadb generator + PostgreSQL host), tracing the pipeline upstream from the website copy to the original ground-truth file. Diagnosis only — no changes to the generator or DB.

Confirmed two independent defects, both in the datasheet renderer templates/datasheet-exact.js (ingestion, DB data, and spec files are all correct). Defect A: RTD modules render the input column as resistance (Rin (ohms)) when the original/ground-truth file shows temperature (Temp. (C)); the stimulus values in raw_data are already temperatures (°C), so the numbers are right but the label is wrong and positive values lose their leading +. Defect B: the entire DSCA Final-Test parameter list is hardcoded as a single layout that does not match real DSCA module subtypes, producing wrong parameter names, garbage specs (< 0 mA, +/- 0 %), values mapped to the wrong rows, a mislabeled output column, and dropped lines. The "missing Final Test lines" complaint is a symptom of Defect B, not a separate bug.

Located the ground truth: the DOS test station writes a fully-rendered .TXT datasheet to C:\STAGE\ before ingestion; it is mirrored to AD2 at C:\Shares\test\STAGE\<TS>\<encoded-SN>.TXT. Verified three examples against their originals — 8B35-04 (SN 179553-13, H9553-13.TXT), DSCA38-05 (SN 180224-7, I0224-7.TXT, a bridge module — NOT RTD), and DSCA34-05C (SN 180007-8, I0007-8.TXT, the actual DSCA RTD analog). Clarified for the thread that DSCA38 is a bridge/strain-gauge module, so its issue is Defect B; the DSCA RTD module Peter likely means is DSCA34, which has both A and B. Wrote the full diagnosis to projects/dataforth-dos/DATASHEET-RTD-BUG-DIAGNOSIS-2026-06-17.md (process doc + diagnosis + proposed fix).

The second half of the session was unplanned infrastructure work needed to /sync the results: AD2 had no Python (so sync.sh could not parse identity.json), no identity.json at all, and an unset git author identity. Installed Python 3.12.8, created a proper identity.json via the documented hand-create + migrate-identity.sh flow, and set git authorship. Then synced — which surfaced that ad2 was 3 months behind main and that sync.sh cannot overwrite itself mid-run on Windows. Recovered cleanly, rebased ad2 onto main (modernizing the fork to harness v1.4.3), relocated the Dataforth context out of the shared .claude/CLAUDE.md into clients/dataforth/CLAUDE.dataforth.md so future syncs stay conflict-free, and force-pushed.

Key Decisions

• Diagnose only, verify against the original file, not assumptions. Pulled the staged DOS-generated .TXT as ground truth rather than trusting the code reading. This confirmed the RTD stimulus values are temperatures and the label is the defect.
• Classified the bug as renderer-only. Ingestion (multiline.js), DB raw_data, and spec SENTYPE are all correct; the fix belongs in datasheet-exact.js. Defect A is surgical (fold RTD/sensorNum 7 into the temperature path); Defect B (DSCA) needs a per-subtype rebuild driven by DSCFIN.DAT and is larger.
• Installed Python from the official installer, not Chocolatey. Choco 2.6.0 requires .NET 4.8 (Server 2019 ships 4.7.2); the python.org silent installer is self-contained and avoids a .NET install + reboot.
• Created identity.json via hand-create + migrate-identity.sh (the documented flow) rather than inventing a generator. Used Mike's identity (strongly signaled by the gitea remote, branch, and user profile memory).
• Did not blind-commit the 205 MB WizTree zip until authorized. sync.sh does git add -A; flagged the binary first. User confirmed AD2 is a Dataforth-ops fork and to sync everything.
• Modernized the ad2 fork by rebasing onto main and relocating the Dataforth doc. Chose "take main's lean CLAUDE.md + move Dataforth context to a fork-specific file" so the fork stays additive and future rebases are conflict-free. The redundant CLAUDE.md-edit commit became empty and was auto-dropped.

Problems Encountered

• sync.sh aborted: "No Python interpreter found." AD2 had no Python at all. Installed Python 3.12.8 (official amd64 installer, all-users, PATH + py launcher); sync.sh now detects py.
• identity.json missing → commits would attribute as unknown and vault/coord fields were absent. Hand-created the core file and ran migrate-identity.sh to fill python/ollama/platform/arch/grok/coord_api. Validated all 13 required identity fields present; confirmed it is gitignored.
• Rebase failed mid-run: "unable to create file .claude/scripts/sync.sh: Permission denied." sync.sh was the executing script while git's checkout-to-main tried to overwrite it (Windows file lock). The partial checkout left the working tree half-converted to origin/main (modified CLAUDE.md, deleted sync.sh, 240 untracked origin/main files). Recovered with git reset --hard HEAD + git clean -fd, then ran git rebase origin/main directly (outside sync.sh) — succeeded with only the expected .claude/CLAUDE.md conflict.
• CLAUDE.md rebase conflict (Dataforth fork doc vs main's lean refactor). Resolved by taking main's version (git checkout origin/main -- .claude/CLAUDE.md); Dataforth content was preserved beforehand into clients/dataforth/CLAUDE.dataforth.md.
• NAS SSH host-key changed / SMB denied. Could not reach \\192.168.0.9\test\STAGE over SSH (host key changed) or SMB (permission denied). Worked around via the rsync daemon (module test, user rsync) for listing, and found the staged originals already mirrored locally at C:\Shares\test\STAGE\.

Configuration Changes

Created:

• C:\ClaudeTools\.claude\identity.json (gitignored, per-machine) — full identity for AD2
• C:\ClaudeTools\projects\dataforth-dos\DATASHEET-RTD-BUG-DIAGNOSIS-2026-06-17.md — diagnosis deliverable
• C:\ClaudeTools\clients\dataforth\CLAUDE.dataforth.md — relocated Dataforth ops context
• C:\ClaudeTools\.claude\memory\project_ad2_dataforth_fork.md (+ MEMORY.md index line)
• Session log (this file)

Modified:

• .claude/CLAUDE.md — now the lean fleet doc (was the Dataforth fork doc); rebased from main
• Git author identity on AD2: user.name="Mike Swanson", user.email="mike@azcomputerguru.com"
• Whole harness advanced to main (v1.4.3): new CLAUDE_EXTENDED.md, harness/, bootstrap/, skills/commands

Installed:

• Python 3.12.8 (64-bit, all-users) → C:\Program Files\Python312\, py launcher at C:\Windows\py

NOT changed (by design — diagnosis only): templates/datasheet-exact.js, the PostgreSQL DB, any datasheet content.

Credentials & Secrets

No new credentials created or discovered this session. Used existing documented access:

• NAS rsync daemon: rsync://rsync@192.168.0.9/test (module test = /data/test), password IQ203s32119 (already in the Dataforth context doc).
• PostgreSQL (local, AD2): default app creds from database/db.js — testdatadb_app / DfTestDB2026! on localhost:5432/testdatadb.

(AD2 has no SOPS vault cloned; these remain documented in clients/dataforth/CLAUDE.dataforth.md.)

Infrastructure & Servers

• AD2 192.168.0.6 — testdatadb (Node/Express :3000) + PostgreSQL 18; this host. Hostname AD2.
• AD1 192.168.0.27 — \\AD1\Engineering.
• D2TESTNAS 192.168.0.9 — SMB1 bridge; rsync daemon port 873 module test; SSH host key changed this session (publickey/password denied).
• testdatadb data: 464,671 records on website; log_types — 5BLOG 196,502 / 7BLOG 121,304 / DSCLOG 79,868 / 8BLOG 63,808 / others.
• Impact of Defect A (RTD label): 8B35 5,476 + DSCA34 3,573 + SCM5B34/35 14,887 ≈ 24K certs. Defect B (DSCA template): up to 78,343 DSCLOG certs.

Commands & Outputs

# Render current generator output vs staged original (read-only diagnosis)
cd C:\Shares\testdatadb
node -e "const db=require('./database/db');const {renderContent}=require('./database/render-datasheet');(async()=>{const r=await db.queryOne('SELECT * FROM test_records WHERE serial_number=$1',['179553-13']);if(r.test_date&&r.test_date.toISOString)r.test_date=r.test_date.toISOString().slice(0,10);console.log(renderContent(r));await db.close();})()"
type C:\Shares\test\STAGE\TS-4L\H9553-13.TXT   # ground truth: header = " Temp. (C)"

# Python install (PowerShell)
Start-Process python-3.12.8-amd64.exe -ArgumentList '/quiet','InstallAllUsers=1','PrependPath=1','Include_launcher=1' -Wait   # exit 0

# Onboarding identity
#   hand-create .claude/identity.json (core fields) then:
bash .claude/scripts/migrate-identity.sh   # filled python/ollama/platform/arch/grok/coord_api

# Rebase recovery (sync.sh self-lock)
git reset --hard HEAD && git clean -fd
git rebase origin/main
git checkout origin/main -- .claude/CLAUDE.md && git add .claude/CLAUDE.md
GIT_EDITOR=true git rebase --continue
git push --force-with-lease origin ad2

Key error + resolution: error: unable to create file .claude/scripts/sync.sh: Permission denied → run the rebase directly (not via the executing sync.sh).

Pending / Incomplete Tasks

• Fix Defect A (RTD label/values) in templates/datasheet-exact.js: in the input-header logic and formatAccuracyLine, route sensorNum === 7 (RTD) through the temperature path (' Temp. (C)' + formatSigned(stim, 2, 8)). Verify leading-space alignment against a thermocouple original. Safe — 7 is reached only by RTD sentypes; no module currently needs Rin (ohms). Awaiting review before changing the generator.
• Fix Defect B (DSCA template): rebuild the DSCA Final-Test parameter list + ACCURACY column titles/units per module subtype, driven by specdata\DSCFIN.DAT / the legacy QB DSC writer. Larger effort. Until done, treat all DSCA (DSCLOG) website datasheets as unreliable.
• After fixes: re-push affected models by clearing api_uploaded_at (RE-PUSH is idempotent).
• AD2 tooling gaps: jq, sops, age not installed; no vault cloned (D:/vault absent) → vault sync N/A. coord_api (172.16.3.30) unreachable from Dataforth LAN.
• sync.sh self-lock: upstream fix candidate (re-exec from a temp copy, or rebase before the script can be overwritten) — offered to file as a CT thought.

Reference Information

• Diagnosis doc: projects/dataforth-dos/DATASHEET-RTD-BUG-DIAGNOSIS-2026-06-17.md
• Generator: C:\Shares\testdatadb\templates\datasheet-exact.js (repo: projects/dataforth-dos/datasheet-pipeline/implementation/templates/datasheet-exact.js)
• Ground-truth originals: C:\Shares\test\STAGE\<TS>\<encoded-SN>.TXT (8.3 hex-prefix SN encoding: first 2 digits → letter, 55+n)
• Examples: 8B35-04 SN 179553-13 (H9553-13.TXT, P1RTD4W, MAXIN 600); DSCA38-05 SN 180224-7 (I0224-7.TXT, FBRIDGE); DSCA34-05C SN 180007-8 (I0007-8.TXT, P1RTD3W)
• Task prompt: Prompt617.txt
• Commits this session (ad2): 49c9eb50 (work sync), bbb19db2 (relocate Dataforth doc), c4de16f6 (memory)
• Contacts: John Lehman jlehman@dataforth.com, Peter Iliya pIliya@dataforth.com