# Test-Datasheet Bug — End-to-End Trace & Diagnosis

**Date:** 2026-06-17
**Host:** AD2 (192.168.0.6) — testdatadb generator + PostgreSQL 18
**Author:** Mike Swanson / AZ Computer Guru
**Status:** DIAGNOSIS ONLY — no code or DB changes made
**Reported by:** John Lehman / Peter Iliya (Dataforth) — customer Wellbore Integrity (Joseph Swinehart) cal-cert audit on 8B35 4-wire RTD certs

---

## 0. TL;DR

There are **two independent defects, both in the datasheet *renderer*** (`templates/datasheet-exact.js`). Ingestion (.DAT parsing), the database contents, and the spec files are all **correct** — the raw test data in the DB holds the right values; the renderer mislabels and mis-maps them.

| # | Defect | Symptom on cert | Scope | Severity |
|---|--------|-----------------|-------|----------|
| **A** | RTD input column rendered as **resistance** instead of **temperature** | Header reads `Rin (ohms)`, should read `Temp. (C)`; positive input values lost their leading `+` | ~24,000 RTD certs (8B35, DSCA34, SCM5B34/35, and any RTD variant) | **HIGH** — this is the audit finding |
| **B** | **Entire DSCA Final-Test parameter list is wrong** | Wrong parameter names, garbage specs (`< 0 mA`, `+/- 0 %`), values aligned to the wrong rows, output column mislabeled (`Vout (V)` vs `Output (mA)`), lines missing/added | up to 78,343 DSCA certs (all DSCLOG) | **HIGH** |

Defect A is a small, surgical fix. Defect B requires rebuilding the DSCA template against the legacy spec (`DSCFIN.DAT`) and is a larger effort.

**Root cause is confirmed against the original ground-truth files** (the DOS-station-generated staged `.TXT`), not assumptions.

---

## 1. The Original File IS the Ground Truth — and We Found It

Mike's key point was correct: **the rendered datasheet exists as a file BEFORE it ever reaches the database.** That original file is produced by the DOS test station itself and is the source of truth. Our DB-based regeneration is what introduces the error.

### Where the original files physically live

The DOS station's QuickBASIC ATE program writes a fully-rendered `.TXT` datasheet to `C:\STAGE\` on the station, then `CTONWTXT.BAT` uploads it to the NAS `STAGE` share. It is mirrored to AD2:

```
ORIGINAL (ground truth) staged datasheets:
  AD2:  C:\Shares\test\STAGE\<TS-station>\<encoded-SN>.TXT
  NAS:  /data/test/STAGE/<TS-station>/<encoded-SN>.TXT   (\\192.168.0.9\test\STAGE)
```

Filenames use the 8.3 hex-prefix serial encoding (first two digits → letter, `55 + n`):
`179553-13` → `17`→`H` → **`H9553-13.TXT`**; `180224-7` → `18`→`I` → **`I0224-7.TXT`**.

**Confirmed files for this investigation:**

| Module | SN | Original staged file (ground truth) | Source `.DAT` |
|--------|----|--------------------------------------|---------------|
| 8B35-04 (4-wire RTD) | 179553-13 | `C:\Shares\test\STAGE\TS-4L\H9553-13.TXT` | `C:\Shares\test\TS-4L\LOGS\8BLOG\35-04.DAT` |
| DSCA38-05 (full bridge) | 180224-7 | `C:\Shares\test\STAGE\TS-11R\I0224-7.TXT` | `C:\Shares\test\TS-11R\LOGS\DSCLOG\38-05.DAT` |
| DSCA34-05C (3-wire RTD) | 180007-8 | `C:\Shares\test\STAGE\TS-4R\I0007-8.TXT` | `C:\Shares\test\TS-4R\LOGS\DSCLOG\34-05C.DAT` |

> Note re Peter: `179553-13` cannot be found on X: / DFWDS / For_Web because the **website copy is regenerated from the DB at upload time** — it is never written to disk. The on-disk ground truth is the **STAGE** copy above, not For_Web. (For_Web on AD2 holds only ~7,500 legacy files and does not contain this SN.)

---

## 2. End-to-End Pipeline (upstream emphasis)

```
[1] DOS Test Station (TS-xx, DOS 6.22, QuickBASIC ATE)
     - Runs the unit test, measures everything.
     - Writes TWO artifacts:
         (a) C:\ATE\...\<model>.DAT   raw CSV-ish multi-line test log  (-> network LOGS)
         (b) C:\STAGE\<encSN>.TXT     FULLY RENDERED datasheet  <-- GROUND TRUTH
     - The station ALREADY knows the sensor type, so it prints the correct
       input column ("Temp. (C)" for RTD) and the correct Final-Test parameter
       list for that exact model. This is the format we must reproduce.

[2] Boot upload (CTONW.BAT / CTONWTXT.BAT)
     - .DAT  -> \\NAS\test\<TS>\LOGS\<logtype>\<model>.DAT
     - .TXT  -> \\NAS\test\STAGE\<TS>\<encSN>.TXT

[3] NAS <-> AD2 sync (Sync-FromNAS.ps1, 15 min)
     - Mirrors to C:\Shares\test\... on AD2.

[4] testdatadb ingest (THIS host)  ----- the original .TXT is IGNORED here -----
     - import.js scans .DAT files (NOT the staged .TXT).
     - parsers/multiline.js parses the .DAT into a record:
         { log_type, model_number, serial_number, test_date, test_station,
           overall_result, raw_data (the verbatim .DAT block), source_file }
     - INSERT ... ON CONFLICT(serial_number) into PostgreSQL test_records.
       raw_data = the exact .DAT text block (this is faithful & correct).

[5] Render + upload (the bug lives here)
     - render-datasheet.js -> templates/datasheet-exact.js regenerates the
       datasheet text FROM raw_data + spec files, in memory.
     - upload-to-api.js POSTs {SerialNumber, Content} to Hoffman bulk API.
     - Hoffman serves it on the public product page.
```

**The pivotal architectural fact:** step [4] throws away the already-correct rendered `.TXT` and keeps only the raw `.DAT` block (`raw_data`). Step [5] then *re-renders* from scratch. Every rendering defect is introduced in step [5]; the upstream data is fine.

### 2.1 What the `.DAT` / `raw_data` actually contains (8B35-04, SN 179553-13)

```
"8B35-04        "                                              <- model
-1.461694,-1.218078E-02,-.014174,-3.986431E-02,"PASS"          <- accuracy pt 1: stim,calc,meas,err,status
151.5394,1.262828,1.26273,-1.966953E-03,"PASS"                 <- pt 2
303.6477,2.530397,2.531,1.204967E-02,"PASS"                    <- pt 3
448.7633,3.739694,3.7414,.0341177,"PASS"                       <- pt 4
598.0475,4.983729,4.9824,-2.658844E-02,"PASS"                  <- pt 5
"0","0",0                                                       <- step-response placeholder
"PASS 28.424741","PASS","PASS 252.21681","PASS","PASS"         <- final-test STATUS groups (5 per line)
"PASS","","PASS","PASS","PASS"
"PASS","PASS 3.478871E-023","PASS 3.986431E-023","PASS","PASS 26.328911"
"PASS","PASS","PASS 40.429370","PASS","PASS 140.50"
```

The **first column of each accuracy point is the stimulus**. For SN 179553-13 it is `-1.46, 151.5, 303.6, 448.8, 598.0` — clearly **temperatures in °C** (model MAXIN = 600 °C), **not** ohms. The spec record confirms `SENTYPE = "P1RTD4W"`, `MAXIN = 600`. So the DB has the right numbers and the right sensor type. Nothing upstream is wrong.

---

## 3. Diffs — Original (correct) vs DB-Generated (wrong)

### 3.1 8B35-04 RTD (SN 179553-13) — Defect A only

ACCURACY block header + first/last rows:

```
ORIGINAL  (H9553-13.TXT, ground truth)        GENERATED (current testdatadb)
-----------------------------------           ------------------------------
 Temp. (C)       Vout (V) ...                  Rin (ohms)       Vout (V) ...     <-- WRONG LABEL
    -1.46  ...                                    -1.46  ...                     (same)
  +151.54  ...                                   151.54  ...                     <-- lost '+'
  +598.05  ...                                   598.05  ...                     <-- lost '+'
```

- **Header:** `Temp. (C)` → rendered as `Rin (ohms)`. **This is the audit discrepancy.**
- **Values:** numerically identical (correct temperatures). Only difference: positive values lose the leading `+` because the resistance formatter omits the sign.
- **Final Test Results: byte-for-byte IDENTICAL** to the original (7 lines: Supply Current Nom, Exc. Current #1, Linearity, Accuracy, Supply Sensitivity, Frequency Response, Output Noise). **No missing lines for 8B35.** The 8B/5B Final-Test rendering is correct.

So for the Wellbore Integrity audit, the **only** defect on the 8B35 cert is the input column header label (and the cosmetic `+` sign). The measured data is right.

### 3.2 DSCA38-05 full-bridge (SN 180224-7) — Defect B

This is not a label tweak — the **whole Final Test table is wrong**:

```
ORIGINAL (I0224-7.TXT)                       GENERATED (current)
Supply Current          23.8 mA  < 30 mA     Supply Current, Nom        23.8 mA  <  0 mA   <- spec garbage
Supply Curr. w/ EXC Load 53.8 mA  < 80 mA    Supply Current @ Max Load  53.8 mA  <  0 mA
Excitation Voltage      10.000 V 10+/-.003V  Linearity, 50mA Load       10.000 % +/- 0 %   <- wrong name+unit
Exc. Load Regulation      -6 ppm/mA ...      Accuracy, 50mA Load             6 % +/- 0 %    <- wrong row
Output Reg. w/ EXC Load  0.00 % +/- .05 %    Positive Current Limit      0.0 mA <  0 mA
Excitation Current Limit  54 mA  < 65 mA     Negative Current Limit       54 mA >  0 mA
Linearity              0.002 % +/- .02 %     Overrange                  0.002 % >  0 %
Accuracy              -0.008 % +/- .05 %     Power Supply Sensitivity   0.008 %/% +/-.0006
Power Supply Sens.    0.0000 %/% +/-.0006    Frequency Response        0.0000 dB 25+/-5 dB <- value=0
Frequency Response      25.0 dB 25+/-5 dB    Compliance                  25.0 % +/- 0 %     <- 25.0 is freq!
Output Noise            1205 uVrms <=2000    (Output Noise line dropped entirely)
```

Also in the ACCURACY block: original column titles are `Output (V)` and separator dashes `----------`; the generator emits `Vout (V)` and `==========`. The input header happens to read `Vin (mV)` in both (bridge module), so the bridge input label is OK, but everything below it is misaligned.

### 3.3 DSCA34-05C 3-wire RTD (SN 180007-8) — Defect A **and** B together

```
ORIGINAL (I0007-8.TXT)                       GENERATED (current)
 Temp. (C)   Output (mA) ...                  Rin (ohms)   Vout (V) ...   <-- A: label + wrong out-unit
Supply Current        50.7 mA < 65 mA         Supply Current, Nom  50.7 mA  <  0 mA
Exc. Current @ -f.s. 264.0 uA   261 uA        Linearity, 0mA Load 264.0 %  +/- 0 %   <-- 264.0 is uA, not %
Exc. Current @ +f.s. 281.0 uA   278 uA        Accuracy, 0mA Load  281.0 %  +/- 0 %
Linearity           0.017 % +/-.03%           Overrange           0.017 %   >  0 %
Accuracy           -0.034 % +/-.05%           Power Supply Sens.  0.034 %/% +/-.0005
... (9 real params)                           ... (wrong names, garbage specs, lines dropped)
```

The excitation currents (264.0 uA, 281.0 uA) get printed under the labels "Linearity, 0mA Load" / "Accuracy, 0mA Load" as **percentages** — visibly nonsensical.

---

## 4. Root-Cause Localization

Tested against the original files, the defect is **(c) the renderer** — `templates/datasheet-exact.js`. Ingestion/parsing (a) and the DB data (b) are correct.

### Defect A — RTD treated as resistance

```js
// getSensorNum(): RTD maps to 7
if (s.includes('RTD')) return 7;                 // line ~150

// Accuracy input-column header (generateExactDatasheet):
} else if (sensorNum === 7) {
    inputHeader = ' Rin (ohms)';                 // line ~564  <-- WRONG for Dataforth RTD
}

// Accuracy value formatting (formatAccuracyLine):
} else if (sensorNum === 7) {
    stimStr = point.stim.toFixed(2).padStart(8);  // line ~447  <-- resistance format, no sign
}
```

Dataforth RTD modules always express the input as **temperature** on the datasheet (the RTD curve converts resistance→°C; the `.DAT` already stores °C). `sensorNum === 7` is reached **only** by RTD sentypes (`P1RTD3W`, `P1RTD4W`, `NIRTD3W`, …). There is currently **no module for which `Rin (ohms)` is correct** — true resistance/potentiometer inputs are not routed to 7 (they fall through to the voltage default). So fixing the `7` branch is safe and will not over-correct.

### Defect B — DSCA parameter list mismatch

`DATA_LINES['DSCA']` is a **single hardcoded list** (Supply Current Nom / @ Max Load / Linearity 0mA / Accuracy 0mA / Linearity 5mA / … / Compliance / Accuracy @ 5 ohm). Real DSCA modules use **different Final-Test layouts per subtype** (bridge/excitation modules list Excitation Voltage / Exc. Load Reg. / Output Reg.; RTD/TC list Exc. Current @ ±f.s.; etc.). The hardcoded list does not match, so:

- `raw_data` STATUS groups are mapped positionally onto the **wrong** parameter names;
- `buildTSpecs()` for DSCA reads spec fields (`ILIMIT`, `PERCOVER`, `COMPLIANCE`, `ACCURACY1/2/3`, `LINEAR1/2/3`) that are zero/absent for these modules → specs print as `< 0`, `+/- 0`;
- the skip rule `if (status.length <= 4) continue` drops every bare-`PASS` slot, but because the list is misaligned the *wrong* lines drop and the survivors land on wrong rows;
- the ACCURACY block also uses the 5B/8B titles (`Vout (V)` / `==========`) instead of DSCA's (`Output (V|mA)` / `----------`).

The "missing Final Test lines" complaint is a **symptom of Defect B** (DSCA misalignment + skip rule), **not** a separate bug. The 8B/5B skip rule is correct — the legacy station also prints only tested parameters (verified: 8B35 matches exactly).

> Clarification for the thread: **DSCA38 is a bridge/strain-gauge module (FBRIDGE/HBRIDGE), not an RTD.** The DSCA RTD analog is **DSCA34**. So DSCA38's problem is Defect B; DSCA34's problem is A+B. If the audit specifically concerns RTD resistance-vs-temperature, the DSCA part to verify with the customer is **DSCA34**, while DSCA38 demonstrates the broader DSCA table breakage.

---

## 5. Correct Output & Proposed Fix

### 5.1 Correct RTD output (Defect A)

Per the originals, RTD modules must render:
- **Input column header:** `Temp. (C)` (same column/format as thermocouples, `sensorNum` 3–6).
- **Input values:** the stimulus value straight from `raw_data` (already °C — **no conversion needed**), signed format (`+598.05`, `-1.46`).

Surgical change in `templates/datasheet-exact.js` (fold RTD into the temperature path):

```js
// (1) Header — replace the sensorNum===7 branch:
if ((sensorNum >= 3 && sensorNum <= 6) || sensorNum === 7) {
    inputHeader = '  Temp. (C)';
} else if (sensorNum === 2 || sensorNum === 9) {
    inputHeader = '   Iin (mA)';
} else {
    inputHeader = (maxIn != null && maxIn < 1) ? '  Vin (mV)' : '   Vin (V)';
}

// (2) Value format — in formatAccuracyLine, treat 7 like 3–6:
if ((sensorNum >= 3 && sensorNum <= 6) || sensorNum === 7) {
    stimStr = formatSigned(point.stim, 2, 8);   // temperature, signed
} else { ... }
```

Leave the existing `i===13 && sensorNum===7 → 'ohm/ohm'` unit override (Lead-R-Effect) as-is; it is a separate, correct detail. **Verify exact leading-space alignment** against a known-good thermocouple original before pushing (the header column should byte-match; this is the same polish already in progress in `generated-v2-*.TXT`).

This fix corrects **header + values** for all RTD modules (8B35, DSCA34, SCM5B34/35, etc.) at once. **Do not** add any resistance→temperature math — the data is already temperature.

### 5.2 DSCA template (Defect B)

Not a one-liner. The fix is to drive the DSCA Final-Test parameter list (and ACCURACY column titles/units) **per module subtype**, matching the legacy QuickBASIC DSC writer. The legacy parameter selection lives in **`specdata\DSCFIN.DAT`** (DSC final-test definitions) alongside `DSCMAIN4.DAT`/`DSCOUT.DAT`. Recommended approach:
1. Reverse the DSCFIN.DAT layout (or read the QB DSC datasheet source) to get the per-subtype parameter name/unit/spec list.
2. Replace the single `DATA_LINES['DSCA']` + DSCA branch of `buildTSpecs()` with subtype-aware selection keyed on SENTYPE / output-signal type.
3. Fix the DSCA ACCURACY block to use `Output (V|mA)` (per `OUTSIGTYPE`) and dash separators.
4. Validate byte-for-byte against staged originals across DSCA subtypes (bridge, RTD, TC, current-out, voltage-out).

Until B is fixed, **all DSCA (DSCLOG) website datasheets should be treated as unreliable** for Final-Test content.

---

## 6. Impact (records currently on the website)

| Group | On-web count | Defect |
|-------|-------------:|--------|
| 8B35* | 5,476 | A |
| DSCA34* (RTD) | 3,573 | A + B |
| SCM5B34/35* (RTD) | 14,887 | A |
| All DSCA (DSCLOG) | 78,343 | B (RTD subset also A) |
| **Total on website** | 464,671 | — |

RTD-label exposure (Defect A) ≈ **24,000 certs**; DSCA table exposure (Defect B) ≈ **78,000 certs**.

After fixes are reviewed and deployed, affected records can be re-pushed by clearing `api_uploaded_at` for the affected models and letting the upload path re-render (RE-PUSH is idempotent; Hoffman returns `Unchanged` when content matches).

---

## 7. How to Reproduce / Verify (read-only)

```powershell
# Render current generator output for a SN and compare to the staged original:
cd C:\Shares\testdatadb
node -e "const db=require('./database/db');const {renderContent}=require('./database/render-datasheet');(async()=>{const r=await db.queryOne('SELECT * FROM test_records WHERE serial_number=$1',['179553-13']);if(r.test_date&&r.test_date.toISOString)r.test_date=r.test_date.toISOString().slice(0,10);console.log(renderContent(r));await db.close();})()"

# Ground truth:
type C:\Shares\test\STAGE\TS-4L\H9553-13.TXT
```

---

## 8. Files Referenced

- Generator: `C:\Shares\testdatadb\templates\datasheet-exact.js` (repo: `projects/dataforth-dos/datasheet-pipeline/implementation/templates/datasheet-exact.js`)
- Render glue: `C:\Shares\testdatadb\database\render-datasheet.js`
- Specs: `C:\Shares\testdatadb\parsers\spec-reader.js`, `C:\Shares\testdatadb\specdata\*.DAT` (incl. `DSCFIN.DAT`)
- Ingest: `C:\Shares\testdatadb\database\import.js`, `C:\Shares\testdatadb\parsers\multiline.js`
- Ground-truth originals: `C:\Shares\test\STAGE\<TS>\<encSN>.TXT`