Files
claudetools/projects/dataforth-dos/EMAIL-TO-JOHN-datasheet-findings-2026-06-17.md
Mike Swanson e30814c2f2 dataforth(datasheet): save email draft to John summarizing all findings
Plain-language summary for John (EE, non-developer) covering all 4 problems
(RTD label, DSCA table, same-day run, missing units), framed as ACG-owned fixes.
Cross-links the four supporting technical docs. Draft - not yet sent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 13:02:33 -07:00

71 lines
6.7 KiB
Markdown

# Email draft — to John Lehman, 2026-06-17
**To:** John Lehman <jlehman@dataforth.com>
**From:** Mike Swanson <mike@azcomputerguru.com>
**Subject:** Test Datasheet Problems — What's Going On and How We're Fixing It
**Status:** DRAFT (not yet sent)
---
John,
I dug all the way into the datasheet issues you and Peter flagged (the 8B35 RTD certs the Wellbore Integrity audit caught), and I found the original problem plus a few related things worth knowing about. Good news up front: **your test data and original datasheets are accurate — nothing is corrupting your measurements.** The problems are all on our side, in the system that loads the data and re-creates the datasheets for the website, and **we'll be making the fixes** — that piece is our responsibility.
**First, how the system works (so the rest makes sense)**
When a unit is tested, the DOS test station produces **two** things:
1. The **datasheet** you're used to seeing — a text file the station writes at test time. This is the original, and it's correct.
2. A **raw data log** (the `.DAT` file) — the underlying numbers.
The website doesn't store that original datasheet. Our system loads the raw log into a database and then **rebuilds** the datasheet from the database when it's needed. That rebuild step is where the errors are. So the unit you tested is fine and the original sheet is fine — the *regenerated* copy on the website is what's off.
**Problem 1 — RTD modules say "resistance" when they should say "temperature" (this is the audit finding)**
An RTD senses temperature by changing its resistance, but on a Dataforth datasheet the input column is shown as **Temperature (°C)** — exactly what your original sheets do. Our rebuild has a bug: for RTD modules it labels that column **"Rin (ohms)"** even though it's printing the temperature numbers underneath, so the header and the values disagree. That's precisely what the auditor caught on the 8B35.
Important: on the 8B35 cert the **measured values are correct** — it's the column label that's wrong (and the positive numbers lost their leading "+"). This affects every RTD product, not just the 8B35 — roughly 24,000 certificates (8B35, DSCA34, the SCM5B34/35 family). We've already written and tested the fix against your original sheets; it produces the correct "Temp. (°C)" column with no effect on any non-RTD product.
**Problem 2 — DSCA datasheets have the wrong Final Test lines (the "missing lines / wrong headers")**
For the DSCA line, our rebuild is using the **wrong master list of test parameters** — so you get wrong parameter names, wrong spec limits, values on the wrong rows, and some lines dropping off. This is the "Final Test lines are missing" part of your report, and it affects the DSCA line broadly (up to ~78,000 certs). One clarification: DSCA38 is a bridge/strain-gauge module — its issue is this one; the DSCA *RTD* module is DSCA34, which has both this and Problem 1. This one's a real rebuild on our end (matching each DSCA sub-type to its spec file), so it'll take longer than the others.
**Problem 3 — some units show an earlier same-day test instead of the final one**
If a unit was tested more than once on the **same day** (a re-trim, say), our system can keep an *earlier* run instead of the final accepted one, so the website cert may show non-final numbers. It affects about **311 units** across your history. It matters for audits because the **last** same-day run is the one that belongs on the cert. We'll change our database rule so the latest same-day run wins.
**Problem 4 — two batches of units never made it into the system**
I also found units that were tested and have an original datasheet but **no** record in the database/website. Two separate reasons:
- **A serial-number format our importer can't read (~9,500 records).** In the DOS days, when a serial was too long for the old 8-character filename limit, the software shortened it with a letter code — e.g. `10243-1` is written as `A243-1`. Our importer only recognizes serials that start with a number, so it **silently skips** every letter-coded unit. The data is sitting right there in the logs; we just aren't reading it. Across all your logs that's about **840 units / 9,500 records / 141 models** never imported. We'll update the importer to recognize and decode those serials, which recovers the whole backlog.
- **A one-time casualty of the cryptolocker incident (~379 units).** This is *not* an ongoing gap — we import every 15 minutes and everything since has come through cleanly. Here's what happened: the DOS stations append all of a given model's results into a single, shared log file. During the incident, stations were failing to sync, and in the recovery we didn't yet realize they were appending to one filename — so for a couple of weeks, fresh logs coming off the DOS machines overwrote the accumulated server-side logs, wiping the earlier history those files held. The data backs this up: the affected units are confined to **October 2025 through January 2026** (most in December 2025) and **stop cold after January**, on three stations (TS-4L, TS-4R, TS-1R) — the exact fingerprint of a one-time overwrite during the incident, not a recurring problem. The good news: the **original datasheet text files for these units still exist** on the server, so we can backfill them from those.
**The bottom line**
- Your **test data and original datasheets are intact and accurate** — I verified the database matches the originals to the number across ~11,000 units, with zero cases of data being read in wrong.
- Everything above is in **our** load-and-rebuild software, or was a one-time incident casualty — not a measurement or test problem.
- For the **audit specifically**: the 8B35 measured values are right; it's the column heading, and that fix is ready to go.
**What we're doing next**
We'll tackle them in this order unless you'd rather reprioritize:
1. RTD column-label fix — clears the audit issue (ready now).
2. Missing-units importer fix — recovers the ~9,500-record backlog.
3. Same-day-run rule — so certs show the final run.
4. DSCA Final Test rebuild — the larger one.
5. Backfill the ~379 incident-era units from their original datasheet files.
The only thing I need from you is a thumbs-up on that priority order (and on backfilling the 379 from their original sheets). Everything else is on us.
I've documented all of this in detail and am glad to walk your team through any of it on a call.
Thanks,
Mike Swanson
AZ Computer Guru
(520) 304-8300
---
*Supporting detail: `DATASHEET-RTD-BUG-DIAGNOSIS-2026-06-17.md` (Problems 1 & 2), `PARSING-FIDELITY-VERDICT-2026-06-17.md` + `CONFLICT-RULE-FIX-PROPOSAL-2026-06-17.md` (Problem 3), `MISSING-UNITS-REPORT-FOR-JOHN-2026-06-17.md` (Problem 4).*