Compare commits

...

5 Commits

Author SHA1 Message Date
c4de16f660 memory: record AD2 Dataforth-fork structure + sync gotchas
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 13:48:31 -07:00
bbb19db2f6 dataforth: preserve fork operational context as clients/dataforth/CLAUDE.dataforth.md
Relocated verbatim from .claude/CLAUDE.md so the ad2 fork stops editing the
shared fleet harness doc. After rebase onto main, .claude/CLAUDE.md = the lean
fleet version; Dataforth ops context lives here. Keeps future ad2 syncs clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 13:45:36 -07:00
49c9eb50aa sync: auto-sync from AD2 at 2026-06-17 13:35:55
Author: Mike Swanson
Machine: AD2
Timestamp: 2026-06-17 13:35:55
2026-06-17 13:45:32 -07:00
sysadmin
53ba8e5f4e Add AD scripts and stage import instructions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-06-17 13:45:28 -07:00
sysadmin
f8d9e3e3ec Session log 2026-04-03: WO import, 7B support, PG migration started
- 33K work orders imported, 2.27M records linked
- 7B exact-match formatter added (31 params, 120VAC, Packing Check List)
- TXT formatting refined to match QB TAB positions exactly
- PostgreSQL 18 installed on AD2, database created
- SQL Server Express uninstalled
- Full Dataforth audit document generated

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-06-17 13:45:28 -07:00
17 changed files with 1704 additions and 0 deletions

View File

@@ -156,3 +156,4 @@
- [CT Thoughts backlog](feedback_ct_thoughts_backlog.md) — ClaudeTools harness ideas go in docs/CT_THOUGHTS.md (trigger "ct thought:"); CT analogue of RMM_THOUGHTS. Don't build until explicit go. First entry = ClaudeTools 3.0 web co-work vision.
- [AI-auth product boundary](project_ai_auth_product_boundary.md) — ClaudeTools/ClaudeTools 3.0 = internal-only, per-person subscription OAuth ok; GuruRMM = sellable, customer brings own API key (never ACG's subscription); backend dev = internal. Anthropic ToS bans subscription auth in third-party products.
- [RMM SYSTEM context can't see user mapped drives](feedback_rmm_system_context_mapped_drives.md) — RMM runs as SYSTEM; `Test-Path F:\` etc. is False even when the user's mapped/redirected drive exists. Diagnose mapped-drive/redirect issues in `context:user_session`. Elevated apps (e.g. QB DB Server Manager "unable to retrieve root folder") need `EnableLinkedConnections=1` + reboot.
- [AD2 = Dataforth-ops fork](project_ad2_dataforth_fork.md) — branch ad2 = main + thin Dataforth layer; keep fork edits ADDITIVE (Dataforth context in clients/dataforth/CLAUDE.dataforth.md, NOT .claude/CLAUDE.md); rebase onto main directly when sync.sh self-lock hits; no vault/jq/sops/age on this box.

View File

@@ -0,0 +1,17 @@
---
name: project_ad2_dataforth_fork
description: AD2 is the Dataforth-operations fork (branch ad2); how it relates to main and how to sync it cleanly
metadata:
type: project
---
AD2 (192.168.0.6, Dataforth domain controller) runs the **`ad2` branch** — a fork dedicated to Dataforth operations. Its `/sync` model is: `git rebase origin/main` then `git push origin ad2`, so `ad2` = `main` + a thin layer of Dataforth-specific commits.
**Keep the fork layer ADDITIVE** — never edit shared fleet files (especially `.claude/CLAUDE.md`) on `ad2`, or every sync re-conflicts. On 2026-06-17 the old fork edit of `.claude/CLAUDE.md` was relocated: `.claude/CLAUDE.md` now tracks the lean fleet doc (from main), and Dataforth operational context (network, credentials, pipeline, batch-file versions) lives in **`clients/dataforth/CLAUDE.dataforth.md`**. Read that file for Dataforth context.
**Sync gotchas on AD2:**
- `sync.sh` cannot overwrite itself while it is the executing script (Windows file lock) → the rebase step fails mid-run with "unable to create file .claude/scripts/sync.sh: Permission denied". When that happens: `git reset --hard HEAD`, `git clean -fd` (removes the partial origin/main checkout debris), then run `git rebase origin/main` **directly** (not via sync.sh), and force-push with `--force-with-lease`.
- Python was absent until 2026-06-17 (installed 3.12.8, `py` launcher); identity.json was created then too (`user=mike`, `machine=AD2`).
- Missing tooling: `jq`, `sops`, `age`. No vault cloned (`D:/vault` absent) → vault sync is N/A here. coord_api (172.16.3.30) is unreachable from the Dataforth LAN.
Related: [[project_ad2_context.md]]

65
Prompt617.txt Normal file
View File

@@ -0,0 +1,65 @@
TASK: Diagnose + fully document the Dataforth test-datasheet bug. You are running ON AD2
(192.168.0.6), the host of the testdatadb generator + PostgreSQL. Do NOT fix anything yet <20>
trace, document, and pinpoint root cause.
=== THE PROBLEM (reported today, 2026-06-17) ===
John Lehman (jlehman@dataforth.com) to Mike: "We are experiencing problems with test
datasheets. Column headers are wrong. Some Final Test lines are missing."
From the thread (apps engineer Peter Iliya):
- For the 8B35: the STAGED datasheet "H9553-13" (T: drive, TS-4L/STAGE) has the CORRECT
input column label "Temp" <20> but the WEBSITE version "179553-13" shows the WRONG label
"resistance". DSCA38 is flagged with the same issue.
- Peter CANNOT find 179553-13 as a file on the X: drive or in the DFWDS report <20> because the
website version is generated from the DB, not stored as a file.
- Core question (Peter): "Is there a reason we list the input for our RTD modules as
resistance, but show the temperature instead?"
- Trigger: customer Wellbore Integrity (Joseph Swinehart) raised a cal-cert AUDIT discrepancy
on 8B35 4-wire RTD certs.
=== THE KEY POINT (Mike) ===
The rendered test sheet EXISTS AS A FILE BEFORE it is ingested into the database. That
original/staged file is the ground truth (correct Temp label). Our DB-based regeneration is
what introduces the error. FOLLOW THE TRAIL UPSTREAM and FIND WHERE THE ORIGINAL FILE IS.
=== ARCHITECTURE (per TEST-DATASHEET-PROCESS.md) ===
Test stations (TS-01..27, DOS) write .dat logs
-> AD1 HISTLOGS: \\ad1\... C:\Shares\test\Ate\HISTLOGS\{log_type}\{model}.DAT (+ per-station \TS-XX\LOGS\)
-> Legacy DFWDS (VB6) historically rendered "For_Web" .TXT datasheets (the ORIGINAL rendered files)
-> testdatadb (THIS host AD2): Node.js + Express + PostgreSQL 18, service on :3000, dashboard
http://localhost:3000/, WinSW wrapper, service account INTRANET\svc_testdatadb
-> Hoffman Product API (/api/v1/TestReportDataFiles/bulk, OAuth2) -> public website.
=== WHAT WE ALREADY FOUND IN THE CODE (verify the DEPLOYED copy matches) ===
Generator template: datasheet-exact.js (find the deployed copy on AD2; repo copy is
projects/dataforth-dos/datasheet-pipeline/implementation/templates/datasheet-exact.js).
- getSensorNum(): RTD -> 7. Input header logic: sensorNum 3-6 (thermocouples) -> " Temp. (C)";
sensorNum 7 (RTD) -> " Rin (ohms)"; so RTD is rendered as RESISTANCE. The accuracy VALUE
formatting is ALSO sensorNum-driven (temperature format vs resistance format) - so values may
be wrong too, not just the label.
- Final Test loop: `for (i < dataLines.length && i < parsed.statusEntries.length)` with
`if (status.length <= 4) continue` -> rows DROP when statusEntries are misaligned/short.
=== YOUR DELIVERABLES ===
1. FULLY DOCUMENT THE END-TO-END PROCESS (write it to a markdown file), especially the UPSTREAM
half: exactly what the test station emits, what the ORIGINAL rendered datasheet file is,
WHERE it physically lives (find TS-4L/STAGE, the staged H9553-13, and/or the DFWDS For_Web
.TXT), and exactly how testdatadb ingests it (parser, which fields, the .DAT format).
2. LOCATE THE ORIGINAL FILE for the 8B35 example (H9553-13 / its For_Web .TXT / the .dat it came
from) and for a DSCA38 example.
3. DIFF original (correct) vs DB-generated (wrong) for 8B35 RTD and DSCA38: the exact column
header difference, any VALUE differences, and which Final Test lines are missing and WHY.
4. ROOT-CAUSE LOCALIZATION: is the defect in (a) ingestion/.DAT parsing, (b) the DB data itself,
or (c) datasheet-exact.js rendering? Confirm against the original file, not assumptions.
5. State the CORRECT RTD output (label = Temp (C); values = temperature, derived how?) per the
original, and the precise code change(s) that would fix both the header/values and the
dropped Final Test lines. Note any modules where resistance IS the correct label (don't
over-correct).
Resources you have locally: PostgreSQL on AD2 (query testdatadb directly), the deployed Node
service + its code, SMB to AD1 (\\ad1\...) and D2TESTNAS (\\192.168.0.9\test, /data/test has
DFWDS, Ate, 8BDATA, DSCDATA, etc.). The ClaudeTools repo has the pipeline source under
projects/dataforth-dos/.
Output: the process doc (file) + a diagnosis summary + the proposed fix. Diagnose only <20> no
changes to the generator or DB until reviewed.

View File

@@ -0,0 +1,68 @@
# Stage TXT Import Task
# Date: 2026-03-28
# Context: CTONWTXT.BAT now uploads C:\STAGE\*.TXT from DOS machines to T:\STAGE\%MACHINE%\
## What happened
1. CTONWTXT.BAT was never being called -- fixed, now called from CTONW.BAT on every boot
2. Destination changed from broken X: (Novell serve.sys check) to T:\STAGE\%MACHINE%\
3. DOS 6.22 can't MD on existing dirs without error, so dirs are pre-created on NAS
4. All TS-* machine folders pre-created under /data/test/STAGE/ on D2TESTNAS
## What needs to run
Save the script below as C:\Shares\testdatadb\import-all-stage.js and run it:
cd C:\Shares\testdatadb
node import-all-stage.js
## What it does
- Scans \\D2TESTNAS\test\STAGE\TS-*\*.TXT (~8,100 files across 10 machines)
- Parses each TXT datasheet (Date, Model, SN)
- Decodes hex-prefix serial numbers for 8.3 filename encoding:
- Letter prefix = hex digit: A=10, B=11, C=12, ..., H=17, etc.
- Example: H8236-12.TXT has SN: 178236-12 inside the file
- Example: A819-1.TXT has SN: A819-1 inside -> decoded to 10819-1
- The SN line inside H-prefix files already has the full numeric serial
- The SN line inside A-prefix files still has the encoded serial
- Cross-references against testdata.db by (serial_number, model_number)
- Inserts MISSING records as log_type='SHT' with test_station from folder name
- Copies ALL files to X:\For_Web\{decoded_serial}.TXT (the web share)
## Machines with data
TS-4L: 3,082 files (largest)
TS-4R: 2,741 files
TS-1R: 509 files
TS-8R: 478 files
TS-3R: 435 files
TS-11R: 325 files
TS-8L: 285 files
TS-11L: 248 files
TS-27: 10 files (already imported this session)
TS-1L: 1 file
## Serial number encoding (8.3 filename scheme)
The QuickBASIC ATE software encodes long serial numbers to fit DOS 8.3 filenames.
The first two digits get replaced with a hex letter if the serial is too long:
178236-12 -> H8236-12.TXT (17 -> H, which is char code 72, 72-55=17)
10819-1 -> A819-1.TXT (10 -> A, which is char code 65, 65-55=10)
Decode: letter.charCodeAt(0) - 55 = numeric prefix
Only applies if filename starts with [A-Z] followed by digits.
## TS-27 already done
10 files from TS-27 were already imported earlier this session into the DB as SHT records.
The import script uses INSERT OR REPLACE so re-running is safe.
## Previous CTONWTXT.BAT issues (resolved)
- v1.0: Never called, checked for Novell serve.sys, used X: drive parameter
- v2.0: Called from CTONW, but used mixed-case "Stage" path -> failed on DOS
- v2.1: All uppercase STAGE, but had MD commands that fail on existing dirs
- v2.2: Same issue
- v2.3: Removed MD entirely, dirs pre-created on NAS. CURRENT VERSION.

View File

@@ -0,0 +1,80 @@
Subject: Test Datasheets - Weekend Update: All 73 Quatronix Sheets Generated, Work Order Search Live
John, Ken,
Quick update on progress since Friday's email. The pipeline is significantly further along.
## Quatronix Customer Issue - RESOLVED
All 73 requested datasheets have been generated (TXT + PDF). The last holdout was SCM5B49-05 (SN 177000-15) — the 5B49DATA.DAT spec file was empty, but John pointed us to 5B49_2.DAT which had the data. All 73 files are ready to send to Peter/Ginger.
## Model Spec Coverage Expanded
We went from 751 model specs to 1,470+ by loading additional spec databases:
| Spec File | Family | Models |
|-----------|--------|--------|
| 5BMAIN.DAT | SCM5B | 481 |
| 5B45DATA.DAT | SCM5B (freq/counter) | 56 |
| DB5B48.DAT | SCM5B (multi-bandwidth) | 3 |
| 5B49_2.DAT | SCM5B (sample & hold) | 15 |
| 8BMAIN.DAT | 8B | 148 |
| DSCOUT.DAT | DSCA (output) | 23 |
| DSCMAIN4.DAT | DSCA (input) | 391 |
| SCTMAIN.DAT | DSCT | 103 |
| 7BMAIN.DAT | SCM7B | 276 |
If there are additional spec files we're missing, let me know the paths and we'll add them.
## SCM7B Support Added
The 7B product family is now fully supported in the datasheet generator:
- 31 test parameters (vs 20 for SCM5B)
- Correct header ("SCM" prefix prepended to model name)
- 120VAC Withstand / Hi-Pot (skipped for 7BPT models)
- "Packing Check List" with blank fields (vs pre-marked checkboxes on 5B/8B)
- "Tested by" and "QC" signature lines
- Note: The 7B DAT format (single CSV line) doesn't include individual accuracy test points, so the accuracy table is omitted. Only the Final Test Results section is generated from DAT data.
## Work Order Search & Linking
Imported all 33,745 work order status reports from the test station Reports folders:
- 63,263 individual test lines parsed (serial number, model, pass/fail, date/time, station)
- 2.27 million test records linked to their work orders
In the web app (http://192.168.0.6:3000):
- New "Work Order #" search field — enter a WO number to find all associated test records
- Click the WO number in any record's detail view to see the full work order:
- All serial numbers tested under that WO
- Pass/fail status for each (including retests)
- Test program and version used
- Test station and timestamps
- New work order reports are automatically imported when synced from the NAS
## Datasheet Formatting Refined
Compared generated datasheets against originals from the DFWDS archive and fixed column alignment to match the QuickBASIC output:
- TAB positions match exactly (parameter names, measured values, spec limits, pass/fail)
- Number formatting matches QB PRINT USING (right-justified, correct decimal places)
- STR$() behavior replicated (leading space for positive numbers, dropped leading zeros)
- Spec limit formatting matches (e.g., "+/- .03 %" not "+/- 0.03 %")
## View Button Updated
The "SHEET" button in the web app now shows a styled HTML page that matches the PDF/TXT layout — white page, monospace font, same column alignment. Includes Print and Download PDF buttons.
## Infrastructure
- Created domain service account (INTRANET\svc_testdatadb) for the TestDataDB Windows service — resolves the file permission issues we were hitting
- Added STAGE folder sync to the NAS sync script — TXT datasheets from DOS machines will now be pulled to AD2 automatically
- Work order report import added to sync script — new reports are ingested automatically every 15 minutes
## Still Open
1. **Website upload** — The old Uploader.aspx endpoints are dead. Need to determine the new upload mechanism for dataforth.com.
2. **STAGE backlog** — ~8,100 TXT files on the NAS from DOS machines need to be processed (script ready, haven't run it yet).
3. **Pending ForWeb export** — ~845K records in the database don't have TXT files in For_Web yet (mostly 7B and older records). Can batch-export as needed.
Let me know if you need anything else.
Mike

View File

@@ -0,0 +1,246 @@
# ClaudeTools on AD2 (Dataforth Domain Controller)
## Identity
This is the AD2 workstation instance of ClaudeTools. This machine is a Windows Server on the Dataforth LAN (192.168.0.6). Your scope is Dataforth-only -- you do not need context about other clients.
## NO EMOJIS
Use ASCII markers: [OK], [ERROR], [WARNING], [SUCCESS], [INFO]
---
## Git & Sync
### Gitea Credentials (no 1Password on this machine)
- URL: https://git.azcomputerguru.com
- Username: mike@azcomputerguru.com
- Password: Gptf*77ttb123!@#-git
- URL-encoded password: Gptf%2A77ttb123%21%40%23-git
- API Token: 9b1da4b79a38ef782268341d25a4b6880572063f
- Remote: https://mike%40azcomputerguru.com:Gptf%2A77ttb123%21%40%23-git@git.azcomputerguru.com/azcomputerguru/claudetools.git
### Branch: ad2
This machine operates on the `ad2` branch. The main workstation merges into main.
### /save behavior
Save session logs to `session-logs/YYYY-MM-DD-session-ad2.md` (note the -ad2 suffix).
After saving, commit and push to origin/ad2.
### /sync behavior
```
git fetch origin
git rebase origin/main
git push origin ad2
```
---
## Dataforth Network
| Host | IP | Role | Notes |
|------|-----|------|-------|
| AD1 | 192.168.0.27 | Primary DC | Disk at 90%, C:\Engineering = 787 GB |
| **AD2** | **192.168.0.6** | **This machine** | Secondary DC, TestDataDB, file shares |
| D2TESTNAS | 192.168.0.9 | SMB1 proxy for DOS | Debian 13, Samba, SSH root/Paper123!@#-nas |
| UDM | 192.168.0.254 | Gateway/Router | UniFi Dream Machine |
| ESXi-122 | 192.168.0.122 | Hypervisor | ESXi |
| ESXi-124 | 192.168.0.124 | Hypervisor | ESXi |
| DOS stations | TS-01 to TS-30+ | Test stations | DOS 6.22, QuickBASIC ATE software |
### Credentials
- AD Sysadmin: INTRANET\sysadmin / Paper123!@#
- D2TESTNAS SSH: root@192.168.0.9 / Paper123!@#-nas
- D2TESTNAS Samba: guest access (no password)
- WINS/NPS: 192.168.0.27:1812/1813
- M365 Tenant: 7dfa3ce8-c496-4b51-ab8d-bd3dcd78b584
- Rsync daemon (NAS): port 873, module "test", user rsync / IQ203s32119
---
## Local Resources
| Resource | Path |
|----------|------|
| TestDataDB app | C:\Shares\testdatadb\ |
| Test database | C:\Shares\testdatadb\database\testdata.db (SQLite, 2.2M+ records) |
| TestDataDB API | http://localhost:3000 |
| Parsers | C:\Shares\testdatadb\parsers\ (multiline.js, csvline.js, shtfile.js, spec-reader.js) |
| Templates | C:\Shares\testdatadb\templates\datasheet-exact.js |
| Import script | C:\Shares\testdatadb\database\import.js |
| Export script | C:\Shares\testdatadb\database\export-datasheets.js |
| Stage import | C:\Shares\testdatadb\import-all-stage.js |
| NAS share | \\D2TESTNAS\test (mapped as T:) |
| Datasheets share | X:\For_Web |
| ProdSW (BAT files) | C:\Shares\test\COMMON\ProdSW\ |
| Sync script | C:\Shares\test\scripts\Sync-FromNAS.ps1 (bidirectional, 15-min schedule) |
---
## DOS Update System - Batch Files
### Boot Sequence on DOS Machines
```
AUTOEXEC.BAT (v4.1)
-> STARTNET.BAT (v2.0) -- init network, map T: and X: drives
-> ATESYNC.BAT
-> CTONW.BAT (v5.0) -- upload test data to network
-> CTONWTXT.BAT (v2.3) -- upload C:\STAGE\*.TXT to T:\STAGE\%MACHINE%
-> NWTOC.BAT (v5.0) -- download updates from network
```
### Current Production Versions (on AD2 & NAS)
| File | Version | Last Update | Purpose |
|------|---------|-------------|---------|
| AUTOEXEC.BAT | v4.1 | 2026-03-12 | Startup config |
| STARTNET.BAT | v2.0 | 2026-01-20 | Network init |
| NWTOC.BAT | v5.0 | 2026-03-16 | Download updates from network |
| CTONW.BAT | v5.0 | 2026-03-28 | Upload test data (5 steps with echo) |
| CTONWTXT.BAT | v2.3 | 2026-03-28 | Upload Stage TXT files (no MD, dirs pre-created) |
| CHECKUPD.BAT | v1.3 | 2026-01-20 | Check for updates |
| UPDATE.BAT | v2.3 | 2026-01-20 | Full system backup |
| STAGE.BAT | v1.0 | Original | Stage system file updates |
| DEPLOY.BAT | v1.0 | 2026-01-20 | One-time deployment installer |
### DOS 6.22 Compatibility Rules
- NO `IF NOT` -- unreliable on DOS 6.22. Use positive `IF EXIST` with GOTO
- NO `IF /I` (case-insensitive compare)
- NO `FOR /F` loops
- NO `%COMPUTERNAME%` -- use `%MACHINE%` (set during DEPLOY)
- `XCOPY /D` requires date parameter (`/D:mm-dd-yy`)
- `MD` fails with error on existing directories -- pre-create dirs server-side
- `COPY` without `/Y` hangs on overwrite prompts
- All paths UPPERCASE for Samba compatibility
- Line endings MUST be CRLF (0D 0A)
---
## Serial Number Encoding (DOS 8.3 filenames)
QuickBASIC ATE encodes long serial numbers for 8.3 filenames:
```
First 2 digits replaced with hex letter if serial too long:
178236-12 -> H8236-12.TXT (17 -> H, charCode 72 - 55 = 17)
10819-1 -> A819-1.TXT (10 -> A, charCode 65 - 55 = 10)
Decode: letter.charCodeAt(0) - 55 = numeric prefix
Only applies when filename starts with [A-Z] followed by digits.
H-prefix files have decoded SN inside the file (SN: 178236-12)
A-prefix files have encoded SN inside the file (SN: A819-1) -- must decode to 10819-1
```
---
## Test Datasheet Pipeline
### 5-Stage Architecture
1. **DOS Test Programs** -> Write DAT files to C:\ATE\*LOG\ and TXT to C:\STAGE\
2. **Boot Upload** -> CTONW.BAT copies DAT to T:\%MACHINE%\LOGS\, CTONWTXT copies TXT to T:\STAGE\%MACHINE%
3. **NAS <-> AD2 Sync** -> Rsync every 15 min (Sync-FromNAS.ps1 scheduled task)
4. **TestDataDB Import** -> import.js parses DAT into SQLite; export-datasheets.js generates TXT to X:\For_Web
5. **Web Share** -> X:\For_Web\ holds validated datasheets (501K+ files)
### import-all-stage.js (ready to run)
Located at `C:\Shares\testdatadb\import-all-stage.js`. Processes ~8,100 TXT files:
- Scans \\D2TESTNAS\test\STAGE\TS-*\*.TXT
- Decodes hex-prefix serial numbers
- Cross-references testdata.db by (serial_number, model_number)
- Inserts missing records as log_type='SHT'
- Copies to X:\For_Web\{decoded_serial}.TXT
```
cd C:\Shares\testdatadb
node import-all-stage.js
```
### Machine data volumes in STAGE
| Machine | Files |
|---------|-------|
| TS-4L | 3,082 |
| TS-4R | 2,741 |
| TS-1R | 509 |
| TS-8R | 478 |
| TS-3R | 435 |
| TS-11R | 325 |
| TS-8L | 285 |
| TS-11L | 248 |
| TS-27 | 10 (already imported) |
| TS-1L | 1 |
### Web Share Layout (X:\)
- X:\For_Web -- Validated datasheets (production)
- X:\For_Web_PDF -- PDF versions (4.7K files)
- X:\Test_Datasheets -- Incoming/staging
- X:\Bad_Datasheets -- Invalid files (18K)
- X:\Datasheets_Log -- Processing logs
---
## Known Issues & Pending Work
### HIGH PRIORITY
1. **Run import-all-stage.js** -- 8,100 TXT files need cross-referencing and ingestion
2. **Website Upload Replacement** -- Old ASP.NET endpoints (Uploader.aspx) return 404. Need new approach.
3. **7B Series Datasheets** -- ~830K records can't generate datasheets (missing 7BMAIN.DAT spec file). Check ENGR share.
4. **Service Permissions** -- testdatadb runs as SYSTEM, causing file permission issues. Change to INTRANET\sysadmin.
### MEDIUM PRIORITY
5. **C2 IP Blocking** -- iptables rules added to UDM for 80.76.49.18 and 45.88.91.99. Need permanent rules in UniFi UI.
6. **MFA Enforcement** -- 19/38 users ready. Report-only until April 4, 2026. Monitor registration.
7. **Joel Lohr Account** -- Retiring March 31. Disable account post-retirement. Auto-reply set to Dan Center.
---
## Security Incident (2026-03-27)
**DF-JOEL2 (192.168.0.143) compromised via phishing:**
- Joel Lohr clicked phishing link in personal Yahoo email
- ScreenConnect C2 installed, "Angel Raya" connected remotely
- Two C2 backdoors deployed via PowerShell
- C2 IPs: 80.76.49.18, 45.88.91.99 (AS399486, suspended by host)
- IC3 Complaint: 1c32ade367084be9acd548f23705736f
- ConnectWise Case: 03464184
- **Remediation complete:** IPs blocked, 3 rogue clients removed, password reset, sessions revoked
- **No lateral movement detected** (32 machines scanned clean)
---
## Key Contacts
| Person | Email | Role |
|--------|-------|------|
| John Lehman | jlehman@dataforth.com | Engineering, QB code, test specs |
| Dan Center | dcenter@dataforth.com | Operations (replacing Joel) |
| Peter Iliya | pIliya@dataforth.com | Applications Engineer |
| AJ | dataforthgit@... | Engineering contact |
| Ken Hoffman | (unresponsive) | TestDataSheetUploader author |
| Georg Haubner | ghaubner@dataforth.com | Has pre-crypto backup on D: drive |
---
## Quick Reference Commands
```powershell
# Check BAT files on NAS
ssh root@192.168.0.9 'ls -la /data/test/COMMON/ProdSW/'
# Trigger NAS sync
Start-ScheduledTask -TaskName 'Sync-FromNAS'
# Check sync log
Get-Content 'C:\Shares\test\scripts\sync-from-nas.log' -Tail 20
# Check TestDataDB health
curl http://localhost:3000/health
# Query test records
node -e "const db=require('better-sqlite3')('C:\\Shares\\testdatadb\\database\\testdata.db',{readonly:true});console.log(db.prepare('SELECT COUNT(*) as cnt FROM test_records').get())"
# Check Stage files on NAS
ssh root@192.168.0.9 'find /data/test/STAGE -name "*.TXT" | wc -l'
```
---
**Last Updated:** 2026-03-29

Binary file not shown.

54
generated-129093-16.TXT Normal file
View File

@@ -0,0 +1,54 @@
DATAFORTH CORPORATION Phone: (520) 741-1404
3331 E. Hemisphere Loop Fax: (520) 741-0762
Tucson, AZ 85706 USA email: info@dataforth.com
TEST DATA SHEET
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Date: 05-15-2018
Model: SCM5B38-05
SN: 129093-16
ACCURACY TEST
Calculated Measured
Vin (mV) Vout (V) Vout (V)* Error (%) Status
========== ========== ========== ========= ========
-20.000 -5.000 -5.000 -0.005 PASS
-9.999 -2.500 -2.499 +0.002 PASS
-0.001 -0.000 +0.001 +0.016 PASS
+10.001 +2.500 +2.502 +0.018 PASS
+20.000 +5.000 +5.000 +0.003 PASS
FINAL TEST RESULTS
Parameter Measured Value Specification Status
======================== =============== ===================== ======
Supply Current, Nom 29.0 mA < 75 mA PASS
Supply Current, Max 130.6 mA < 194 mA PASS
Output Resistance 22 ohms < 55 ohms PASS
Exc. Voltage 10.001 V 10.0+/-0.003 V PASS
Exc. Load Reg. 5 ppm/mA +/-11 ppm/mA PASS
Vout Reg. w/ Load 0.0 % +/-0.1 % PASS
Exc. Current Limit 54.4 mA < 63 mA PASS
Linearity 0.010 % +/- 0.03 % PASS
Accuracy 0.018 % +/- 0.08 % PASS
Supply Sensitivity 0.8 uV/% +/- 2.4 uV/% PASS
Frequency Response 29.2 dB 27+/-7 dB PASS
Output Noise 566 uVrms < 4000 uVrms PASS
240 VAC Withstand PASS
Hi-Pot PASS
_______________________________________________________________________
Check List
Module Appearance: __X__ Mounting Screw: __X__
Pins Straight: __X__ Module Header: __X__
It is hereby certified that the above product is in conformance with
all requirements to the extent specified. This product is not
authorized or warranted for use in life support devices and/or systems.
* NIST traceable calibration certificates support Measured Value data.
Calibration services are available through ANSI/NCSL Z540-1 and
ISO Guide 25 Certified Metrology Labs.

View File

@@ -0,0 +1,53 @@
DATAFORTH CORPORATION Phone: (520) 741-1404
3331 E. Hemisphere Loop Fax: (520) 741-0762
Tucson, AZ 85706 USA email: info@dataforth.com
TEST DATA SHEET
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Date: 05-15-2018
Model: SCM5B38-05
SN: 129093-16
ACCURACY TEST
Calculated Measured
Vin (mV) Vout (V) Vout (V)* Error (%) Status
========== ========== ========== ========= ========
-20.000 -5.000 -5.000 -0.005 PASS
-9.999 -2.500 -2.499 +0.002 PASS
-0.001 -0.000 +0.001 +0.016 PASS
+10.001 +2.500 +2.502 +0.018 PASS
+20.000 +5.000 +5.000 +0.003 PASS
FINAL TEST RESULTS
Parameter Measured Value Specification Status
======================= =============== ===================== ======
Supply Current, Nom 29.0 mA < 75 mA PASS
Supply Current, Max 130.6 mA < 194 mA PASS
Output Resistance 22 ohms < 55 ohms PASS
Exc. Voltage 10.001 V 10.0+/-0.003 V PASS
Exc. Load Reg. 5 ppm/mA +/-11 ppm/mA PASS
Vout Reg. w/ Load 0.0 % +/-0.1 % PASS
Exc. Current Limit 54.4 mA < 63 mA PASS
Linearity 0.010 % +/- 0.03 % PASS
Accuracy 0.018 % +/- 0.08 % PASS
Supply Sensitivity 0.8 uV/% +/- 2.4 uV/% PASS
Frequency Response 29.2 dB 27+/-7 dB PASS
Output Noise 566 uVrms < 4000 uVrms PASS
240 VAC Withstand PASS
Hi-Pot PASS
_______________________________________________________________________
Check List
Module Appearance: __X__ Mounting Screw: __X__
Pins Straight: __X__ Module Header: __X__
It is hereby certified that the above product is in conformance with
all requirements to the extent specified. This product is not
authorized or warranted for use in life support devices and/or systems.
* NIST traceable calibration certificates support Measured Value data.
Calibration services are available through ANSI/NCSL Z540-1 and
ISO Guide 25 Certified Metrology Labs.

View File

@@ -0,0 +1,285 @@
# Test-Datasheet Bug — End-to-End Trace & Diagnosis
**Date:** 2026-06-17
**Host:** AD2 (192.168.0.6) — testdatadb generator + PostgreSQL 18
**Author:** Mike Swanson / AZ Computer Guru
**Status:** DIAGNOSIS ONLY — no code or DB changes made
**Reported by:** John Lehman / Peter Iliya (Dataforth) — customer Wellbore Integrity (Joseph Swinehart) cal-cert audit on 8B35 4-wire RTD certs
---
## 0. TL;DR
There are **two independent defects, both in the datasheet *renderer*** (`templates/datasheet-exact.js`). Ingestion (.DAT parsing), the database contents, and the spec files are all **correct** — the raw test data in the DB holds the right values; the renderer mislabels and mis-maps them.
| # | Defect | Symptom on cert | Scope | Severity |
|---|--------|-----------------|-------|----------|
| **A** | RTD input column rendered as **resistance** instead of **temperature** | Header reads `Rin (ohms)`, should read `Temp. (C)`; positive input values lost their leading `+` | ~24,000 RTD certs (8B35, DSCA34, SCM5B34/35, and any RTD variant) | **HIGH** — this is the audit finding |
| **B** | **Entire DSCA Final-Test parameter list is wrong** | Wrong parameter names, garbage specs (`< 0 mA`, `+/- 0 %`), values aligned to the wrong rows, output column mislabeled (`Vout (V)` vs `Output (mA)`), lines missing/added | up to 78,343 DSCA certs (all DSCLOG) | **HIGH** |
Defect A is a small, surgical fix. Defect B requires rebuilding the DSCA template against the legacy spec (`DSCFIN.DAT`) and is a larger effort.
**Root cause is confirmed against the original ground-truth files** (the DOS-station-generated staged `.TXT`), not assumptions.
---
## 1. The Original File IS the Ground Truth — and We Found It
Mike's key point was correct: **the rendered datasheet exists as a file BEFORE it ever reaches the database.** That original file is produced by the DOS test station itself and is the source of truth. Our DB-based regeneration is what introduces the error.
### Where the original files physically live
The DOS station's QuickBASIC ATE program writes a fully-rendered `.TXT` datasheet to `C:\STAGE\` on the station, then `CTONWTXT.BAT` uploads it to the NAS `STAGE` share. It is mirrored to AD2:
```
ORIGINAL (ground truth) staged datasheets:
AD2: C:\Shares\test\STAGE\<TS-station>\<encoded-SN>.TXT
NAS: /data/test/STAGE/<TS-station>/<encoded-SN>.TXT (\\192.168.0.9\test\STAGE)
```
Filenames use the 8.3 hex-prefix serial encoding (first two digits → letter, `55 + n`):
`179553-13``17``H`**`H9553-13.TXT`**; `180224-7``18``I`**`I0224-7.TXT`**.
**Confirmed files for this investigation:**
| Module | SN | Original staged file (ground truth) | Source `.DAT` |
|--------|----|--------------------------------------|---------------|
| 8B35-04 (4-wire RTD) | 179553-13 | `C:\Shares\test\STAGE\TS-4L\H9553-13.TXT` | `C:\Shares\test\TS-4L\LOGS\8BLOG\35-04.DAT` |
| DSCA38-05 (full bridge) | 180224-7 | `C:\Shares\test\STAGE\TS-11R\I0224-7.TXT` | `C:\Shares\test\TS-11R\LOGS\DSCLOG\38-05.DAT` |
| DSCA34-05C (3-wire RTD) | 180007-8 | `C:\Shares\test\STAGE\TS-4R\I0007-8.TXT` | `C:\Shares\test\TS-4R\LOGS\DSCLOG\34-05C.DAT` |
> Note re Peter: `179553-13` cannot be found on X: / DFWDS / For_Web because the **website copy is regenerated from the DB at upload time** — it is never written to disk. The on-disk ground truth is the **STAGE** copy above, not For_Web. (For_Web on AD2 holds only ~7,500 legacy files and does not contain this SN.)
---
## 2. End-to-End Pipeline (upstream emphasis)
```
[1] DOS Test Station (TS-xx, DOS 6.22, QuickBASIC ATE)
- Runs the unit test, measures everything.
- Writes TWO artifacts:
(a) C:\ATE\...\<model>.DAT raw CSV-ish multi-line test log (-> network LOGS)
(b) C:\STAGE\<encSN>.TXT FULLY RENDERED datasheet <-- GROUND TRUTH
- The station ALREADY knows the sensor type, so it prints the correct
input column ("Temp. (C)" for RTD) and the correct Final-Test parameter
list for that exact model. This is the format we must reproduce.
[2] Boot upload (CTONW.BAT / CTONWTXT.BAT)
- .DAT -> \\NAS\test\<TS>\LOGS\<logtype>\<model>.DAT
- .TXT -> \\NAS\test\STAGE\<TS>\<encSN>.TXT
[3] NAS <-> AD2 sync (Sync-FromNAS.ps1, 15 min)
- Mirrors to C:\Shares\test\... on AD2.
[4] testdatadb ingest (THIS host) ----- the original .TXT is IGNORED here -----
- import.js scans .DAT files (NOT the staged .TXT).
- parsers/multiline.js parses the .DAT into a record:
{ log_type, model_number, serial_number, test_date, test_station,
overall_result, raw_data (the verbatim .DAT block), source_file }
- INSERT ... ON CONFLICT(serial_number) into PostgreSQL test_records.
raw_data = the exact .DAT text block (this is faithful & correct).
[5] Render + upload (the bug lives here)
- render-datasheet.js -> templates/datasheet-exact.js regenerates the
datasheet text FROM raw_data + spec files, in memory.
- upload-to-api.js POSTs {SerialNumber, Content} to Hoffman bulk API.
- Hoffman serves it on the public product page.
```
**The pivotal architectural fact:** step [4] throws away the already-correct rendered `.TXT` and keeps only the raw `.DAT` block (`raw_data`). Step [5] then *re-renders* from scratch. Every rendering defect is introduced in step [5]; the upstream data is fine.
### 2.1 What the `.DAT` / `raw_data` actually contains (8B35-04, SN 179553-13)
```
"8B35-04 " <- model
-1.461694,-1.218078E-02,-.014174,-3.986431E-02,"PASS" <- accuracy pt 1: stim,calc,meas,err,status
151.5394,1.262828,1.26273,-1.966953E-03,"PASS" <- pt 2
303.6477,2.530397,2.531,1.204967E-02,"PASS" <- pt 3
448.7633,3.739694,3.7414,.0341177,"PASS" <- pt 4
598.0475,4.983729,4.9824,-2.658844E-02,"PASS" <- pt 5
"0","0",0 <- step-response placeholder
"PASS 28.424741","PASS","PASS 252.21681","PASS","PASS" <- final-test STATUS groups (5 per line)
"PASS","","PASS","PASS","PASS"
"PASS","PASS 3.478871E-023","PASS 3.986431E-023","PASS","PASS 26.328911"
"PASS","PASS","PASS 40.429370","PASS","PASS 140.50"
```
The **first column of each accuracy point is the stimulus**. For SN 179553-13 it is `-1.46, 151.5, 303.6, 448.8, 598.0` — clearly **temperatures in °C** (model MAXIN = 600 °C), **not** ohms. The spec record confirms `SENTYPE = "P1RTD4W"`, `MAXIN = 600`. So the DB has the right numbers and the right sensor type. Nothing upstream is wrong.
---
## 3. Diffs — Original (correct) vs DB-Generated (wrong)
### 3.1 8B35-04 RTD (SN 179553-13) — Defect A only
ACCURACY block header + first/last rows:
```
ORIGINAL (H9553-13.TXT, ground truth) GENERATED (current testdatadb)
----------------------------------- ------------------------------
Temp. (C) Vout (V) ... Rin (ohms) Vout (V) ... <-- WRONG LABEL
-1.46 ... -1.46 ... (same)
+151.54 ... 151.54 ... <-- lost '+'
+598.05 ... 598.05 ... <-- lost '+'
```
- **Header:** `Temp. (C)` → rendered as `Rin (ohms)`. **This is the audit discrepancy.**
- **Values:** numerically identical (correct temperatures). Only difference: positive values lose the leading `+` because the resistance formatter omits the sign.
- **Final Test Results: byte-for-byte IDENTICAL** to the original (7 lines: Supply Current Nom, Exc. Current #1, Linearity, Accuracy, Supply Sensitivity, Frequency Response, Output Noise). **No missing lines for 8B35.** The 8B/5B Final-Test rendering is correct.
So for the Wellbore Integrity audit, the **only** defect on the 8B35 cert is the input column header label (and the cosmetic `+` sign). The measured data is right.
### 3.2 DSCA38-05 full-bridge (SN 180224-7) — Defect B
This is not a label tweak — the **whole Final Test table is wrong**:
```
ORIGINAL (I0224-7.TXT) GENERATED (current)
Supply Current 23.8 mA < 30 mA Supply Current, Nom 23.8 mA < 0 mA <- spec garbage
Supply Curr. w/ EXC Load 53.8 mA < 80 mA Supply Current @ Max Load 53.8 mA < 0 mA
Excitation Voltage 10.000 V 10+/-.003V Linearity, 50mA Load 10.000 % +/- 0 % <- wrong name+unit
Exc. Load Regulation -6 ppm/mA ... Accuracy, 50mA Load 6 % +/- 0 % <- wrong row
Output Reg. w/ EXC Load 0.00 % +/- .05 % Positive Current Limit 0.0 mA < 0 mA
Excitation Current Limit 54 mA < 65 mA Negative Current Limit 54 mA > 0 mA
Linearity 0.002 % +/- .02 % Overrange 0.002 % > 0 %
Accuracy -0.008 % +/- .05 % Power Supply Sensitivity 0.008 %/% +/-.0006
Power Supply Sens. 0.0000 %/% +/-.0006 Frequency Response 0.0000 dB 25+/-5 dB <- value=0
Frequency Response 25.0 dB 25+/-5 dB Compliance 25.0 % +/- 0 % <- 25.0 is freq!
Output Noise 1205 uVrms <=2000 (Output Noise line dropped entirely)
```
Also in the ACCURACY block: original column titles are `Output (V)` and separator dashes `----------`; the generator emits `Vout (V)` and `==========`. The input header happens to read `Vin (mV)` in both (bridge module), so the bridge input label is OK, but everything below it is misaligned.
### 3.3 DSCA34-05C 3-wire RTD (SN 180007-8) — Defect A **and** B together
```
ORIGINAL (I0007-8.TXT) GENERATED (current)
Temp. (C) Output (mA) ... Rin (ohms) Vout (V) ... <-- A: label + wrong out-unit
Supply Current 50.7 mA < 65 mA Supply Current, Nom 50.7 mA < 0 mA
Exc. Current @ -f.s. 264.0 uA 261 uA Linearity, 0mA Load 264.0 % +/- 0 % <-- 264.0 is uA, not %
Exc. Current @ +f.s. 281.0 uA 278 uA Accuracy, 0mA Load 281.0 % +/- 0 %
Linearity 0.017 % +/-.03% Overrange 0.017 % > 0 %
Accuracy -0.034 % +/-.05% Power Supply Sens. 0.034 %/% +/-.0005
... (9 real params) ... (wrong names, garbage specs, lines dropped)
```
The excitation currents (264.0 uA, 281.0 uA) get printed under the labels "Linearity, 0mA Load" / "Accuracy, 0mA Load" as **percentages** — visibly nonsensical.
---
## 4. Root-Cause Localization
Tested against the original files, the defect is **(c) the renderer** — `templates/datasheet-exact.js`. Ingestion/parsing (a) and the DB data (b) are correct.
### Defect A — RTD treated as resistance
```js
// getSensorNum(): RTD maps to 7
if (s.includes('RTD')) return 7; // line ~150
// Accuracy input-column header (generateExactDatasheet):
} else if (sensorNum === 7) {
inputHeader = ' Rin (ohms)'; // line ~564 <-- WRONG for Dataforth RTD
}
// Accuracy value formatting (formatAccuracyLine):
} else if (sensorNum === 7) {
stimStr = point.stim.toFixed(2).padStart(8); // line ~447 <-- resistance format, no sign
}
```
Dataforth RTD modules always express the input as **temperature** on the datasheet (the RTD curve converts resistance→°C; the `.DAT` already stores °C). `sensorNum === 7` is reached **only** by RTD sentypes (`P1RTD3W`, `P1RTD4W`, `NIRTD3W`, …). There is currently **no module for which `Rin (ohms)` is correct** — true resistance/potentiometer inputs are not routed to 7 (they fall through to the voltage default). So fixing the `7` branch is safe and will not over-correct.
### Defect B — DSCA parameter list mismatch
`DATA_LINES['DSCA']` is a **single hardcoded list** (Supply Current Nom / @ Max Load / Linearity 0mA / Accuracy 0mA / Linearity 5mA / … / Compliance / Accuracy @ 5 ohm). Real DSCA modules use **different Final-Test layouts per subtype** (bridge/excitation modules list Excitation Voltage / Exc. Load Reg. / Output Reg.; RTD/TC list Exc. Current @ ±f.s.; etc.). The hardcoded list does not match, so:
- `raw_data` STATUS groups are mapped positionally onto the **wrong** parameter names;
- `buildTSpecs()` for DSCA reads spec fields (`ILIMIT`, `PERCOVER`, `COMPLIANCE`, `ACCURACY1/2/3`, `LINEAR1/2/3`) that are zero/absent for these modules → specs print as `< 0`, `+/- 0`;
- the skip rule `if (status.length <= 4) continue` drops every bare-`PASS` slot, but because the list is misaligned the *wrong* lines drop and the survivors land on wrong rows;
- the ACCURACY block also uses the 5B/8B titles (`Vout (V)` / `==========`) instead of DSCA's (`Output (V|mA)` / `----------`).
The "missing Final Test lines" complaint is a **symptom of Defect B** (DSCA misalignment + skip rule), **not** a separate bug. The 8B/5B skip rule is correct — the legacy station also prints only tested parameters (verified: 8B35 matches exactly).
> Clarification for the thread: **DSCA38 is a bridge/strain-gauge module (FBRIDGE/HBRIDGE), not an RTD.** The DSCA RTD analog is **DSCA34**. So DSCA38's problem is Defect B; DSCA34's problem is A+B. If the audit specifically concerns RTD resistance-vs-temperature, the DSCA part to verify with the customer is **DSCA34**, while DSCA38 demonstrates the broader DSCA table breakage.
---
## 5. Correct Output & Proposed Fix
### 5.1 Correct RTD output (Defect A)
Per the originals, RTD modules must render:
- **Input column header:** `Temp. (C)` (same column/format as thermocouples, `sensorNum` 36).
- **Input values:** the stimulus value straight from `raw_data` (already °C — **no conversion needed**), signed format (`+598.05`, `-1.46`).
Surgical change in `templates/datasheet-exact.js` (fold RTD into the temperature path):
```js
// (1) Header — replace the sensorNum===7 branch:
if ((sensorNum >= 3 && sensorNum <= 6) || sensorNum === 7) {
inputHeader = ' Temp. (C)';
} else if (sensorNum === 2 || sensorNum === 9) {
inputHeader = ' Iin (mA)';
} else {
inputHeader = (maxIn != null && maxIn < 1) ? ' Vin (mV)' : ' Vin (V)';
}
// (2) Value format — in formatAccuracyLine, treat 7 like 36:
if ((sensorNum >= 3 && sensorNum <= 6) || sensorNum === 7) {
stimStr = formatSigned(point.stim, 2, 8); // temperature, signed
} else { ... }
```
Leave the existing `i===13 && sensorNum===7 → 'ohm/ohm'` unit override (Lead-R-Effect) as-is; it is a separate, correct detail. **Verify exact leading-space alignment** against a known-good thermocouple original before pushing (the header column should byte-match; this is the same polish already in progress in `generated-v2-*.TXT`).
This fix corrects **header + values** for all RTD modules (8B35, DSCA34, SCM5B34/35, etc.) at once. **Do not** add any resistance→temperature math — the data is already temperature.
### 5.2 DSCA template (Defect B)
Not a one-liner. The fix is to drive the DSCA Final-Test parameter list (and ACCURACY column titles/units) **per module subtype**, matching the legacy QuickBASIC DSC writer. The legacy parameter selection lives in **`specdata\DSCFIN.DAT`** (DSC final-test definitions) alongside `DSCMAIN4.DAT`/`DSCOUT.DAT`. Recommended approach:
1. Reverse the DSCFIN.DAT layout (or read the QB DSC datasheet source) to get the per-subtype parameter name/unit/spec list.
2. Replace the single `DATA_LINES['DSCA']` + DSCA branch of `buildTSpecs()` with subtype-aware selection keyed on SENTYPE / output-signal type.
3. Fix the DSCA ACCURACY block to use `Output (V|mA)` (per `OUTSIGTYPE`) and dash separators.
4. Validate byte-for-byte against staged originals across DSCA subtypes (bridge, RTD, TC, current-out, voltage-out).
Until B is fixed, **all DSCA (DSCLOG) website datasheets should be treated as unreliable** for Final-Test content.
---
## 6. Impact (records currently on the website)
| Group | On-web count | Defect |
|-------|-------------:|--------|
| 8B35* | 5,476 | A |
| DSCA34* (RTD) | 3,573 | A + B |
| SCM5B34/35* (RTD) | 14,887 | A |
| All DSCA (DSCLOG) | 78,343 | B (RTD subset also A) |
| **Total on website** | 464,671 | — |
RTD-label exposure (Defect A) ≈ **24,000 certs**; DSCA table exposure (Defect B) ≈ **78,000 certs**.
After fixes are reviewed and deployed, affected records can be re-pushed by clearing `api_uploaded_at` for the affected models and letting the upload path re-render (RE-PUSH is idempotent; Hoffman returns `Unchanged` when content matches).
---
## 7. How to Reproduce / Verify (read-only)
```powershell
# Render current generator output for a SN and compare to the staged original:
cd C:\Shares\testdatadb
node -e "const db=require('./database/db');const {renderContent}=require('./database/render-datasheet');(async()=>{const r=await db.queryOne('SELECT * FROM test_records WHERE serial_number=$1',['179553-13']);if(r.test_date&&r.test_date.toISOString)r.test_date=r.test_date.toISOString().slice(0,10);console.log(renderContent(r));await db.close();})()"
# Ground truth:
type C:\Shares\test\STAGE\TS-4L\H9553-13.TXT
```
---
## 8. Files Referenced
- Generator: `C:\Shares\testdatadb\templates\datasheet-exact.js` (repo: `projects/dataforth-dos/datasheet-pipeline/implementation/templates/datasheet-exact.js`)
- Render glue: `C:\Shares\testdatadb\database\render-datasheet.js`
- Specs: `C:\Shares\testdatadb\parsers\spec-reader.js`, `C:\Shares\testdatadb\specdata\*.DAT` (incl. `DSCFIN.DAT`)
- Ingest: `C:\Shares\testdatadb\database\import.js`, `C:\Shares\testdatadb\parsers\multiline.js`
- Ground-truth originals: `C:\Shares\test\STAGE\<TS>\<encSN>.TXT`

View File

@@ -0,0 +1,88 @@
<#
.SYNOPSIS
Configures PowerShell transcript logging for remote sessions.
.DESCRIPTION
Enables comprehensive transcript logging via registry settings,
creates the logging directory with proper permissions, and sets up
automatic log rotation.
.NOTES
Author: ClaudeTools Automation
Version: 1.0
Run as Administrator
#>
$ErrorActionPreference = 'Stop'
$transcriptPath = "C:\ClaudeTools\Logs\Transcripts"
Write-Host "Configuring PowerShell Transcript Logging..." -ForegroundColor Cyan
# Create transcript directory
if (-not (Test-Path $transcriptPath)) {
New-Item -ItemType Directory -Path $transcriptPath -Force | Out-Null
Write-Host "Created transcript directory: $transcriptPath" -ForegroundColor Green
}
# Set permissions on transcript directory
# Administrators: Full Control, SYSTEM: Full Control, Remote Management Users: Read/Write
$acl = Get-Acl $transcriptPath
$acl.SetAccessRuleProtection($true, $false) # Disable inheritance
# Add Administrators - Full Control
$adminRule = New-Object System.Security.AccessControl.FileSystemAccessRule(
"Administrators", "FullControl", "ContainerInherit,ObjectInherit", "None", "Allow"
)
$acl.AddAccessRule($adminRule)
# Add SYSTEM - Full Control
$systemRule = New-Object System.Security.AccessControl.FileSystemAccessRule(
"SYSTEM", "FullControl", "ContainerInherit,ObjectInherit", "None", "Allow"
)
$acl.AddAccessRule($systemRule)
# Add Remote Management Users - Modify (so they can write transcripts)
$rmRule = New-Object System.Security.AccessControl.FileSystemAccessRule(
"Remote Management Users", "Modify", "ContainerInherit,ObjectInherit", "None", "Allow"
)
$acl.AddAccessRule($rmRule)
Set-Acl $transcriptPath $acl
Write-Host "Set permissions on transcript directory" -ForegroundColor Green
# Configure PowerShell transcript logging via registry
$psPath = "HKLM:\SOFTWARE\Policies\Microsoft\Windows\PowerShell\Transcription"
if (-not (Test-Path $psPath)) {
New-Item -Path $psPath -Force | Out-Null
}
# Enable transcription
Set-ItemProperty -Path $psPath -Name "EnableTranscripting" -Value 1 -Type DWord
Set-ItemProperty -Path $psPath -Name "EnableInvocationHeader" -Value 1 -Type DWord
Set-ItemProperty -Path $psPath -Name "OutputDirectory" -Value $transcriptPath -Type String
Write-Host "Enabled PowerShell transcription via registry" -ForegroundColor Green
# Also enable module logging for additional audit trail
$modulePath = "HKLM:\SOFTWARE\Policies\Microsoft\Windows\PowerShell\ModuleLogging"
if (-not (Test-Path $modulePath)) {
New-Item -Path $modulePath -Force | Out-Null
}
Set-ItemProperty -Path $modulePath -Name "EnableModuleLogging" -Value 1 -Type DWord
# Enable script block logging
$scriptPath = "HKLM:\SOFTWARE\Policies\Microsoft\Windows\PowerShell\ScriptBlockLogging"
if (-not (Test-Path $scriptPath)) {
New-Item -Path $scriptPath -Force | Out-Null
}
Set-ItemProperty -Path $scriptPath -Name "EnableScriptBlockLogging" -Value 1 -Type DWord
Write-Host "Enabled module and script block logging" -ForegroundColor Green
Write-Host "`nTranscript logging configuration complete!" -ForegroundColor Green
Write-Host "Transcripts will be saved to: $transcriptPath"
# Display current settings
Write-Host "`n--- Current Settings ---" -ForegroundColor Yellow
Get-ItemProperty -Path $psPath | Select-Object EnableTranscripting, EnableInvocationHeader, OutputDirectory

View File

@@ -0,0 +1,87 @@
<#
.SYNOPSIS
Generates a report of all Active Directory computers.
.DESCRIPTION
This script queries Active Directory for all computer accounts and exports
key properties including name, operating system, last logon, and OU location.
.PARAMETER OutputPath
Optional. Path to export CSV report. If not specified, outputs to console.
.PARAMETER OperatingSystem
Optional. Filter by operating system (e.g., "Windows Server*", "*Windows 10*").
.EXAMPLE
.\Get-ADComputerReport.ps1
Lists all computers to console.
.EXAMPLE
.\Get-ADComputerReport.ps1 -OperatingSystem "Windows Server*" -OutputPath "C:\ClaudeTools\Logs\servers.csv"
Exports all Windows Server computers to CSV.
.NOTES
Author: ClaudeTools Automation
Version: 1.0
Requires: ActiveDirectory PowerShell module
#>
[CmdletBinding()]
param(
[Parameter(Mandatory=$false)]
[string]$OutputPath,
[Parameter(Mandatory=$false)]
[string]$OperatingSystem = "*"
)
# Import AD module
Import-Module ActiveDirectory -ErrorAction Stop
Write-Host "Querying Active Directory computers..." -ForegroundColor Cyan
# Get computers with properties
$computers = Get-ADComputer -Filter "OperatingSystem -like '$OperatingSystem'" -Properties `
OperatingSystem,
OperatingSystemVersion,
LastLogonDate,
Created,
Enabled,
IPv4Address,
Description,
DistinguishedName |
Select-Object `
@{N='Name';E={$_.Name}},
@{N='OperatingSystem';E={$_.OperatingSystem}},
@{N='OSVersion';E={$_.OperatingSystemVersion}},
@{N='Enabled';E={$_.Enabled}},
@{N='IPv4Address';E={$_.IPv4Address}},
@{N='LastLogon';E={$_.LastLogonDate}},
@{N='Created';E={$_.Created}},
@{N='OU';E={($_.DistinguishedName -split ',',2)[1]}},
@{N='Description';E={$_.Description}}
$computerCount = ($computers | Measure-Object).Count
Write-Host "Found $computerCount computers." -ForegroundColor Green
if ($OutputPath) {
$computers | Export-Csv -Path $OutputPath -NoTypeInformation
Write-Host "Report exported to: $OutputPath" -ForegroundColor Green
} else {
$computers | Format-Table -AutoSize
}
# Summary by OS
Write-Host "`n--- Operating System Summary ---" -ForegroundColor Yellow
$computers | Group-Object OperatingSystem | Sort-Object Count -Descending |
Format-Table @{N='Operating System';E={$_.Name}}, Count -AutoSize
# Summary by status
$enabledCount = ($computers | Where-Object { $_.Enabled -eq $true } | Measure-Object).Count
$disabledCount = ($computers | Where-Object { $_.Enabled -eq $false } | Measure-Object).Count
Write-Host "Enabled: $enabledCount | Disabled: $disabledCount"
# Stale computers (no logon in 90 days)
$staleDate = (Get-Date).AddDays(-90)
$staleCount = ($computers | Where-Object { $_.LastLogon -lt $staleDate -or $null -eq $_.LastLogon } | Measure-Object).Count
Write-Host "Stale (no logon 90+ days): $staleCount" -ForegroundColor $(if ($staleCount -gt 0) { 'Yellow' } else { 'Green' })

View File

@@ -0,0 +1,92 @@
<#
.SYNOPSIS
Generates a report of all Active Directory users with key properties.
.DESCRIPTION
This script queries Active Directory for all user accounts and exports
key properties including name, email, last logon, account status, and group memberships.
.PARAMETER OutputPath
Optional. Path to export CSV report. If not specified, outputs to console.
.PARAMETER IncludeDisabled
Switch to include disabled accounts in the report.
.EXAMPLE
.\Get-ADUserReport.ps1
Lists all enabled users to console.
.EXAMPLE
.\Get-ADUserReport.ps1 -OutputPath "C:\ClaudeTools\Logs\users.csv" -IncludeDisabled
Exports all users (including disabled) to CSV file.
.NOTES
Author: ClaudeTools Automation
Version: 1.0
Requires: ActiveDirectory PowerShell module
#>
[CmdletBinding()]
param(
[Parameter(Mandatory=$false)]
[string]$OutputPath,
[Parameter(Mandatory=$false)]
[switch]$IncludeDisabled
)
# Import AD module
Import-Module ActiveDirectory -ErrorAction Stop
Write-Host "Querying Active Directory users..." -ForegroundColor Cyan
# Build filter
$filter = if ($IncludeDisabled) { "*" } else { "Enabled -eq 'True'" }
# Get users with properties
$users = Get-ADUser -Filter $filter -Properties `
DisplayName,
EmailAddress,
Department,
Title,
Manager,
LastLogonDate,
PasswordLastSet,
PasswordNeverExpires,
Enabled,
Created,
MemberOf,
Description |
Select-Object `
@{N='SamAccountName';E={$_.SamAccountName}},
@{N='DisplayName';E={$_.DisplayName}},
@{N='Email';E={$_.EmailAddress}},
@{N='Department';E={$_.Department}},
@{N='Title';E={$_.Title}},
@{N='Enabled';E={$_.Enabled}},
@{N='LastLogon';E={$_.LastLogonDate}},
@{N='PasswordLastSet';E={$_.PasswordLastSet}},
@{N='PasswordNeverExpires';E={$_.PasswordNeverExpires}},
@{N='Created';E={$_.Created}},
@{N='GroupCount';E={($_.MemberOf | Measure-Object).Count}},
@{N='Description';E={$_.Description}}
$userCount = ($users | Measure-Object).Count
Write-Host "Found $userCount users." -ForegroundColor Green
if ($OutputPath) {
$users | Export-Csv -Path $OutputPath -NoTypeInformation
Write-Host "Report exported to: $OutputPath" -ForegroundColor Green
} else {
$users | Format-Table -AutoSize
}
# Summary statistics
Write-Host "`n--- Summary ---" -ForegroundColor Yellow
Write-Host "Total Users: $userCount"
$enabledCount = ($users | Where-Object { $_.Enabled -eq $true } | Measure-Object).Count
$disabledCount = ($users | Where-Object { $_.Enabled -eq $false } | Measure-Object).Count
Write-Host "Enabled: $enabledCount"
Write-Host "Disabled: $disabledCount"
$neverExpire = ($users | Where-Object { $_.PasswordNeverExpires -eq $true } | Measure-Object).Count
Write-Host "Password Never Expires: $neverExpire"

111
scripts/Get-GPOStatus.ps1 Normal file
View File

@@ -0,0 +1,111 @@
<#
.SYNOPSIS
Reports on Group Policy Object status and replication.
.DESCRIPTION
This script checks all GPOs in the domain and reports their status,
including version information, links, and replication status between
AD and SYSVOL.
.PARAMETER OutputPath
Optional. Path to export CSV report. If not specified, outputs to console.
.PARAMETER CheckReplication
Switch to perform detailed replication check between AD and SYSVOL.
.EXAMPLE
.\Get-GPOStatus.ps1
Lists all GPOs with basic status.
.EXAMPLE
.\Get-GPOStatus.ps1 -CheckReplication -OutputPath "C:\ClaudeTools\Logs\gpo-status.csv"
Full replication check with CSV export.
.NOTES
Author: ClaudeTools Automation
Version: 1.0
Requires: GroupPolicy PowerShell module
#>
[CmdletBinding()]
param(
[Parameter(Mandatory=$false)]
[string]$OutputPath,
[Parameter(Mandatory=$false)]
[switch]$CheckReplication
)
# Import required modules
Import-Module GroupPolicy -ErrorAction Stop
Import-Module ActiveDirectory -ErrorAction Stop
Write-Host "Querying Group Policy Objects..." -ForegroundColor Cyan
# Get all GPOs
$gpos = Get-GPO -All | Select-Object `
@{N='Name';E={$_.DisplayName}},
@{N='ID';E={$_.Id}},
@{N='Status';E={$_.GpoStatus}},
@{N='CreationTime';E={$_.CreationTime}},
@{N='ModificationTime';E={$_.ModificationTime}},
@{N='UserVersion';E={$_.User.DSVersion}},
@{N='ComputerVersion';E={$_.Computer.DSVersion}},
@{N='WMIFilter';E={$_.WmiFilter.Name}}
$gpoCount = ($gpos | Measure-Object).Count
Write-Host "Found $gpoCount GPOs." -ForegroundColor Green
# Check GPO links
Write-Host "`nChecking GPO links..." -ForegroundColor Cyan
$gpoLinks = @()
foreach ($gpo in (Get-GPO -All)) {
$report = [xml](Get-GPOReport -Guid $gpo.Id -ReportType Xml)
$links = $report.GPO.LinksTo.SOMPath
$gpoLinks += [PSCustomObject]@{
Name = $gpo.DisplayName
LinkCount = if ($links) { ($links | Measure-Object).Count } else { 0 }
Links = if ($links) { $links -join "; " } else { "Not Linked" }
}
}
if ($CheckReplication) {
Write-Host "`nChecking SYSVOL replication status..." -ForegroundColor Cyan
$domain = (Get-ADDomain).DNSRoot
$dcs = Get-ADDomainController -Filter *
foreach ($dc in $dcs) {
Write-Host " Checking $($dc.HostName)..." -ForegroundColor Gray
$sysvolPath = "\\$($dc.HostName)\SYSVOL\$domain\Policies"
if (Test-Path $sysvolPath) {
$sysvolGPOs = Get-ChildItem $sysvolPath -Directory | Where-Object { $_.Name -match '^{' }
Write-Host " SYSVOL GPO count: $($sysvolGPOs.Count)" -ForegroundColor Green
} else {
Write-Host " Unable to access SYSVOL" -ForegroundColor Red
}
}
}
# Output results
if ($OutputPath) {
$gpos | Export-Csv -Path $OutputPath -NoTypeInformation
Write-Host "`nReport exported to: $OutputPath" -ForegroundColor Green
} else {
Write-Host "`n--- GPO List ---" -ForegroundColor Yellow
$gpos | Format-Table Name, Status, ModificationTime, UserVersion, ComputerVersion -AutoSize
Write-Host "`n--- GPO Links ---" -ForegroundColor Yellow
$gpoLinks | Format-Table Name, LinkCount, Links -AutoSize
}
# Summary
Write-Host "`n--- Summary ---" -ForegroundColor Yellow
Write-Host "Total GPOs: $gpoCount"
$unlinked = ($gpoLinks | Where-Object { $_.LinkCount -eq 0 } | Measure-Object).Count
Write-Host "Unlinked GPOs: $unlinked" -ForegroundColor $(if ($unlinked -gt 0) { 'Yellow' } else { 'Green' })
$disabled = ($gpos | Where-Object { $_.Status -ne 'AllSettingsEnabled' } | Measure-Object).Count
Write-Host "Disabled/Partial GPOs: $disabled" -ForegroundColor $(if ($disabled -gt 0) { 'Yellow' } else { 'Green' })

View File

@@ -0,0 +1,173 @@
<#
.SYNOPSIS
Checks Active Directory replication health across domain controllers.
.DESCRIPTION
This script performs comprehensive AD replication health checks including
replication status, partner connectivity, and identifies any replication failures.
.PARAMETER OutputPath
Optional. Path to export results. If not specified, outputs to console.
.PARAMETER Detailed
Switch to show detailed replication information per DC.
.EXAMPLE
.\Get-ReplicationHealth.ps1
Basic replication health check.
.EXAMPLE
.\Get-ReplicationHealth.ps1 -Detailed -OutputPath "C:\ClaudeTools\Logs\repl-health.txt"
Detailed check with output to file.
.NOTES
Author: ClaudeTools Automation
Version: 1.0
Requires: ActiveDirectory PowerShell module, repadmin.exe
#>
[CmdletBinding()]
param(
[Parameter(Mandatory=$false)]
[string]$OutputPath,
[Parameter(Mandatory=$false)]
[switch]$Detailed
)
# Import AD module
Import-Module ActiveDirectory -ErrorAction Stop
$output = @()
$output += "=" * 60
$output += "AD REPLICATION HEALTH REPORT"
$output += "Generated: $(Get-Date -Format 'yyyy-MM-dd HH:mm:ss')"
$output += "=" * 60
Write-Host "Checking AD Replication Health..." -ForegroundColor Cyan
# Get all DCs
$dcs = Get-ADDomainController -Filter *
$output += "`nDomain Controllers Found: $($dcs.Count)"
foreach ($dc in $dcs) {
$output += "`n--- $($dc.HostName) ---"
Write-Host "Checking $($dc.HostName)..." -ForegroundColor Gray
}
# Check replication summary using repadmin
$output += "`n" + "=" * 60
$output += "REPLICATION SUMMARY (repadmin /replsummary)"
$output += "=" * 60
try {
$replSummary = repadmin /replsummary 2>&1
$output += $replSummary
Write-Host "Replication summary retrieved." -ForegroundColor Green
} catch {
$output += "ERROR: Unable to run repadmin /replsummary"
Write-Host "Error running repadmin" -ForegroundColor Red
}
# Check for replication failures
$output += "`n" + "=" * 60
$output += "REPLICATION FAILURES (repadmin /showrepl * /errorsonly)"
$output += "=" * 60
try {
$replErrors = repadmin /showrepl * /errorsonly 2>&1
if ($replErrors -match "error" -or $replErrors -match "fail") {
$output += $replErrors
Write-Host "Replication ERRORS detected!" -ForegroundColor Red
} else {
$output += "No replication errors detected."
Write-Host "No replication errors." -ForegroundColor Green
}
} catch {
$output += "ERROR: Unable to check replication errors"
}
# Queue length
$output += "`n" + "=" * 60
$output += "REPLICATION QUEUE (repadmin /queue)"
$output += "=" * 60
try {
$replQueue = repadmin /queue 2>&1
$output += $replQueue
} catch {
$output += "ERROR: Unable to check replication queue"
}
if ($Detailed) {
$output += "`n" + "=" * 60
$output += "DETAILED REPLICATION STATUS (repadmin /showrepl)"
$output += "=" * 60
try {
$replDetail = repadmin /showrepl 2>&1
$output += $replDetail
} catch {
$output += "ERROR: Unable to get detailed replication status"
}
# DFSR Health (if applicable)
$output += "`n" + "=" * 60
$output += "DFSR SYSVOL REPLICATION STATUS"
$output += "=" * 60
try {
$dfsrStatus = Get-DfsrMember -ErrorAction SilentlyContinue
if ($dfsrStatus) {
$output += "DFSR Members:"
foreach ($member in $dfsrStatus) {
$output += " - $($member.ComputerName): $($member.DomainName)"
}
} else {
$output += "DFSR not configured or FRS in use."
}
} catch {
$output += "Unable to query DFSR status (may be using FRS)"
}
}
# AD Database health
$output += "`n" + "=" * 60
$output += "AD DATABASE INTEGRITY"
$output += "=" * 60
$adDb = Get-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Services\NTDS\Parameters" -ErrorAction SilentlyContinue
if ($adDb) {
$dbPath = $adDb.'DSA Database file'
$logPath = $adDb.'Database log files path'
$output += "Database Path: $dbPath"
$output += "Log Path: $logPath"
if (Test-Path $dbPath) {
$dbSize = (Get-Item $dbPath).Length / 1MB
$output += "Database Size: $([math]::Round($dbSize, 2)) MB"
}
}
# Final summary
$output += "`n" + "=" * 60
$output += "HEALTH CHECK COMPLETE"
$output += "=" * 60
# Output results
if ($OutputPath) {
$output | Out-File -FilePath $OutputPath -Encoding UTF8
Write-Host "`nReport saved to: $OutputPath" -ForegroundColor Green
} else {
$output | ForEach-Object { Write-Host $_ }
}
# Quick status summary
Write-Host "`n--- Quick Status ---" -ForegroundColor Yellow
Write-Host "Domain Controllers: $($dcs.Count)"
$errorMatch = $replErrors -match "error|fail"
if ($errorMatch) {
Write-Host "Replication Status: ERRORS DETECTED" -ForegroundColor Red
} else {
Write-Host "Replication Status: HEALTHY" -ForegroundColor Green
}

View File

@@ -0,0 +1,107 @@
<#
.SYNOPSIS
Rotates and cleans up old log files.
.DESCRIPTION
Removes transcript and log files older than the specified retention period.
Designed to run as a scheduled task daily.
.PARAMETER RetentionDays
Number of days to retain log files. Default is 30.
.PARAMETER LogPath
Path to the logs directory. Default is C:\ClaudeTools\Logs.
.PARAMETER WhatIf
Shows what would be deleted without actually deleting.
.EXAMPLE
.\Invoke-LogRotation.ps1
Removes logs older than 30 days.
.EXAMPLE
.\Invoke-LogRotation.ps1 -RetentionDays 14 -WhatIf
Shows what would be deleted with 14-day retention.
.NOTES
Author: ClaudeTools Automation
Version: 1.0
#>
[CmdletBinding(SupportsShouldProcess)]
param(
[Parameter(Mandatory=$false)]
[int]$RetentionDays = 30,
[Parameter(Mandatory=$false)]
[string]$LogPath = "C:\ClaudeTools\Logs"
)
$rotationLog = Join-Path $LogPath "rotation.log"
$timestamp = Get-Date -Format "yyyy-MM-dd HH:mm:ss"
function Write-Log {
param([string]$Message)
$logEntry = "[$timestamp] $Message"
Add-Content -Path $rotationLog -Value $logEntry
Write-Host $logEntry
}
Write-Log "=== Log Rotation Started ==="
Write-Log "Retention Period: $RetentionDays days"
Write-Log "Log Path: $LogPath"
$cutoffDate = (Get-Date).AddDays(-$RetentionDays)
$totalDeleted = 0
$totalSizeFreed = 0
# Find and delete old files
$oldFiles = Get-ChildItem -Path $LogPath -Recurse -File |
Where-Object { $_.LastWriteTime -lt $cutoffDate -and $_.Name -ne "rotation.log" }
$fileCount = ($oldFiles | Measure-Object).Count
Write-Log "Found $fileCount files older than $RetentionDays days"
foreach ($file in $oldFiles) {
$fileSize = $file.Length
$filePath = $file.FullName
if ($PSCmdlet.ShouldProcess($filePath, "Delete")) {
try {
Remove-Item $filePath -Force
$totalDeleted++
$totalSizeFreed += $fileSize
Write-Log "Deleted: $filePath ($([math]::Round($fileSize/1KB, 2)) KB)"
} catch {
Write-Log "ERROR deleting $filePath : $_"
}
} else {
Write-Log "WhatIf: Would delete $filePath ($([math]::Round($fileSize/1KB, 2)) KB)"
}
}
# Delete empty subdirectories
$emptyDirs = Get-ChildItem -Path $LogPath -Directory -Recurse |
Where-Object { (Get-ChildItem $_.FullName -Force).Count -eq 0 }
foreach ($dir in $emptyDirs) {
if ($PSCmdlet.ShouldProcess($dir.FullName, "Remove empty directory")) {
try {
Remove-Item $dir.FullName -Force
Write-Log "Removed empty directory: $($dir.FullName)"
} catch {
Write-Log "ERROR removing directory $($dir.FullName) : $_"
}
}
}
# Summary
$sizeMB = [math]::Round($totalSizeFreed / 1MB, 2)
Write-Log "=== Rotation Complete ==="
Write-Log "Files Deleted: $totalDeleted"
Write-Log "Space Freed: $sizeMB MB"
# Show current disk usage
$currentSize = (Get-ChildItem -Path $LogPath -Recurse -File | Measure-Object -Property Length -Sum).Sum
$currentSizeMB = [math]::Round($currentSize / 1MB, 2)
Write-Log "Current Log Directory Size: $currentSizeMB MB"

View File

@@ -0,0 +1,177 @@
# Session Log - AD2 - 2026-04-03
## Session Summary
Continued test datasheet pipeline work from the 03-27/28/29 session. Major focus on:
1. Fixing formatting issues in generated TXT datasheets (column alignment matching QuickBASIC)
2. Adding 7B product family support to the exact-match formatter
3. Loading additional spec files (5B49_2.DAT for the last missing Quatronix datasheet)
4. Work order report ingestion (33K WOs, 63K lines, 2.27M records linked)
5. UI improvements (WO search, WO detail popup, View button styled HTML)
6. Generating full Dataforth audit document
7. Planning and beginning PostgreSQL migration (SQLite hitting 4.4GB, queries timing out)
8. Uninstalling SQL Server Express, installing PostgreSQL 18
## Key Decisions
- PostgreSQL over SQL Server Express (no 10GB limit, better FTS, no licensing)
- Install PG locally on AD2 (not SAGE-SQL or new VM)
- Clean cutover (no parallel run with SQLite)
- MSSQL Express uninstalled to free resources
## Credentials
### PostgreSQL (newly installed)
- Host: localhost (AD2, 192.168.0.6)
- Port: 5432
- Superuser: postgres / DfPgSQL2026!
- App role: testdatadb_app / DfTestDB2026!
- Database: testdatadb
- Service: postgresql-18 (runs as INTRANET\sysadmin)
- Install path: C:\Program Files\PostgreSQL\18\
- Data dir: C:\Program Files\PostgreSQL\18\data\
### TestDataDB (existing)
- Service: testdatadb (runs as INTRANET\svc_testdatadb / DfSvcTDB2026!)
- URL: http://192.168.0.6:3000
- SQLite DB: C:\Shares\testdatadb\database\testdata.db (4.4 GB)
### Existing (from CLAUDE.md)
- AD Sysadmin: INTRANET\sysadmin / Paper123!@#
- D2TESTNAS SSH: root@192.168.0.9 / Paper123!@#-nas
- Rsync: port 873, user rsync / IQ203s32119
- M365 Tenant: 7dfa3ce8-c496-4b51-ab8d-bd3dcd78b584
- Gitea: mike@azcomputerguru.com / Gptf*77ttb123!@#-git
- Gitea API Token: 9b1da4b79a38ef782268341d25a4b6880572063f
## Work Completed
### Formatting Fixes (datasheet-exact.js)
- Compared generated TXT against originals from X:\For_Web\2025\
- Fixed Final Test Results column alignment to match QB TAB positions:
- TAB(5) param name, TAB(31) measured value, TAB(60-speclen) spec, TAB(61) unit, TAB(71) PASS/FAIL
- Added setCol() and padToCol() helpers for exact column positioning
- Fixed STR$() emulation: leading space for positive, drops leading zero (.03 not 0.03)
- Fixed header spacing (Date, Model, SN fields)
- Fixed checklist spacing to match QB TAB(45)
- Fixed separator line length (23 chars not 24)
### 7B Product Family Support
- Added SCM7B DATA_LINES (31 parameters vs 20 for SCM5B)
- Added SCM7B TSPECS (supply current, linearity, accuracy, excitation, CJC, noise, etc.)
- Added 7B raw_data parser (single CSV line format vs multi-line for other families)
- 7B-specific footer: 120VAC (not 240), "Packing Check List" (not "Check List"), blank checkmarks, Tested by/QC lines
- Accuracy section suppressed for 7B (DAT format doesn't include individual test points)
- 7B model names prepend "SCM" in header
### Additional Spec File: 5B49_2.DAT
- 5B49DATA.DAT was 0 bytes (empty), John pointed to 5B49_2.DAT
- 15 models, 93 bytes/record, added SCM5B49_FIELDS to spec-reader.js
- Completed final missing Quatronix datasheet (177000-15, SCM5B49-05)
- All 73/73 Quatronix datasheets now generated
### View Button Enhancement
- /api/datasheet/:id now uses exact-match formatter with styled HTML
- White page on gray background, monospace font, print-optimized
- Includes Print and Download PDF buttons
- Falls back to generic template if exact-match not available
### PDF Endpoint Fallback
- /api/datasheet/:id/pdf falls back to generic template when exact-match fails
- Prevents 422 errors for unsupported families
### Work Order Reports
- Created parsers/wo-report.js (parses WO status report TXT format)
- Created database/import-work-orders.js (imports WOs, creates tables, links to test records)
- Created work_orders table (33,745 records) and work_order_lines table (63,263 records)
- Added work_order column to test_records, 2,277,183 records linked via serial number pattern
- Added /api/workorder/:wo and /api/workorder-search endpoints
- Added WO# search field and clickable WO link in UI detail view
- WO detail popup shows all test lines, pass/fail history, program version
- Added auto-import hooks to Sync-FromNAS-rsync.ps1 for Reports and STAGE folders
### Sync Script Updates (Sync-FromNAS-rsync.ps1)
- Added STAGE folder sync (NAS STAGE -> AD2)
- Added $syncedReportFiles and $syncedStageFiles tracking
- Added WO report auto-import after sync
- Added STAGE TXT auto-import after sync
- Updated status file with WO and STAGE counts
### Dataforth Audit Document
- Generated C:\Users\sysadmin\Desktop\Dataforth-Audit-2026-04-02.txt
- 22 sections: company info, network, AD computers/users/groups, GPO, DNS, shares, tasks, services, firewall, credentials, M365, security incidents, manufacturing infrastructure, pipeline, applications, backups, known issues, contacts
### PostgreSQL Migration (IN PROGRESS)
- Plan approved: PostgreSQL 18 on AD2, clean cutover
- PostgreSQL 18.3 installed via Chocolatey
- Database cluster initialized (C:\Program Files\PostgreSQL\18\data\)
- Service registered as postgresql-18 (runs as INTRANET\sysadmin)
- Database created: testdatadb owned by testdatadb_app
- SQL Server Express 2022 uninstalled
### SQLite Issues Documented
- Database grown to 4.4GB
- Single search query took 5,157 seconds (85 minutes)
- Stats query took 216 seconds
- WAL file grew to 454MB
- ANALYZE ran for 30+ minutes without completing
- SHM/WAL permission conflicts between SYSTEM and sysadmin
### Service Account (from previous session, still relevant)
- INTRANET\svc_testdatadb created in OU=ServiceAccounts
- testdatadb Windows service runs under this account
- SeServiceLogonRight granted
- Permissions on C:\Shares\testdatadb and C:\Shares\webshare
## Files Created
- C:\Shares\testdatadb\fix-sysadmin-logon.ps1
- C:\ClaudeTools\Test Datasheets\weekend-update-draft.md
- C:\Users\sysadmin\Desktop\Dataforth-Audit-2026-04-02.txt
- C:\Users\sysadmin\.claude\plans\generic-petting-lovelace.md (PG migration plan)
## Files Modified
- C:\Shares\testdatadb\templates\datasheet-exact.js — Column alignment, 7B support, STR$() emulation
- C:\Shares\testdatadb\parsers\spec-reader.js — Added 5B49 TYPE, fuzzy model matching
- C:\Shares\testdatadb\parsers\wo-report.js — Created (WO report parser)
- C:\Shares\testdatadb\database\import-work-orders.js — Created (WO importer)
- C:\Shares\testdatadb\database\schema.sql — Added work_order column
- C:\Shares\testdatadb\routes\api.js — Added WO endpoints, PDF fallback, view button exact-match
- C:\Shares\testdatadb\public\index.html — WO search, WO popup, view button styling
- C:\Shares\test\scripts\Sync-FromNAS-rsync.ps1 — STAGE sync, WO auto-import
## Database Stats (as of session end)
- Test records: 2,294,877
- Work orders: 33,745
- Work order lines: 63,263
- Records with WO linked: 2,277,183
- ForWeb exported: ~1,436,000
- Model specs loaded: 1,470+
## Pending / Next Steps
1. **PostgreSQL Migration (Step 2-8):**
- Create PG schema (schema-pg.sql) with tsvector FTS
- Create database abstraction layer (db.js)
- Migrate routes/api.js to async
- Migrate import/export scripts
- Build data migration script
- Migrate 2.28M records
- Test and cutover
2. **Tune PostgreSQL config:**
- shared_buffers=1GB, work_mem=64MB, maintenance_work_mem=256MB
3. **MSSQL cleanup:**
- May need reboot to fully remove SQL Server services
4. **Website upload:**
- Old ASP.NET endpoints still dead (404)
- Need to determine new upload mechanism
5. **Joel Lohr account:**
- Retired March 31 — account needs to be disabled
## Infrastructure
- AD2: 192.168.0.6 (Windows Server 2016)
- PostgreSQL 18: localhost:5432 (newly installed, service running)
- TestDataDB: localhost:3000 (SQLite, service running)
- NAS: 192.168.0.9 (rsync port 873)
- STAGE backlog: cleared (0 files remaining)