Results pane now fills the full width on load (the empty ~500px inspector no longer
reserves space). Inspector column expands when a record opens; Esc closes it back to
full-width results. Same behavior at the 1180px breakpoint.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause of the ~5,148 unpublished 8B/5B/SCM PASS records (driving the fix):
(1) parseRawData wrongly consumes a PASS/FAIL line as the step-response line for
non-DSCA families that omit the "0","0",v line (8B45/8B49/5B39/SCM5B33...) ->
drops the first Final-Test group -> measurement-count mismatch -> null render.
(2) Even parsed, the renderer has ONE hardcoded DATA_LINES['8B'] (RTD-shaped), so
models like 8B45 (frequency input, == DSCA45 structurally) get wrong param
names/specs. Same class as DSCA -> needs per-model templates.
Mined per-model templates from the Hoffman originals (published siblings) for all
136 mineable models via tools/mine-hoffman-dsca.py (family-agnostic extractor):
8b5bscm-templates.json = {accOut, accHeader, rows[name,spec], _srcSerial}.
Input-type split: 72 voltage / 18 temp / 12 current (accuracy already handled) +
10 frequency (8B45/5B45 — same unsolved freq-accuracy as DSCA45) + 24 7B/other.
Only 3 niche models (17 units) have no Hoffman original.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Backend (deployed live on AD2, service restarted, + repo copy resynced — it was
far behind the deployed server):
- /api/search: add whitelisted sort/dir (NULLS LAST) so sortable headers and the
"Latest uploads" preset work. web_status filter and POST /api/upload already
existed on the server; the stale repo copy now matches live.
Frontend (redesign prototype):
- "Latest uploads" preset (web_status=on + sort=api_uploaded_at desc) and
"Not yet published" (web_status=off) are now active presets.
- Push to Web (inspector) + Re-push (multi-select) wired to POST /api/upload
behind a confirm() gate; refresh WEB status after. Validated idempotently on a
published record (unchanged:1, errors:0).
- "Retested units" stays disabled — needs a retest flag in the pipeline (next).
tools/preview-proxy.py: forward POST so the publish buttons work in same-origin preview.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- fitCert: replace the flaky CSS `zoom` (Firefox support is recent/inconsistent)
with transform:scale() measured against the widest line (+ right margin and
font-load retries) so the cert always scales to fit the inspector with no
horizontal clip. Validated live on a narrow 5B cert (0.74x) and a wide DSCA45
cert (0.55x) against the real AD2 dataset.
- inspector Web field -> Published (green) / Not published (amber) chips.
- widen default inspector 480 -> 500px.
- tools/preview-proxy.py: serve the prototype AND reverse-proxy /api to the live
AD2 server so the cert iframe is same-origin during preview — styleCert/fitCert
read iframe.contentDocument, which silently no-ops when the iframe is loaded
cross-origin straight from AD2 (why the fit looked broken in earlier previews).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The DSCA33/DSCA45 main spec files lost in the cryptolocker wipe are recoverable:
the original software published correct certs to the Hoffman product API before
the wipe and our null-skipping renderer never overwrote them. Mine per-model
Final-Test templates (names + specs + verbatim accuracy headers) straight from
those originals instead of requesting spec files from Dataforth/John.
- dsca33-45-templates.json: 56 models (DSCA33 34/35, DSCA45 22/23); only
DSCA33-1948 + DSCA45-1746 (24 units) lack an original.
- mine-hoffman-dsca.py: the re-runnable miner.
- DSCA33-45-HOFFMAN-RECOVERY handoff for the AD2 session (incl. the gate:
validate each render vs its Hoffman original before enabling live rendering).
- memories: Hoffman recovery (supersedes the spec-gap "need John" note) and the
AD2 SSH MTU-blackhole root cause/fix; errorlog entries (syncro jq, ssh correction).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Vanilla single-file (no build/CDN) command-center redesign of the testdatadb search
UI: omni-search with serial/model/text routing + auto-select fast path, dense
monospace results table with PASS/FAIL pills + web-published indicator, persistent
split-pane datasheet inspector (iframe to /api/datasheet), left filter rail
(result/date/model/station/log), server pagination, CSV export, URL state, keyboard
nav (/ ↑↓ ↵ Esc), clinical light theme. Hits the existing API; deployed to AD2 as
public/index.redesign.html (preview at :3000/index.redesign.html). Synthesized from
Grok + Gemini concepts (both converged on this command-center design).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remote SSH/VPN to AD2 keeps flapping; hand the remaining datasheet fixes to the
local AD2 session. Includes the per-subtype approach (DSCA_TEMPLATES from staged
originals — STAGE 1 done, dsca-templates.json on AD2 = 126 models), the render-wiring
+ per-subtype byte-validation gate, Fix 5 (379 backfill via legacy_cert_text), the
discipline (backup/save-state/validate-before-publish), and the derive-dsca-templates
tool. Ref ticket #32441.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- UNREGISTERED_MODELS set: 9 model numbers not in Hoffman API catalog; skipped
silently instead of generating errors
- batch-500 fallback: when a bulk batch returns HTTP 500, retry each record
individually so good records get stamped and only truly-bad records count
as errors
- FAIL-parameter filter: records with any FAIL on a parameter line are excluded
from the push before the batch is assembled
- notify.js integration: wired in existing notification module
Files added:
- projects/dataforth-dos/database/upload-to-api.js
- projects/dataforth-dos/database/notify.js
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Establishes inter-session coordination for 29 projects/clients:
- Full lock/component format for active projects (dataforth-dos,
radio-show, cascades-tucson, valleywide, instrumental-music-center,
lens-auto-brokerage, msp-audit-scripts)
- Light format for complete/stalled/planning (msp-pricing, pavon,
wrightstown-*, gururmm-agent, community-forum, glaztech, etc.)
- Onboarding stubs for recently added clients
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- TEST-DATASHEET-PROCESS.md: comprehensive pipeline documentation for
Dataforth engineering (10 sections, data flow, state diagram, FAQ)
- signing-attestation/: domain ownership attestation letter with
in-place signature for Azure Trusted Signing identity validation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Installed C:\ProgramData\dataforth-uploader\ on AD2 with:
- credentials.json (SYSTEM+Administrators ACL only)
- run-pipeline.ps1 (DFWDS-process -> enumerate For_Web -> upload-delta)
- dfwds-process.js + upload-delta.js (copied from prior install dir)
- logs/ with 60-day retention
Scheduled Task 'DataforthTestDatasheetUploader' registered as SYSTEM,
hourly trigger, 30-min execution limit. First SYSTEM-context run verified:
received=7061 unchanged=7061 errors=0 in 8.7s.
Initial registration via inline base64 mangled the backslashes in the -File
argument (resulted in ERROR_DIRECTORY 0x8007010B). Fixed by running the
registration PowerShell from a file rather than an encoded command string.
Also deleted throwaway tmp/list_amtransit.py + tmp/reset_cansley.py which
had hardcoded ACG admin password.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Built the missing piece between the test datasheet pipeline and Dataforth's
new product API. End-to-end:
- Pulled DFWDS (Dataforth Web Datasheet System) VB6 source from
AD1\Engineering\ENGR\ATE\Test Datasheets\DFWDS to local for analysis
- Decoded its filename validation: A-J prefix decodes (A=10..J=19), all-
numeric WO# valid (no leading 0), anything else bad
- Ported the validation + move logic to Node (dfwds-process.js)
- Built bulk uploader (upload-delta.js) for Hoffman's Swagger API
(POST /api/v1/TestReportDataFiles/bulk with OAuth client_credentials)
Sanitized 3 prior reference scripts (fetch-server-inventory, test-scenarios,
test-upload-two) to read CF_* env vars instead of hardcoded creds.
Live drain results:
- 897 files moved Test_Datasheets -> For_Web (all valid, no renames, no
bad), DFWDS port summary in 1.1s
- Pushed entire For_Web (7,061 files) to Hoffman API in 49.7s @ 142/s:
Created=803 Updated=114 Unchanged=6,144 Errors=0
- Server count: 489,579 -> 490,382 (+803 net new)
Also:
- Added clients/dataforth/.gitignore to exclude plaintext Oauth.txt note
- Added clients/instrumental-music-center/docs/2026-04-13-ticket-notes.md
(ticket write-up of 2026-04-11/12/13 IMC1 RDS removal/SQL migration work)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comprehensive record of 2026-04-11/12 work extending the Dataforth Test
Datasheet Pipeline: discovery, implementation, deploy to AD2, full
backfill of 27,937 datasheets, post-deploy regex patch for QB plain-
decimal PASS lines, and repo commit 0dd3d82.
Includes credentials, infrastructure paths, commit reference, open
items (vault hygiene, rsync coverage), and accuracy-extraction
reference logic for future sessions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extends the Test Datasheet Pipeline on AD2:C:\Shares\testdatadb to
generate web-published datasheets for the SCMVAS-Mxxx (obsolete) and
SCMHVAS-Mxxxx (replacement) High Voltage Input Module product lines.
Both are tested either with the existing TESTHV3 software (production
VASLOG .DAT logs) or in Engineering with plain .txt output.
Key changes on AD2 (all deployed 2026-04-12 with dated backups):
- parsers/spec-reader.js: getSpecs() returns a `{_family:'SCMVAS',
_noSpecs:true}` sentinel for SCMVAS/SCMHVAS/VAS-M/HVAS-M model prefixes
so the export pipeline does not silently skip them for missing specs.
- templates/datasheet-exact.js: new Accuracy-only template branch
(generateSCMVASDatasheet + helpers) that mirrors the existing shipped
format byte-for-byte. Extraction regex covers both QuickBASIC STR$()
output formats: scientific-with-trailing-status-digit (98.4% of
records) and plain-decimal (1.6% of records above QB's threshold).
- parsers/vaslog-engtxt.js (new): parses the Engineering-Tested .txt
files in TS-3R\LOGS\VASLOG\VASLOG - Engineering Tested\. Filename SN
regex strips optional trailing 14-digit timestamp; in-file "SN:"
header is the authoritative source when the filename is malformed.
- database/import.js: LOG_TYPES grows a VASLOG_ENG entry with
subfolder + recursive flags. Pre-existing 7 log types keep their
implicit recursive=true behaviour (config.recursive !== false).
importFiles() routes VASLOG_ENG paths before the generic loop so a
VASLOG - Engineering Tested/*.txt path does not mis-dispatch to the
multiline parser.
- database/export-datasheets.js: VASLOG_ENG records are written
verbatim via fs.copyFileSync(source_file, For_Web/<SN>.TXT) for true
byte-level pass-through, with a graceful raw_data fallback when the
source file is no longer on disk.
Deploy outcome:
- 27,503 SCMVAS/SCMHVAS datasheets rendered (27,065 from scientific +
438 from plain-decimal PASS lines, post-patch rerun)
- 434 Engineering-Tested .txt files pass-through-copied to For_Web
- 0 errors across both batches
Repo layout added here:
- scmvas-hvas-research/: discovery artifacts (source .BAS, hvin.dat,
sample .DAT + .txt, binary-format notes, IMPLEMENTATION_PLAN.md)
- implementation/: staged final code + deploy helpers + local test
harness + per-step verification scripts
- backups/pre-deploy-20260412/: independent local snapshot of the 4
AD2 files replaced, pulled byte-for-byte before deploy
All helper scripts fetch the AD2 password at runtime from the SOPS
vault (clients/dataforth/ad2.sops.yaml). None of the committed files
contain the plaintext credential. Known vault-entry hygiene issue
(stale shell-escape backslash before the `!`) is documented in the
fetcher comments and stripped at read-time; flagged separately for
cleanup.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
DOS 6.22 MD fails on existing dirs, then COPY creates a file instead of
copying into a directory. Dirs now pre-created on NAS server side.
Also removed trailing backslash from COPY target path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mixed-case paths caused "invalid directory" on TS-27. All paths now uppercase
to match DOS convention and Samba default case = upper setting.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
No conditionals, no DEL. Just MD dirs and COPY. Deployed to AD2 and synced to D2TESTNAS.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CTONWTXT.BAT was never being called. Now invoked from CTONW.BAT on every boot.
Destination changed from X: (broken Novell serve.sys check) to T:\Stage\%MACHINE%\.
Removed IF NOT clauses for DOS 6.22 compatibility. Deployed to AD2 and synced to D2TESTNAS.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Added EXE copy from Ate\ProdSW to C:\ATE in NWTOC.BAT
- Added /Y overwrite flag to all COPY commands
- Removed cyclic DATA folder copies from NWTOC
- Session log for 2026-03-16 DF DOS troubleshooting
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Problem: Data syncs to AD2 but import.js not importing to database
- Newest DB record stuck at 2026-01-19 (2 months stale)
- rsync transfers working (9.8MB at 06:30 & 07:15)
- import.js not being called or failing silently
Instructions for PC to diagnose and fix.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
DEPLOY v4.1 fixes critical bug where ECHO >> redirects included
trailing space in MACHINE variable, causing "Too many parameters"
on all COPY commands with subdirectory paths. TS-4L data upload
confirmed working - 84 test files + 90 reports on NAS.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DOS 6.22 IF EXIST with wildcards on SMB1 network paths causes
"Bad command" and "Too many parameters" errors. Rewrote CTONW,
NWTOC, and AUTOEXEC to v4.0 with direct COPY/MD commands.
Pre-created all station LOGS directories on new D2TESTNAS.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Built Debian 13 VM replacement for aging ReadyNAS, deployed rsync-based
sync script to AD2, transferred data, completed IP cutover to 192.168.0.9.
Includes setup scripts, sync fixes, and comprehensive session logs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
SolverBot:
- Inject active project path into agent system prompts so agents
know which directory to scope file operations to
GuruRMM:
- Bump agent version to 0.6.0
- Add serde aliases for PowerShell/ClaudeTask command types
- Add typed CommandType enum on server for proper serialization
- Support claude_task command type in send_command API
Dataforth:
- Fix SCP space-escaping in Sync-FromNAS.ps1
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Video Analysis Agent (.claude/agents/video-analysis.md):
- Frame extraction with ffmpeg
- DOS console text recognition
- Boot sequence documentation
- Integration with Photo Agent and DOS Coding Agent
Debug batch files for video recording:
- ATESYNC-DEBUG.BAT: Orchestrator with PAUSE at each step
- CTONW-DEBUG.BAT: Upload with 10 step-by-step pauses
- NWTOC-DEBUG.BAT: Download with 11 step-by-step pauses
Each step clearly labeled with ECHO for video analysis.
Run ATESYNC-DEBUG TS-3R to capture boot process.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
ATESYNC.BAT v1.0:
- Boot-time orchestrator (ARCHBAT equivalent from TS-27)
- Calls CTONW (upload) then NWTOC (download)
- Creates machine folder structure if missing
- Accepts machine name as parameter or MACHINE env var
CTONW.BAT v3.1:
- Fixed upload path: now uploads to T:\%MACHINE%\LOGS\*LOG
- Added safeguards to prevent data overwriting:
- Refuses to run if MACHINE not set
- Refuses to run if T:\%MACHINE% folder missing
- Logs machine name, date/time, target path
- Uploads all 8 LOG folders plus Reports
Based on analysis of TS-27 golden example machine backup.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Investigation and Documentation:
- Discovered and documented test database system on AD2 server
- Created comprehensive TEST_DATABASE_ARCHITECTURE.md with full system details
- Retrieved all key database files from AD2 (import.js, schema.sql, server configs)
- Documented data flow: DOS machines → NAS → AD2 → SQLite → Web interface
- Verified database health: 1,027,517 records, 1075 MB, dates back to 1990
Database System Architecture:
- SQLite database with Node.js/Express.js web server (port 3000)
- Automated import via Sync-FromNAS.ps1 (runs every 15 minutes)
- 8 log types supported: DSCLOG, 5BLOG, 7BLOG, 8BLOG, PWRLOG, SCTLOG, VASLOG, SHT
- FTS5 full-text search, comprehensive indexes for performance
- API endpoints: search, stats, export, datasheet generation
Troubleshooting Scripts Created:
- Database diagnostics: check-db-simple.ps1, test-db-directly.ps1
- Server status checks: check-node-running.ps1, check-db-server.ps1
- Performance analysis: check-db-performance.ps1, check-wal-files.ps1
- API testing: test-api-endpoint.ps1, test-query.js
- Import monitoring: check-new-records.ps1
- Database optimization attempts: api-js-optimized.js, api-js-fixed.js
- Deployment scripts: deploy-db-optimization.ps1, deploy-db-fix.ps1, restore-original.ps1
Key Findings:
- Database file healthy and queryable (verified with test-query.js)
- Node.js server not running (port 3000 closed) - root cause of web interface issues
- Database last updated 8 days ago (01/13/2026) - automated sync may be broken
- Attempted performance optimizations (WAL mode) incompatible with readonly connections
- Original api.js restored from backup after optimization conflicts
Retrieved Documentation:
- QUICKSTART-retrieved.md: Quick start guide for database server
- SESSION_NOTES-retrieved.md: Complete session notes from database creation
- Sync-FromNAS-retrieved.ps1: Full sync script with database import logic
- import-js-retrieved.js: Node.js import script (12,774 bytes)
- schema-retrieved.sql: SQLite schema with FTS5 triggers
- server-js-retrieved.js: Express.js server configuration
- api-js-retrieved.js: API routes and endpoints
- package-retrieved.json: Node.js dependencies
Action Items Identified:
1. Start Node.js server on AD2 to restore web interface functionality
2. Investigate why automated sync hasn't updated database in 8 days
3. Check Windows Task Scheduler for Sync-FromNAS.ps1 scheduled task
4. Run manual import to catch up on 8 days of test data if needed
Technical Details:
- Database path: C:\Shares\testdatadb\database\testdata.db
- Web interface: http://192.168.0.6:3000 (when running)
- Database size: 1075.14 MB (1,127,362,560 bytes)
- Total records: 1,027,517 (slight variance from original 1,030,940)
- Pass rate: 99.82% (1,029,046 passed, 1,888 failed)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
NWTOC.BAT v3.5:
- Switch from XCOPY to COPY (more reliable in DOS 6.22)
- Remove all >NUL redirects that cause issues
- Add IF NOT EXIST checks before MD to avoid errors
- Add 8 ATE data folder copies (5BDATA, 7BDATA, 8BDATA, DSCDATA,
HVDATA, PWRDATA, RMSDATA, SCTDATA)
- Remove machine-specific section (no longer needed)
- Remove MACHINE variable requirement
DEPLOY.BAT v2.4:
- Switch all XCOPY to COPY for DOS 6.22 compatibility
- Simplify output messages
Also fixed AD2->NAS sync issue:
- Ate/ProdSW folder was not being synced to NAS
- DOS machines were getting outdated DSCDATA files (Dec 2025 vs Jan 2026)
- Updated Sync-FromNAS.ps1 on AD2 to include Ate/ProdSW folder
- Manually synced correct files to NAS (DSCMAIN4.DAT 65508 bytes)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Session log updates for 2026-01-20 with additional work documentation.
This checkpoint also marks completion of GuruRMM security remediation:
- Phase 1: 10 critical security fixes deployed
- Phase 2: 8 major fixes deployed
- Production server updated at 172.16.3.30
- Gitea tracking issue #1 updated
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
