Compare commits
47 Commits
06c44563be
...
44280dfe98
| Author | SHA1 | Date | |
|---|---|---|---|
| 44280dfe98 | |||
| 05f7f1cde2 | |||
| 78d694848a | |||
| a35858058e | |||
| 94330e0ff9 | |||
| 7fb0d95af5 | |||
| bde5ad8706 | |||
| 31cdfa4b02 | |||
| 80065b13e4 | |||
| 5cd4e5c6fb | |||
| be1186aab0 | |||
| 344fbc31be | |||
| 6e42e39b2f | |||
| ac616ae3f5 | |||
| a1a09cd0fb | |||
| 42fa576692 | |||
| 10552b74f6 | |||
| 344c467be4 | |||
| ae9b49a75f | |||
| 2fc1b934eb | |||
| fc80a918ba | |||
| 89ffc6fbe0 | |||
| 88ae32e22e | |||
| d5499741ee | |||
| cfeec7f9f6 | |||
| 120bd8059b | |||
|
|
b7656258b7 | ||
|
|
d2529e9e80 | ||
| a458b7cbef | |||
| ee0df93489 | |||
| 27170e1a5c | |||
| 0745f5d09b | |||
| 82ae349941 | |||
| 760c2264dc | |||
| 72a2fbe6ce | |||
| 419d6e58b5 | |||
| c76bdc4db1 | |||
| b405523a62 | |||
| 852396cebe | |||
| ba3636430f | |||
| 3d59218906 | |||
| 719789b499 | |||
| a700435174 | |||
| 15d367e5d7 | |||
| b8e0988569 | |||
| f84a6cc901 | |||
| 6e5ce8fa0a |
@@ -32,6 +32,9 @@
|
||||
- [AAD Connect msDS-KeyCredentialLink writeback](reference_aadconnect_keycredlink_writeback.md) — "completed-export-errors" + 8344 INSUFF_ACCESS_RIGHTS on a protected admin account = WHfB key writeback blocked by AdminSDHolder. Diagnose with csexport /f:x; fix with dsacls WP;msDS-KeyCredentialLink on AdminSDHolder + SDProp.
|
||||
- [UniFi Site Manager cloud API](reference_unifi_site_manager_api.md) — `api.ui.com` + `X-API-KEY` (vault `services/unifi-site-manager`) = remote access to the WHOLE ACG UniFi fleet (~36 consoles) outside UOS. Tier1 `/v1/hosts|sites|devices|isp-metrics` = inventory+health+WAN. Tier2 CONNECTOR `/v1/connector/consoles/{id}/proxy/network/api/s/default/stat/{device,sta}` = **full UOS parity** (per-radio cu_total airtime + per-client RSSI) for ANY console, remote. Backend `unifi-wifi/scripts/gw-sitemanager.sh` (`fleet|devices|sites|isp|net`). Standalone UDM WAN SSH usually firewalled; per-console SSH pw at `clients/<slug>/udm-ssh`.
|
||||
- [reference_sqlx_migrations_immutable](reference_sqlx_migrations_immutable.md) -- NEVER edit an already-applied sqlx migration file — even a comment. sqlx::migrate! checksums each file at compile time and validates against _sqlx_migrations at startup; a changed checksum crash-loops the server with "migration N was previously applied but has been modified". Code review MUST flag any edit to an applied migration.
|
||||
- [AD2 SSH MTU blackhole](ad2-ssh-mtu-blackhole.md) — AD2 SSH "lockouts"/mid-session read-errors over the Dataforth OpenVPN were a PMTU blackhole (tunnel PMTU ~1424 vs adapter MTU 1500), NOT a ban/account-lockout/flaky tunnel. Fix: pin the OpenVPN adapter MTU to 1400 (done on GURU-5070 via its SYSTEM RMM agent); permanent = `mssfix 1360` on the OpenVPN server. Diagnose over RMM, not SSH.
|
||||
- [DSCA33/45 resolved via Hoffman](project_dsca33_45_resolved_via_hoffman.md) — The "lost" DSCA33/45 spec files are recoverable from the Hoffman API (original certs survived the wipe); do NOT ask John. 56/58 models mined into projects/dataforth-dos/dsca33-45-templates.json; only DSCA33-1948 + DSCA45-1746 (24 units) lack an original. AD2 handoff: DSCA33-45-HOFFMAN-RECOVERY-2026-06-18.md.
|
||||
- [AD2 comms via sync only](ad2-comms-via-sync-only.md) — The AD2 Dataforth-box Claude session is coord-API-isolated (Gitea only); coord msg/lock/todo never reach it. Coordinate with AD2 ONLY via git /sync (committed docs + ## Note blocks).
|
||||
|
||||
## Users
|
||||
- [Howard Enos](user_howard.md) — Mike's brother, technician, full access. Machines: ACG-TECH03L, Howard-Home (authoritative in users.json).
|
||||
@@ -109,6 +112,7 @@
|
||||
- [GURU-BEAST-ROG Setup Status](machine_windows_guru_setup_status.md) — Windows workstation fully configured except SSH key deployment to servers.
|
||||
|
||||
## Project
|
||||
- [DSCA33/DSCA45 spec gap](project_dsca33_45_spec_gap.md) — Datasheet pipeline: DSCA33/DSCA45 render null because their MAIN specs are missing from all recovered DAT files (DSCA33 nowhere; DSCA45 only in DSCFIN.DAT). Data gap, not a bug; ~8,763 certs blocked. Final-Test renders via slotMaps+stub; need real specs from Dataforth or accuracy-block templating.
|
||||
- [CyndyOffice physical HP lockups](cyndyoffice-physical-hp-lockups.md) — RMM "Howard-VM" site agent CyndyOffice is a PHYSICAL HP Pavilion TP01 (not a VM); ~20 hard freezes/6wk = Kernel-Power 41 bugcheck-0, no dump/WHEA = hardware (RAM/PSU/BIOS), SSD healthy. UUID re-enrolls.
|
||||
- [Automate memory consolidation/lint (phased)](project_memory_consolidation_automation.md) — Eventually auto-run /memory-dream; lint+additive fixes can automate early, merges/deletes stay human-approved. Engine: .claude/skills/memory-dream/ + .claude/scripts/sync-memory.sh.
|
||||
- [Trebesch PST consolidation (staged)](project_trebesch_pst_consolidation.md) — Address-book CSV from 24 PSTs on DESKTOP-QNP3ON5; scripts staged at .claude/tmp/treb-*.ps1, WAITING for Howard's 6pm-MST 2026-06-01 go signal (attended run). See [[reference_trebesch_qnp3on5]].
|
||||
@@ -157,3 +161,4 @@
|
||||
- [CT Thoughts backlog](feedback_ct_thoughts_backlog.md) — ClaudeTools harness ideas go in docs/CT_THOUGHTS.md (trigger "ct thought:"); CT analogue of RMM_THOUGHTS. Don't build until explicit go. First entry = ClaudeTools 3.0 web co-work vision.
|
||||
- [AI-auth product boundary](project_ai_auth_product_boundary.md) — ClaudeTools/ClaudeTools 3.0 = internal-only, per-person subscription OAuth ok; GuruRMM = sellable, customer brings own API key (never ACG's subscription); backend dev = internal. Anthropic ToS bans subscription auth in third-party products.
|
||||
- [RMM SYSTEM context can't see user mapped drives](feedback_rmm_system_context_mapped_drives.md) — RMM runs as SYSTEM; `Test-Path F:\` etc. is False even when the user's mapped/redirected drive exists. Diagnose mapped-drive/redirect issues in `context:user_session`. Elevated apps (e.g. QB DB Server Manager "unable to retrieve root folder") need `EnableLinkedConnections=1` + reboot.
|
||||
- [AD2 = Dataforth-ops fork](project_ad2_dataforth_fork.md) — branch ad2 = main + thin Dataforth layer; keep fork edits ADDITIVE (Dataforth context in clients/dataforth/CLAUDE.dataforth.md, NOT .claude/CLAUDE.md); rebase onto main directly when sync.sh self-lock hits; no vault/jq/sops/age on this box.
|
||||
|
||||
21
.claude/memory/ad2-comms-via-sync-only.md
Normal file
21
.claude/memory/ad2-comms-via-sync-only.md
Normal file
@@ -0,0 +1,21 @@
|
||||
---
|
||||
name: ad2-comms-via-sync-only
|
||||
description: The AD2 (Dataforth) Claude session is coord-API-isolated — reach it ONLY via git /sync (committed notes/docs), never coord messages/locks
|
||||
metadata:
|
||||
type: feedback
|
||||
---
|
||||
|
||||
The AD2 Dataforth-box Claude session is **network-isolated from the ACG coord API** (172.16.3.30 —
|
||||
the Dataforth network can't reach it); it only has Gitea/git access. So coord-API **messages, locks,
|
||||
and todos NEVER reach AD2**. ALL inter-session coordination with AD2 must go through git **`/sync`**:
|
||||
committed handoff docs and `## Note for <user>` blocks in synced session logs, which AD2 reads when
|
||||
it pulls. A coord lock on an AD2-only file (e.g. `datasheet-exact.js`) is also meaningless — only the
|
||||
AD2 session edits that box.
|
||||
|
||||
**Why:** burned a round of `coord msg send AD2` + lock that were silent no-ops (Mike: "You can't
|
||||
coord with AD2 — all comms needs to be via sync").
|
||||
|
||||
**How to apply:** to hand work to or coordinate with the AD2 session, write it into a committed doc
|
||||
(e.g. `projects/dataforth-dos/*HANDOFF*.md`) and/or a `## Note for <user>` block in a session log,
|
||||
then `/sync`. Do NOT use the coord skill for AD2. (Coord API is still fine for non-isolated ACG
|
||||
machines.) [[prefer-ssh-over-rmm]]
|
||||
40
.claude/memory/ad2-ssh-mtu-blackhole.md
Normal file
40
.claude/memory/ad2-ssh-mtu-blackhole.md
Normal file
@@ -0,0 +1,40 @@
|
||||
---
|
||||
name: ad2-ssh-mtu-blackhole
|
||||
description: AD2 SSH "lockouts"/mid-session timeouts over the Dataforth OpenVPN were an MTU/PMTU blackhole, not a ban/account-lockout/flaky tunnel; fix = pin the tunnel adapter MTU to 1400
|
||||
metadata:
|
||||
type: project
|
||||
---
|
||||
|
||||
AD2 (Dataforth, `192.168.0.6`) SSH from the fleet over OpenVPN (client subnet `192.168.6.x`)
|
||||
intermittently looked "locked out": sessions **authenticated fine**, then died mid-session with
|
||||
`Read error from remote host 192.168.6.2 ... Unknown error [postauth]` and
|
||||
`ssh_dispatch_run_fatal: Connection from authenticating user sysadmin ... Connection timed out [preauth]`.
|
||||
Small/interactive commands often worked; bulk reads + `scp` stalled.
|
||||
|
||||
**Root cause (diagnosed 2026-06-18 via RMM — SSH itself was the failing channel, so don't diagnose it over SSH):**
|
||||
- NOT account lockout — Windows lockout threshold is 5/30min but **zero 4740 events**; `sysadmin` never locked.
|
||||
- NOT an IP ban — **no IPBan/wail2ban/RdpGuard**, **0 inbound firewall block rules**.
|
||||
- NOT auth — **every** `Accepted publickey for sysadmin` succeeded.
|
||||
- NOT load — AD2 was CPU ~11%, 11.7 GB RAM free.
|
||||
- It was a **PMTU blackhole.** OpenVPN tunnel path MTU is **~1424** (DF ping: wire 1424 passes,
|
||||
1428 drops). But GURU-5070's OpenVPN adapter (`Local Area Connection`, ifIndex 12, IP
|
||||
`192.168.6.2`) was set to **MTU 1500** → TCP negotiated MSS 1460 → full-size bulk/scp segments
|
||||
exceeded the tunnel and were **silently dropped (DF set)**, while sub-MTU interactive packets
|
||||
passed. That is why it presented as random "lockouts" that got worse with bulk transfer.
|
||||
|
||||
**Fix applied (2026-06-18):** `Set-NetIPInterface -InterfaceIndex 12 -AddressFamily IPv4 -NlMtuBytes 1400`
|
||||
run via **GURU-5070's own RMM agent** (`819df0c8...`, runs as `nt authority\system` = elevated; the
|
||||
elevated lever on the local box when you can't self-elevate from the Claude shell). Validated: a
|
||||
**1.41 MB single-session SSH transfer to AD2 completed in 9s, no read error** (previously blackholed).
|
||||
`~/.ssh/config` `ad2` block annotated + tightened keepalives (`ServerAliveInterval 15`,
|
||||
`ServerAliveCountMax 4`, `ConnectTimeout 20`).
|
||||
|
||||
**Durability / permanent fix:** `Set-NetIPInterface` is registry-persistent, but **OpenVPN Connect may
|
||||
reset the adapter MTU to 1500 on reconnect** — re-apply if SSH bulk transfers start stalling again
|
||||
(check `Get-NetIPInterface -InterfaceIndex 12`). The real permanent fix is **server-side on the
|
||||
Dataforth OpenVPN server: `mssfix 1360` (or `push "tun-mtu 1400"`)** so every fleet client clamps
|
||||
automatically — `192.168.6.4` showed the identical symptom, so this is fleet-wide, not 5070-only.
|
||||
|
||||
Corrects the earlier wrong attribution ("flaky VPN tunnel" / "my rapid scp+ssh bursts triggering a
|
||||
ban") — the tunnel is up and stable for small packets; only over-MSS segments were dropped. See
|
||||
[[prefer-ssh-over-rmm]] (RMM-as-fallback guidance still holds; the *reason* was MTU, not a flaky VPN).
|
||||
19
.claude/memory/project_ad2_dataforth_fork.md
Normal file
19
.claude/memory/project_ad2_dataforth_fork.md
Normal file
@@ -0,0 +1,19 @@
|
||||
---
|
||||
name: project_ad2_dataforth_fork
|
||||
description: AD2 is the Dataforth-operations fork (branch ad2); how it relates to main and how to sync it cleanly
|
||||
metadata:
|
||||
type: project
|
||||
---
|
||||
|
||||
AD2 (192.168.0.6, Dataforth domain controller) runs the **`ad2` branch** — a fork dedicated to Dataforth operations. Its `/sync` model is: `git rebase origin/main` then `git push origin ad2`, so `ad2` = `main` + a thin layer of Dataforth-specific commits.
|
||||
|
||||
**Keep the fork layer ADDITIVE** — never edit shared fleet files (especially `.claude/CLAUDE.md`) on `ad2`, or every sync re-conflicts. On 2026-06-17 the old fork edit of `.claude/CLAUDE.md` was relocated: `.claude/CLAUDE.md` now tracks the lean fleet doc (from main), and Dataforth operational context (network, credentials, pipeline, batch-file versions) lives in **`clients/dataforth/CLAUDE.dataforth.md`**. Read that file for Dataforth context.
|
||||
|
||||
**Sync gotchas on AD2:**
|
||||
- `sync.sh` is NOT fork-aware on the PUSH step: it pushes `main` (`error: src refspec main does not match any`), so `/sync` and `/save` auto-sync ALWAYS fail at Phase 4 on this fork. It DOES rebase + commit fine. After running it, push manually: `git push --force-with-lease origin ad2` (force needed because the rebase rewrote history). The documented fork flow (fetch → rebase origin/main → push origin ad2) is the reliable path.
|
||||
- `sync.sh` cannot overwrite itself while it is the executing script (Windows file lock) → the rebase step fails mid-run with "unable to create file .claude/scripts/sync.sh: Permission denied" the FIRST time main has a new sync.sh. When that happens: `git reset --hard HEAD`, `git clean -fd` (removes the partial origin/main checkout debris), then run `git rebase origin/main` **directly** (not via sync.sh), and force-push with `--force-with-lease`.
|
||||
- Submodule `projects/msp-tools/guru-connect` fails to clone from 172.16.3.20 (ACG-internal, unreachable from Dataforth LAN) — non-fatal sync warning.
|
||||
- Python was absent until 2026-06-17 (installed 3.12.8, `py` launcher); identity.json was created then too (`user=mike`, `machine=AD2`).
|
||||
- Missing tooling: `jq`, `sops`, `age`. No vault cloned (`D:/vault` absent) → vault sync is N/A here. coord_api (172.16.3.30) is unreachable from the Dataforth LAN.
|
||||
|
||||
Related: [[project_ad2_context.md]]
|
||||
27
.claude/memory/project_dsca33_45_resolved_via_hoffman.md
Normal file
27
.claude/memory/project_dsca33_45_resolved_via_hoffman.md
Normal file
@@ -0,0 +1,27 @@
|
||||
---
|
||||
name: project_dsca33_45_resolved_via_hoffman
|
||||
description: DSCA33/45 "lost spec files" are recoverable from the Hoffman API (original certs survived the wipe) — do NOT request spec files from John; mine templates from Hoffman instead
|
||||
metadata:
|
||||
type: project
|
||||
---
|
||||
|
||||
The DSCA33/DSCA45 main spec files lost in Dataforth's cryptolocker wipe (which blocked rendering
|
||||
~8,763 certs and prompted "ask John for the spec files") are **recoverable** — the original software
|
||||
published correct DSCA33/45 certs to the **Hoffman API** before the wipe, and our null-skipping
|
||||
pipeline never overwrote them. **Do not ask John for spec files.** Supersedes the FIX2-5 handoff's
|
||||
TODO 2 and the `ad2`-branch memory `project_dsca33_45_spec_gap` (which says "blocked, need John").
|
||||
|
||||
Mined **56 of 58 models** straight from Hoffman into `projects/dataforth-dos/dsca33-45-templates.json`
|
||||
(per model: `accOut`, verbatim 2-line `accHeader`, Final-Test `rows` of name+spec, and a known-good
|
||||
`_srcSerial`). Only 2 niche models have no original anywhere: **DSCA33-1948 (16u)**, **DSCA45-1746 (8u)**.
|
||||
Coverage: ~7,157 units already correct + live on Hoffman (no action); ~1,580 not-yet-uploaded units
|
||||
need rendering from the mined templates + AD2's already-derived slotMaps.
|
||||
|
||||
AD2 handoff + the critical gate: `projects/dataforth-dos/DSCA33-45-HOFFMAN-RECOVERY-2026-06-18.md`.
|
||||
**Critical:** validate each model's render byte-for-byte against its Hoffman original BEFORE enabling
|
||||
live DSCA33/45 rendering — once the renderer returns non-null, the pipeline stops skipping these and
|
||||
will re-push/UPDATE the 7,157 good originals on the next cycle (safe only if the render matches).
|
||||
|
||||
Hoffman read API: `GET https://www.dataforth.com/api/v1/TestReportDataFiles/{serial}` (returns
|
||||
`{SerialNumber,Content,CreatedAtUtc,UpdatedAtUtc}`); creds vault `clients/dataforth/hoffman-product-api`.
|
||||
Miner: `projects/dataforth-dos/tools/mine-hoffman-dsca.py`. AD2 access notes: [[ad2-ssh-mtu-blackhole]].
|
||||
34
.claude/memory/project_dsca33_45_spec_gap.md
Normal file
34
.claude/memory/project_dsca33_45_spec_gap.md
Normal file
@@ -0,0 +1,34 @@
|
||||
---
|
||||
name: project_dsca33_45_spec_gap
|
||||
description: Why DSCA33/DSCA45 datasheets won't render — missing main specs + special layouts, not a bug
|
||||
metadata:
|
||||
type: project
|
||||
---
|
||||
|
||||
In the Dataforth test-datasheet pipeline (deployed `C:\Shares\testdatadb`, AD2 fork `ad2`
|
||||
branch), the **DSCA33** and **DSCA45** model families render null (no datasheet) for a
|
||||
**data-gap** reason, not a code bug:
|
||||
|
||||
- The main spec DAT files we have (`specdata/DSCOUT.DAT`, `DSCMAIN4.DAT`) contain DSCA prefixes
|
||||
30,31,32,34,36,37,38,39,40,41,42,43,47,49 — but **NOT 33 or 45**. DSCA33 appears in **no** DAT
|
||||
file at all; DSCA45 appears only in `DSCFIN.DAT` (the Final-Test layout file, which lacks the
|
||||
`SENTYPE`/`MAXIN`/input-type fields the accuracy block needs). Likely lost in the cryptolocker wipe.
|
||||
- `database/render-datasheet.js` bails (`if (!modelSpecs) return null`) before rendering, so these
|
||||
models produce nothing. ~8,763 PASS certs blocked: DSCA33 = 3,350 (35 models), DSCA45 = 5,413 (23 models).
|
||||
|
||||
The Fix 2 Final-Test rebuild is NOT the blocker — the per-model templates + slotMaps render the
|
||||
Final-Test block correctly for these (proven: DSCA45-01 Final-Test vs golden = 2 trivial diffs with a
|
||||
stubbed spec). The real blockers to publishing them correctly:
|
||||
1. **Missing main specs** → `render-datasheet.js` won't even call the renderer. For DSCA, specs are
|
||||
only needed for the accuracy-block header + model name now (Final-Test is template/slotMap-driven).
|
||||
2. **Special accuracy headers** the sensor-type logic can't produce: DSCA45 = `Frequency (Hz)` /
|
||||
`Output (V)` (frequency-response table); DSCA33 = `Vin (mVAC)` / `Output (VDC)` (AC-RMS module).
|
||||
3. **Non-status template rows**: DSCA45 has informational rows like `Zero-Crossing Input` / `TTL Input`
|
||||
with NO PASS in the golden, but the renderer appends PASS to every empty-spec row (like the old
|
||||
loadNote artifact — see [[project_test_datasheet_pipeline]]).
|
||||
|
||||
Two paths to fix: (a) obtain the authoritative DSCA33/DSCA45 main spec files from Dataforth/engineering
|
||||
(clean, fixes accuracy headers properly — flag to John Lehman), or (b) extend the template approach to
|
||||
capture the accuracy header + special-row handling from the staged originals (self-contained, but a
|
||||
STAGE-4-sized effort across the special layouts). Final-Test data for the matchable models is already
|
||||
ready (slotMaps derived). Related: [[project_pipeline_rebuilt]].
|
||||
12
.gitignore
vendored
12
.gitignore
vendored
@@ -112,3 +112,15 @@ temp/
|
||||
.claude/wiki_staging/*
|
||||
!.claude/wiki_staging/README.md
|
||||
.tmp-*
|
||||
|
||||
# Controller/API scratch files (transient session temp; never commit)
|
||||
.sta.json
|
||||
.dev.json
|
||||
.nc.json
|
||||
.nc2.json
|
||||
.cj_tmp
|
||||
.q*
|
||||
.po_*.json
|
||||
.put*.json
|
||||
.hdr
|
||||
.csrf_tmp
|
||||
|
||||
65
Prompt617.txt
Normal file
65
Prompt617.txt
Normal file
@@ -0,0 +1,65 @@
|
||||
TASK: Diagnose + fully document the Dataforth test-datasheet bug. You are running ON AD2
|
||||
(192.168.0.6), the host of the testdatadb generator + PostgreSQL. Do NOT fix anything yet <20>
|
||||
trace, document, and pinpoint root cause.
|
||||
|
||||
=== THE PROBLEM (reported today, 2026-06-17) ===
|
||||
John Lehman (jlehman@dataforth.com) to Mike: "We are experiencing problems with test
|
||||
datasheets. Column headers are wrong. Some Final Test lines are missing."
|
||||
|
||||
From the thread (apps engineer Peter Iliya):
|
||||
- For the 8B35: the STAGED datasheet "H9553-13" (T: drive, TS-4L/STAGE) has the CORRECT
|
||||
input column label "Temp" <20> but the WEBSITE version "179553-13" shows the WRONG label
|
||||
"resistance". DSCA38 is flagged with the same issue.
|
||||
- Peter CANNOT find 179553-13 as a file on the X: drive or in the DFWDS report <20> because the
|
||||
website version is generated from the DB, not stored as a file.
|
||||
- Core question (Peter): "Is there a reason we list the input for our RTD modules as
|
||||
resistance, but show the temperature instead?"
|
||||
- Trigger: customer Wellbore Integrity (Joseph Swinehart) raised a cal-cert AUDIT discrepancy
|
||||
on 8B35 4-wire RTD certs.
|
||||
|
||||
=== THE KEY POINT (Mike) ===
|
||||
The rendered test sheet EXISTS AS A FILE BEFORE it is ingested into the database. That
|
||||
original/staged file is the ground truth (correct Temp label). Our DB-based regeneration is
|
||||
what introduces the error. FOLLOW THE TRAIL UPSTREAM and FIND WHERE THE ORIGINAL FILE IS.
|
||||
|
||||
=== ARCHITECTURE (per TEST-DATASHEET-PROCESS.md) ===
|
||||
Test stations (TS-01..27, DOS) write .dat logs
|
||||
-> AD1 HISTLOGS: \\ad1\... C:\Shares\test\Ate\HISTLOGS\{log_type}\{model}.DAT (+ per-station \TS-XX\LOGS\)
|
||||
-> Legacy DFWDS (VB6) historically rendered "For_Web" .TXT datasheets (the ORIGINAL rendered files)
|
||||
-> testdatadb (THIS host AD2): Node.js + Express + PostgreSQL 18, service on :3000, dashboard
|
||||
http://localhost:3000/, WinSW wrapper, service account INTRANET\svc_testdatadb
|
||||
-> Hoffman Product API (/api/v1/TestReportDataFiles/bulk, OAuth2) -> public website.
|
||||
|
||||
=== WHAT WE ALREADY FOUND IN THE CODE (verify the DEPLOYED copy matches) ===
|
||||
Generator template: datasheet-exact.js (find the deployed copy on AD2; repo copy is
|
||||
projects/dataforth-dos/datasheet-pipeline/implementation/templates/datasheet-exact.js).
|
||||
- getSensorNum(): RTD -> 7. Input header logic: sensorNum 3-6 (thermocouples) -> " Temp. (C)";
|
||||
sensorNum 7 (RTD) -> " Rin (ohms)"; so RTD is rendered as RESISTANCE. The accuracy VALUE
|
||||
formatting is ALSO sensorNum-driven (temperature format vs resistance format) - so values may
|
||||
be wrong too, not just the label.
|
||||
- Final Test loop: `for (i < dataLines.length && i < parsed.statusEntries.length)` with
|
||||
`if (status.length <= 4) continue` -> rows DROP when statusEntries are misaligned/short.
|
||||
|
||||
=== YOUR DELIVERABLES ===
|
||||
1. FULLY DOCUMENT THE END-TO-END PROCESS (write it to a markdown file), especially the UPSTREAM
|
||||
half: exactly what the test station emits, what the ORIGINAL rendered datasheet file is,
|
||||
WHERE it physically lives (find TS-4L/STAGE, the staged H9553-13, and/or the DFWDS For_Web
|
||||
.TXT), and exactly how testdatadb ingests it (parser, which fields, the .DAT format).
|
||||
2. LOCATE THE ORIGINAL FILE for the 8B35 example (H9553-13 / its For_Web .TXT / the .dat it came
|
||||
from) and for a DSCA38 example.
|
||||
3. DIFF original (correct) vs DB-generated (wrong) for 8B35 RTD and DSCA38: the exact column
|
||||
header difference, any VALUE differences, and which Final Test lines are missing and WHY.
|
||||
4. ROOT-CAUSE LOCALIZATION: is the defect in (a) ingestion/.DAT parsing, (b) the DB data itself,
|
||||
or (c) datasheet-exact.js rendering? Confirm against the original file, not assumptions.
|
||||
5. State the CORRECT RTD output (label = Temp (C); values = temperature, derived how?) per the
|
||||
original, and the precise code change(s) that would fix both the header/values and the
|
||||
dropped Final Test lines. Note any modules where resistance IS the correct label (don't
|
||||
over-correct).
|
||||
|
||||
Resources you have locally: PostgreSQL on AD2 (query testdatadb directly), the deployed Node
|
||||
service + its code, SMB to AD1 (\\ad1\...) and D2TESTNAS (\\192.168.0.9\test, /data/test has
|
||||
DFWDS, Ate, 8BDATA, DSCDATA, etc.). The ClaudeTools repo has the pipeline source under
|
||||
projects/dataforth-dos/.
|
||||
|
||||
Output: the process doc (file) + a diagnosis summary + the proposed fix. Diagnose only <20> no
|
||||
changes to the generator or DB until reviewed.
|
||||
68
STAGE-IMPORT-INSTRUCTIONS.md
Normal file
68
STAGE-IMPORT-INSTRUCTIONS.md
Normal file
@@ -0,0 +1,68 @@
|
||||
# Stage TXT Import Task
|
||||
# Date: 2026-03-28
|
||||
# Context: CTONWTXT.BAT now uploads C:\STAGE\*.TXT from DOS machines to T:\STAGE\%MACHINE%\
|
||||
|
||||
## What happened
|
||||
|
||||
1. CTONWTXT.BAT was never being called -- fixed, now called from CTONW.BAT on every boot
|
||||
2. Destination changed from broken X: (Novell serve.sys check) to T:\STAGE\%MACHINE%\
|
||||
3. DOS 6.22 can't MD on existing dirs without error, so dirs are pre-created on NAS
|
||||
4. All TS-* machine folders pre-created under /data/test/STAGE/ on D2TESTNAS
|
||||
|
||||
## What needs to run
|
||||
|
||||
Save the script below as C:\Shares\testdatadb\import-all-stage.js and run it:
|
||||
|
||||
cd C:\Shares\testdatadb
|
||||
node import-all-stage.js
|
||||
|
||||
## What it does
|
||||
|
||||
- Scans \\D2TESTNAS\test\STAGE\TS-*\*.TXT (~8,100 files across 10 machines)
|
||||
- Parses each TXT datasheet (Date, Model, SN)
|
||||
- Decodes hex-prefix serial numbers for 8.3 filename encoding:
|
||||
- Letter prefix = hex digit: A=10, B=11, C=12, ..., H=17, etc.
|
||||
- Example: H8236-12.TXT has SN: 178236-12 inside the file
|
||||
- Example: A819-1.TXT has SN: A819-1 inside -> decoded to 10819-1
|
||||
- The SN line inside H-prefix files already has the full numeric serial
|
||||
- The SN line inside A-prefix files still has the encoded serial
|
||||
- Cross-references against testdata.db by (serial_number, model_number)
|
||||
- Inserts MISSING records as log_type='SHT' with test_station from folder name
|
||||
- Copies ALL files to X:\For_Web\{decoded_serial}.TXT (the web share)
|
||||
|
||||
## Machines with data
|
||||
|
||||
TS-4L: 3,082 files (largest)
|
||||
TS-4R: 2,741 files
|
||||
TS-1R: 509 files
|
||||
TS-8R: 478 files
|
||||
TS-3R: 435 files
|
||||
TS-11R: 325 files
|
||||
TS-8L: 285 files
|
||||
TS-11L: 248 files
|
||||
TS-27: 10 files (already imported this session)
|
||||
TS-1L: 1 file
|
||||
|
||||
## Serial number encoding (8.3 filename scheme)
|
||||
|
||||
The QuickBASIC ATE software encodes long serial numbers to fit DOS 8.3 filenames.
|
||||
The first two digits get replaced with a hex letter if the serial is too long:
|
||||
|
||||
178236-12 -> H8236-12.TXT (17 -> H, which is char code 72, 72-55=17)
|
||||
10819-1 -> A819-1.TXT (10 -> A, which is char code 65, 65-55=10)
|
||||
|
||||
Decode: letter.charCodeAt(0) - 55 = numeric prefix
|
||||
Only applies if filename starts with [A-Z] followed by digits.
|
||||
|
||||
## TS-27 already done
|
||||
|
||||
10 files from TS-27 were already imported earlier this session into the DB as SHT records.
|
||||
The import script uses INSERT OR REPLACE so re-running is safe.
|
||||
|
||||
## Previous CTONWTXT.BAT issues (resolved)
|
||||
|
||||
- v1.0: Never called, checked for Novell serve.sys, used X: drive parameter
|
||||
- v2.0: Called from CTONW, but used mixed-case "Stage" path -> failed on DOS
|
||||
- v2.1: All uppercase STAGE, but had MD commands that fail on existing dirs
|
||||
- v2.2: Same issue
|
||||
- v2.3: Removed MD entirely, dirs pre-created on NAS. CURRENT VERSION.
|
||||
80
Test Datasheets/weekend-update-draft.md
Normal file
80
Test Datasheets/weekend-update-draft.md
Normal file
@@ -0,0 +1,80 @@
|
||||
Subject: Test Datasheets - Weekend Update: All 73 Quatronix Sheets Generated, Work Order Search Live
|
||||
|
||||
John, Ken,
|
||||
|
||||
Quick update on progress since Friday's email. The pipeline is significantly further along.
|
||||
|
||||
## Quatronix Customer Issue - RESOLVED
|
||||
|
||||
All 73 requested datasheets have been generated (TXT + PDF). The last holdout was SCM5B49-05 (SN 177000-15) — the 5B49DATA.DAT spec file was empty, but John pointed us to 5B49_2.DAT which had the data. All 73 files are ready to send to Peter/Ginger.
|
||||
|
||||
## Model Spec Coverage Expanded
|
||||
|
||||
We went from 751 model specs to 1,470+ by loading additional spec databases:
|
||||
|
||||
| Spec File | Family | Models |
|
||||
|-----------|--------|--------|
|
||||
| 5BMAIN.DAT | SCM5B | 481 |
|
||||
| 5B45DATA.DAT | SCM5B (freq/counter) | 56 |
|
||||
| DB5B48.DAT | SCM5B (multi-bandwidth) | 3 |
|
||||
| 5B49_2.DAT | SCM5B (sample & hold) | 15 |
|
||||
| 8BMAIN.DAT | 8B | 148 |
|
||||
| DSCOUT.DAT | DSCA (output) | 23 |
|
||||
| DSCMAIN4.DAT | DSCA (input) | 391 |
|
||||
| SCTMAIN.DAT | DSCT | 103 |
|
||||
| 7BMAIN.DAT | SCM7B | 276 |
|
||||
|
||||
If there are additional spec files we're missing, let me know the paths and we'll add them.
|
||||
|
||||
## SCM7B Support Added
|
||||
|
||||
The 7B product family is now fully supported in the datasheet generator:
|
||||
- 31 test parameters (vs 20 for SCM5B)
|
||||
- Correct header ("SCM" prefix prepended to model name)
|
||||
- 120VAC Withstand / Hi-Pot (skipped for 7BPT models)
|
||||
- "Packing Check List" with blank fields (vs pre-marked checkboxes on 5B/8B)
|
||||
- "Tested by" and "QC" signature lines
|
||||
- Note: The 7B DAT format (single CSV line) doesn't include individual accuracy test points, so the accuracy table is omitted. Only the Final Test Results section is generated from DAT data.
|
||||
|
||||
## Work Order Search & Linking
|
||||
|
||||
Imported all 33,745 work order status reports from the test station Reports folders:
|
||||
- 63,263 individual test lines parsed (serial number, model, pass/fail, date/time, station)
|
||||
- 2.27 million test records linked to their work orders
|
||||
|
||||
In the web app (http://192.168.0.6:3000):
|
||||
- New "Work Order #" search field — enter a WO number to find all associated test records
|
||||
- Click the WO number in any record's detail view to see the full work order:
|
||||
- All serial numbers tested under that WO
|
||||
- Pass/fail status for each (including retests)
|
||||
- Test program and version used
|
||||
- Test station and timestamps
|
||||
- New work order reports are automatically imported when synced from the NAS
|
||||
|
||||
## Datasheet Formatting Refined
|
||||
|
||||
Compared generated datasheets against originals from the DFWDS archive and fixed column alignment to match the QuickBASIC output:
|
||||
- TAB positions match exactly (parameter names, measured values, spec limits, pass/fail)
|
||||
- Number formatting matches QB PRINT USING (right-justified, correct decimal places)
|
||||
- STR$() behavior replicated (leading space for positive numbers, dropped leading zeros)
|
||||
- Spec limit formatting matches (e.g., "+/- .03 %" not "+/- 0.03 %")
|
||||
|
||||
## View Button Updated
|
||||
|
||||
The "SHEET" button in the web app now shows a styled HTML page that matches the PDF/TXT layout — white page, monospace font, same column alignment. Includes Print and Download PDF buttons.
|
||||
|
||||
## Infrastructure
|
||||
|
||||
- Created domain service account (INTRANET\svc_testdatadb) for the TestDataDB Windows service — resolves the file permission issues we were hitting
|
||||
- Added STAGE folder sync to the NAS sync script — TXT datasheets from DOS machines will now be pulled to AD2 automatically
|
||||
- Work order report import added to sync script — new reports are ingested automatically every 15 minutes
|
||||
|
||||
## Still Open
|
||||
|
||||
1. **Website upload** — The old Uploader.aspx endpoints are dead. Need to determine the new upload mechanism for dataforth.com.
|
||||
2. **STAGE backlog** — ~8,100 TXT files on the NAS from DOS machines need to be processed (script ready, haven't run it yet).
|
||||
3. **Pending ForWeb export** — ~845K records in the database don't have TXT files in For_Web yet (mostly 7B and older records). Can batch-export as needed.
|
||||
|
||||
Let me know if you need anything else.
|
||||
|
||||
Mike
|
||||
76
clients/cascades-tucson/docs/network/network-logging-plan.md
Normal file
76
clients/cascades-tucson/docs/network/network-logging-plan.md
Normal file
@@ -0,0 +1,76 @@
|
||||
# Cascades — Network Logging / Observability Plan (SPEC — build later)
|
||||
|
||||
- **Created:** 2026-06-17 (Howard-Home / claude-main)
|
||||
- **Status:** PLAN ONLY — no infra changes made. For a scheduled build.
|
||||
- **Goal:** Capture + retain a searchable record of **device drops / kicks / disconnects** and the
|
||||
telemetry to **root-cause the ongoing Cascades network issues** (2.4 GHz congestion, sticky
|
||||
clients, roaming/min-RSSI deauths — see `reports/2026-06-16-unifi-full-audit.md`).
|
||||
|
||||
## The problem we found (2026-06-17)
|
||||
- **The UniFi controller is NOT retaining client history.** A 7-day pull of the Cascades site's
|
||||
`stat/event` AND `stat/alarm` returned **zero** records (auth/site fine — client/device queries
|
||||
return data). So when a phone/device drops or is kicked, **nothing is recorded** -> the network
|
||||
is a black box after the fact.
|
||||
- **pfSense logs locally but in tiny circular buffers** (clog) that roll over in hours — no useful
|
||||
history, no search.
|
||||
- => We must **capture events at the source and ship them to a store with retention + search**.
|
||||
pfSense and UniFi are log *sources*; neither is a retention/search platform on its own.
|
||||
|
||||
## Where the collector lives — decision
|
||||
| Candidate | Verdict |
|
||||
|---|---|
|
||||
| **CS-SERVER** | **NO.** Fragile EOL DC (Dell R610, ~16 yr, **degraded OS RAID-1**, single-DC data-loss risk, I/O-bound). Adding syslog ingestion load is unacceptable. |
|
||||
| **pfSense / UniFi alone** | Sources only. pfSense local retention ~hours; UniFi retains ~0 client events. Live view yes, forensics no. |
|
||||
| **Synology cascadesDS (`192.168.0.120`)** | **PREFERRED on-site collector.** DSM up on :5001 (vault `clients/cascades-tucson/synology-cascadesds`). Built-in **Log Center** = a syslog server (retention + search + notifications), no Docker needed. Becoming backup-only anyway -> light syslog duty fits, keeps logs local + off CS-SERVER. |
|
||||
| Jupiter (`172.16.3.20`, ACG office Docker) | Fallback if a richer stack (Graylog/Loki) is wanted; cross-site (Cascades -> office). Use only if on-site Synology is ruled out. |
|
||||
|
||||
**Recommendation:** Synology **Log Center** as the on-site syslog collector. If cascadesDS turns
|
||||
out to be a Plus/x86 model with spare RAM, **Container Manager** can later add **Graylog** or
|
||||
**Grafana Loki** for richer search/dashboards/alerting — but Log Center alone meets the core ask.
|
||||
|
||||
## Sources to configure (ship syslog -> Synology Log Center, UDP/TCP 514)
|
||||
1. **pfSense** (`192.168.0.1`): Status -> System Logs -> Settings -> **Remote Logging**: server =
|
||||
Synology IP:514; select **System, Firewall, DHCP, Gateway**. (DHCP lease grant/expire/decline =
|
||||
device-drop + IP-churn signal; firewall = blocked traffic.)
|
||||
2. **UniFi controller + Cascades APs** (UOS `172.16.3.29`, site `va6iba3v`): Settings -> System ->
|
||||
enable **Remote Logging / syslog** to the Synology, include **client events / debug** so the
|
||||
**APs emit assoc/DEAUTH-with-reason-code + RSSI-at-disconnect + roam** events — the gold data for
|
||||
"who got kicked and why." Confirm AP syslog is forwarded (not just controller app log).
|
||||
3. **(Optional) switches** — port up/down/flap events (the ~25 underspeed ports + 3 offline
|
||||
switches in the audit are suspects).
|
||||
|
||||
## Client time-series snapshotter (fills the controller's history gap)
|
||||
Because the controller isn't keeping client history, add a small **poller** (every 1-2 min) that
|
||||
hits the controller API `/stat/sta` for the Cascades site and appends per-client rows:
|
||||
`ts, mac, hostname, ap, band, channel, rssi, tx_retry%, satisfaction, is_wired`.
|
||||
- **Where to run:** Synology Task Scheduler + a script, or a small container; or cross-site on
|
||||
GuruRMM (`172.16.3.30`) via cron; or a coord-scheduled job. Store as SQLite/CSV (or into the
|
||||
collector if Graylog/Loki is chosen).
|
||||
- **Why:** lets us answer "did device X drop because RSSI cratered / it stuck to a far AP / 2.4 GHz
|
||||
airtime saturated" — correlating drops with the documented RF problems. Pairs with the existing
|
||||
`unifi-wifi` skill collectors (`watch-ap.sh`, `radio-usage.sh`, `neighbor-collect.sh`).
|
||||
|
||||
## Alerting (phase 2)
|
||||
From Log Center (or Graylog/Loki): notify (Discord via `post-bot-alert.sh` / `discord-dm`) on AP
|
||||
reboot, switch-port flap, repeated deauths for a tracked device, or DHCP pool pressure.
|
||||
|
||||
## Retention
|
||||
Target 30-90 days searchable (HIPAA-adjacent network metadata; no PHI in syslog). Size the Synology
|
||||
Log Center archive / volume accordingly; rotate/compress older.
|
||||
|
||||
## Build steps (when scheduled)
|
||||
1. Confirm cascadesDS **model + RAM + DSM version** (determines Log Center-only vs Container Manager
|
||||
for Graylog/Loki). Cred: vault `clients/cascades-tucson/synology-cascadesds`.
|
||||
2. Install/enable **Log Center** (Package Center) -> enable **syslog server** (514), set retention.
|
||||
3. Point **pfSense** remote syslog at it (sources above) — verify receipt.
|
||||
4. Enable **UniFi controller + AP** remote syslog (with client/deauth events) — verify AP deauth
|
||||
events arrive with reason + RSSI.
|
||||
5. Deploy the **client snapshotter** (cron/Task Scheduler) — verify rows accumulating.
|
||||
6. (Optional) Container Manager -> Graylog/Loki+Grafana for dashboards; wire alerting.
|
||||
7. Validate: force a test device off WiFi -> confirm a searchable deauth event with reason + RSSI.
|
||||
|
||||
## Open items
|
||||
- Confirm cascadesDS model/RAM/Docker capability (step 1).
|
||||
- Confirm no PHI traverses syslog (network metadata only) for the HIPAA file.
|
||||
- Decide retention window + alert thresholds.
|
||||
- If on-site is rejected -> fall back to Jupiter (Graylog/Loki) cross-site.
|
||||
@@ -0,0 +1,42 @@
|
||||
# Cascades — Voice (VLAN 30) Device Inventory
|
||||
|
||||
Living tracker of devices migrated onto the isolated **VOICE VLAN 30** (`10.0.30.0/24`).
|
||||
Built/cutover started 2026-06-17. Runbook: `voice-vlan-cutover.md`. PPSK key (Poly WiFi):
|
||||
vault `clients/cascades-tucson/wifi-voice-ppsk`.
|
||||
|
||||
- DHCP pool `10.0.30.100-.250` (dynamic; no reservations). Gateway `10.0.30.1`, DNS `8.8.8.8/1.1.1.1`.
|
||||
- Isolation: internet/cloud-PBX only; blocked from PHI/LAN/VLAN20/mgmt (verified `pfctl -sr`).
|
||||
|
||||
## On VOICE so far
|
||||
|
||||
| Lease IP | MAC | Type | Location / Owner | Status |
|
||||
|---|---|---|---|---|
|
||||
| 10.0.30.201 | e4:e7:49:52:3a:06 | Vertical-Remote desktop (wired, USW-16-PoE p16) | Vertical mgmt jump box (LogMeIn/RDP) | On VOICE; re-leased after power-on |
|
||||
| 10.0.30.202 | 48:25:67:64:8a:88 | Poly (WiFi, CSCNet voice PPSK) | **Accounting Director office — Lauren Hasselman** | On VOICE; **dial tone + outbound call to cell verified** |
|
||||
| 10.0.30.203 | 48:25:67:d0:b8:ac | Poly (WiFi, CSCNet voice PPSK) | **Life Enrichment office 132** | On VOICE |
|
||||
| 10.0.30.204 | 48:25:67:a3:f8:3b | Poly (WiFi, CSCNet voice PPSK) | **Front desk phone** | On VOICE |
|
||||
| 10.0.30.205 | 48:25:67:64:93:34 | Poly (WiFi, CSCNet voice PPSK) | **Front desk courtesy phone** | On VOICE |
|
||||
| 10.0.30.206 | 48:25:67:64:91:ea | Poly (WiFi, CSCNet voice PPSK) | **Kitchen Director — Alyssa** (via Dining Room AP) | On VOICE |
|
||||
| 10.0.30.207 | 48:25:67:64:8f:0b | Poly (WiFi, CSCNet voice PPSK) | **Kitchen phone** (via Kitchen AP) | On VOICE |
|
||||
| 10.0.30.208 | 48:25:67:64:94:ba | Poly (WiFi, CSCNet voice PPSK) | **Chef phone** (via Kitchen AP) | On VOICE |
|
||||
| 10.0.30.209 | 48:25:67:64:89:6e | Poly (WiFi, CSCNet voice PPSK) | **Tamra** (via AP 217) | On VOICE |
|
||||
| 10.0.30.210 | 48:25:67:d0:b1:83 | Poly (WiFi, CSCNet voice PPSK) | **Crystal** (via AP 217) | On VOICE |
|
||||
| 10.0.30.211 | 48:25:67:64:8e:ae | Poly (WiFi, CSCNet voice PPSK) | **Megan** (via AP 217) | On VOICE |
|
||||
| 10.0.30.212 | 48:25:67:64:93:25 | Poly (WiFi, CSCNet voice PPSK) | **Lois Lane** — room 206 | On VOICE |
|
||||
| 10.0.30.213 | 48:25:67:64:93:4f | Poly (WiFi, CSCNet voice PPSK) | **Medtech phone** — room 206 | On VOICE |
|
||||
| 10.0.30.214 | 48:25:67:64:8f:1d | Poly (WiFi, CSCNet voice PPSK) | **Christina Durpas** — room 206 | On VOICE |
|
||||
| 10.0.30.215 | 48:25:67:64:86:aa | Poly (WiFi, CSCNet voice PPSK) | **Veronica Feller** — room 206 | On VOICE |
|
||||
| 10.0.30.216 | 48:25:67:64:92:6b | Poly (WiFi, CSCNet voice PPSK) | **Lupe Sanchez** (via AP 324) | On VOICE |
|
||||
| 10.0.30.217 | 48:25:67:d0:ae:3e | Poly (WiFi, CSCNet voice PPSK) | **Memory Care reception** (via Memcare Nurse Station AP) | On VOICE |
|
||||
| 10.0.30.218 | 48:25:67:d0:af:10 | Poly (WiFi, CSCNet voice PPSK) | **Shelby Trozzi** — MemCare Director | On VOICE |
|
||||
| 10.0.30.219 | 48:25:67:d0:b4:26 | Poly (WiFi, CSCNet voice PPSK) | **Karen Rossini** — room 515 | On VOICE |
|
||||
| 10.0.30.220 | 48:25:67:64:95:6b | Poly (WiFi, CSCNet voice PPSK) | **Christine** (last name ~Nyuda — VERIFY) — room 515 | On VOICE |
|
||||
| 10.0.30.221 | 48:25:67:64:91:cf | Poly (WiFi, CSCNet voice PPSK) | **Salon phone** (via salon AP) | On VOICE |
|
||||
| 10.0.30.222 | 48:25:67:64:81:8e | Poly (WiFi, CSCNet voice PPSK) | **Meredith** — room 140 | On VOICE |
|
||||
| 10.0.30.223 | 48:25:67:64:8f:14 | Poly (WiFi, CSCNet voice PPSK) | **Ashley** — room 103 | On VOICE |
|
||||
|
||||
## Still to migrate
|
||||
- **Poly (WiFi): 22 of 22 DONE** ✓ — all wireless phones migrated to VOICE.
|
||||
- **AudioCodes (8, wired USW-16-PoE ports 1-8): 0 of 8 done** — flip port -> VOICE **+ PoE Power-Cycle** each to re-DHCP. MACs (OUI `00:90:8f`): see runbook appendix.
|
||||
|
||||
> Note: lease IPs are dynamic — a phone may pull a different `.1xx`/`.2xx` on renewal. Track by **MAC + location**, not IP.
|
||||
@@ -91,6 +91,20 @@ restore from on-box auto-backups, eliminating the duplicate dhcpd, **resetting +
|
||||
5. **Optional monitoring:** alert on DHCP-not-completing / duplicate-dhcpd / mass device-disconnect
|
||||
so a future event is caught in minutes.
|
||||
|
||||
## Post-Recovery Casualties / Lessons (updated 2026-06-18)
|
||||
|
||||
- **Kitchen thermal printer (iPad POS ticket printer)** — reported "disconnected from the network"
|
||||
and would not print the morning after. Root cause: it powered up **during the DHCP-down window**
|
||||
of the recovery (duplicate dhcpd + 2nd-floor switch not passing offers), never got an IP, and
|
||||
cached a disconnected state without retrying once the network was healthy. **Power-cycling the
|
||||
printer** forced a fresh DHCP request against the now-healthy network and it resumed printing
|
||||
(confirmed printing iPad tickets). Not a printer or network fault — a device that needed a manual
|
||||
kick post-recovery.
|
||||
- **Recovery-checklist item:** after a network/power outage, **power-cycle any device that booted
|
||||
during the DHCP-down window (printers, POS, IoT, cameras, appliances)** — they frequently do not
|
||||
re-acquire DHCP on their own even after the network is restored. Sweep for stragglers proactively
|
||||
rather than waiting for user reports.
|
||||
|
||||
## Reference Information
|
||||
|
||||
- **pfSense:** `192.168.0.1`, Plus 25.07-RELEASE, ZFS. WAN: Cox (`igc0`, 184.191.143.x). LAN: `igc1`
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
# Cascades — power-outage follow-up: OpenVPN flapping root cause + kitchen printer post-outage casualty
|
||||
|
||||
- **Date:** 2026-06-18
|
||||
- **Machine:** Howard-Home
|
||||
- **Client:** Cascades of Tucson
|
||||
- **Continuation of:** 2026-06-17 power-outage incident (`clients/cascades-tucson/reports/2026-06-17-power-outage-incident.md`)
|
||||
|
||||
## User
|
||||
- **User:** Howard Enos (howard)
|
||||
- **Machine:** Howard-Home
|
||||
- **Role:** tech
|
||||
|
||||
## Session Summary
|
||||
|
||||
Short follow-up session on the 2026-06-17 Cascades power outage. Two items.
|
||||
|
||||
Diagnosed why the Howard-Home OpenVPN Connect tunnel to Cascades pfSense kept disconnecting/
|
||||
reconnecting. Read the pfSense OpenVPN server log (`/var/log/openvpn.log`): the disconnects are
|
||||
caused by a configured **inactivity timeout** — `Howard/... Inactivity timeout (--inactive),
|
||||
exiting` firing at ~5 min (connected 23:23:52 -> dropped 23:28:57 ~= 305s), after which OpenVPN
|
||||
Connect auto-reconnects. Ruled OUT duplicate-CN (0 "will cause previous active session" events),
|
||||
WAN instability (Cox gateway stable since the 20:47 recovery), and TLS/auth errors (clean auth each
|
||||
time; the "IP packet with unknown IP version=0" line is cosmetic). It is a configured idle-disconnect,
|
||||
not a fault. Fix = raise/disable the OpenVPN server `--inactive` timeout (keepalive pings do NOT
|
||||
reset it — `--inactive` measures tunnel data). Proposed, not applied (standing no-change-without-go rule).
|
||||
|
||||
Second: the kitchen thermal printer (iPad POS ticket printer) reported "disconnected from the network"
|
||||
and would not print the morning after the outage; Howard power-cycled it and it resumed printing iPad
|
||||
tickets. Root cause: it powered up DURING the DHCP-down window of the recovery (duplicate dhcpd +
|
||||
2nd-floor switch not passing offers), never got an IP, cached a disconnected state, and did not retry
|
||||
once the network was healthy. The power-cycle forced a fresh DHCP request against the now-healthy
|
||||
network. Not a printer or network fault. Ran a read-only straggler sweep on pfSense (pulled recent
|
||||
dhcpd.log, per-MAC DISCOVER vs ACK): 13/13 active DISCOVER senders are completing, 0 stuck — network
|
||||
healthy. Noted that "gave-up" casualties like the printer are INVISIBLE to a DHCP scan (they stopped
|
||||
requesting), so expect a few more "won't connect" reports today, each fixed by a power-cycle.
|
||||
Updated the incident report with the printer casualty + a recovery-checklist lesson; synced.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **OpenVPN flapping = configured `--inactive` idle timeout, not instability** — diagnosed from the
|
||||
server log rather than guessing; ruled out duplicate-CN / WAN / TLS. Fix proposed (raise/disable inactive), not applied.
|
||||
- **Printer = power-outage DHCP-down-window casualty** — correct fix was the power-cycle (re-DHCP);
|
||||
no network change needed. Captured as a recovery-checklist item (power-cycle devices that booted during the DHCP-down window).
|
||||
- **A DHCP-log scan cannot find gave-up casualties** (they stop requesting) — so the realistic plan is
|
||||
reactive (power-cycle as reports come in), not a proactive scan.
|
||||
|
||||
## Configuration Changes
|
||||
|
||||
- No infrastructure changes. pfSense access was read-only (OpenVPN log, dhcpd log).
|
||||
- Repo: updated `clients/cascades-tucson/reports/2026-06-17-power-outage-incident.md` (added "Post-Recovery
|
||||
Casualties / Lessons" section — kitchen printer + the power-cycle-stragglers checklist item).
|
||||
|
||||
## Infrastructure & Servers
|
||||
|
||||
- **Cascades pfSense** `192.168.0.1`, Plus 25.07. OpenVPN server `ovpns1`, user `Howard` (client IP pool
|
||||
192.168.10.x; this session it got 192.168.10.2). Server has an **`--inactive` idle timeout ~300s** that
|
||||
drops idle clients. WAN = Cox (igc0, 184.191.143.x / dpinger WAN_DHCP + WANCOAX_DHCP). pfSense logs are
|
||||
PLAIN TEXT (read with tail/grep, not clog).
|
||||
- OpenVPN client on Howard-Home: OpenVPN Connect (IV_GUI_VER=OCWindows_3.9.0-5008), public src 98.168.18.21.
|
||||
- **Kitchen thermal printer:** iPad POS ticket printer (exact IP/MAC not captured); resolved by power-cycle.
|
||||
|
||||
## Commands & Outputs
|
||||
|
||||
- OpenVPN flap cause: `grep -i "inactivity timeout" /var/log/openvpn.log` -> `Howard/... Inactivity timeout (--inactive), exiting`; duplicate-CN count = 0.
|
||||
- Straggler sweep: pulled `tail -5000 /var/log/dhcpd.log` locally -> python per-MAC DISCOVER vs ACK -> 13 senders, 13 completing, 0 stuck.
|
||||
|
||||
## Pending / Incomplete Tasks
|
||||
|
||||
- **OpenVPN flapping fix:** raise/disable the pfSense OpenVPN server `--inactive` timeout (proposed; needs go).
|
||||
- **Watch for more post-outage stragglers** (printers/POS/IoT that gave up) — power-cycle each as reported.
|
||||
- Carryover from the outage (unchanged): rotate the exposed Synology credential (vault history commit 1fbc0e1);
|
||||
enable AutoConfigBackup; UPS coverage/runtime/clean-shutdown review; 5GHz Option B + 2.4 Low->Medium bump
|
||||
(plus the auto-channel change still needs a proper data-driven re-plan).
|
||||
|
||||
## Reference Information
|
||||
|
||||
- Incident report: `clients/cascades-tucson/reports/2026-06-17-power-outage-incident.md` (updated).
|
||||
- Prior session logs (same outage): `2026-06-17-howard-cascades-power-outage-recovery-and-5ghz.md`,
|
||||
`2026-06-17-howard-cascades-poly-phone-drops-network-smoothing.md`.
|
||||
- Memory: `reference_pfsense_25_07_ops.md`, `feedback_cascades.md` #4 (no prod change without discussing).
|
||||
- pfSense access: `bash .claude/skills/unifi-wifi/scripts/pfsense-ssh.sh cascades-tucson run "<cmd>"`.
|
||||
@@ -0,0 +1,79 @@
|
||||
## User
|
||||
- **User:** Howard Enos (howard)
|
||||
- **Machine:** Howard-Home
|
||||
- **Role:** tech
|
||||
|
||||
## Session Summary
|
||||
|
||||
Diagnosed slow performance on Lupe Sanchez's workstation **DESKTOP-TRCIEJA** (Cascades of Tucson), reported as the machine being slow when opening Excel files on the desktop. Located the agent via `rmm-search` (GuruRMM agent `c9bf1a2d-bfdc-401e-9cc8-f9e90bb19587`, online, Windows 11 Pro build 22000) and ran a full performance diagnostic over the GuruRMM agent in SYSTEM context.
|
||||
|
||||
The diagnostic identified two converging root causes. First, the hardware is end-of-life: a Gateway ZX6971 all-in-one with an **Intel Core i3-2120 (Sandy Bridge, 2011, 2C/4T)** and **8 GB RAM with only ~2.1 GB free** (27%), running Windows 11 on an unsupported CPU. The SSD itself (DREVO X1 SSD, 224 GB, 44% free) is healthy. Second, and the direct cause of slow Excel opens, the machine is running **two real-time antivirus engines simultaneously**: ACG's Bitdefender Endpoint Security Tools (keep) plus the previous MSP's leftover Datto stack, which bundles a real-time scanner. Both engines scan every file on access, and on a weak 2-core CPU under memory pressure that produces the exact "slow to open Excel" symptom. OneDrive was ruled out — the desktop is local (`C:\Users\LupeSanchez\Desktop`, 50 files / 278 MB, no cloud placeholders), not redirected via Known Folder Move.
|
||||
|
||||
Recon of uninstall entries showed the "Datto AV" reported by SecurityCenter2 is actually the leftover Datto EDR/RMM stack: **Datto RMM (CentraStage)**, **Datto EDR Agent (Infocyte)**, and the bundled **Endpoint Protection SDK (DattoAV)** under `C:\Program Files\infocyte\agent\dattoav\`. The Cascades wiki already flags this same leftover Datto (CentraStage + Infocyte) stack for fleet-wide cleanup. Bitdefender (v8.26.6.644, services running) is properly installed.
|
||||
|
||||
I proposed removing the leftover Datto stack in order (RMM -> EDR -> AV SDK, so Datto RMM cannot re-push the AV) and disabling the Adobe Acrobat PDFMaker Excel add-in. Before acting on a security/management-agent removal I asked Howard to confirm scope. **Howard declined all changes** — the decision is to replace the machine instead. No changes were made to the endpoint; this was diagnostic-only.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **No remediation performed — machine to be replaced.** Given the 2011-era i3-2120 / 8 GB hardware is EOL and unsupported for Win11, Howard opted to order a replacement rather than spend time uninstalling the Datto stack on a box being retired.
|
||||
- **Split the diagnostic into small section scripts.** A combined ~7 KB multi-line PowerShell body failed agent-side (exit -1, "Failed to execute command"); breaking it into <2 KB section scripts (`diag_a/b/c`) each ran cleanly. Logged as friction in `errorlog.md`.
|
||||
- **Ruled out OneDrive/Known Folder Move early** — desktop is local with no cloud placeholders, so the slowness is not on-demand-file hydration.
|
||||
- **Identified dual real-time AV as the direct cause** of slow Excel opens, distinct from the hardware weakness.
|
||||
|
||||
## Problems Encountered
|
||||
|
||||
- **Heredoc quoting failure** building the combined diagnostic inline in Bash ("unexpected EOF") due to embedded single quotes — switched to writing the script to a file and feeding it via `jq --rawfile`.
|
||||
- **Agent rejected the large combined script** (exit -1, "Failed to execute command") despite only ~7 KB and valid syntax; a minimal sanity command ran fine, confirming the agent chokes on a large multi-line `-Command` body. Resolved by splitting into three small section scripts. Logged via `log-skill-error.sh ... --friction`.
|
||||
|
||||
## Configuration Changes
|
||||
|
||||
- **Endpoint (DESKTOP-TRCIEJA): none.** Diagnostic-only; no software removed or settings changed.
|
||||
- Repo: created and then deleted temporary diagnostic scripts in the working tree (`diag_perf.ps1`, `diag_a.ps1`, `diag_b.ps1`, `diag_c.ps1`, `diag_recon.ps1`) — none committed.
|
||||
- `errorlog.md`: appended one `--friction` entry (rmm large-command-body failure).
|
||||
|
||||
## Credentials & Secrets
|
||||
|
||||
None discovered or created this session. RMM auth via existing vault path `infrastructure/gururmm-server.sops.yaml` (unchanged).
|
||||
|
||||
## Infrastructure & Servers
|
||||
|
||||
- **DESKTOP-TRCIEJA** — Lupe Sanchez workstation, Cascades of Tucson, site CascadesTucson.
|
||||
- GuruRMM agent ID: `c9bf1a2d-bfdc-401e-9cc8-f9e90bb19587` (resolve live by hostname; UUIDs change on re-enroll). Agent version 0.6.66.
|
||||
- Hardware: Gateway ZX6971 AIO, serial `DOGDGAA0012200249A6300`, Intel i3-2120 2C/4T, 7.9 GB RAM, DREVO X1 SSD 224 GB (Healthy, SMART PredictFailure=False), C: 97.2 GB free of 222.8 GB.
|
||||
- OS: Windows 11 Pro build 22000 (i3-2120 not Win11-supported).
|
||||
- Active console user: `lupesanchez` (profile `C:\Users\LupeSanchez`).
|
||||
- **Leftover Datto stack (previous MSP, flagged for cleanup):**
|
||||
- Datto RMM (CentraStage) — service `CagService`, uninstall `C:\Program Files (x86)\CentraStage\uninst.exe` (NSIS, `/S`).
|
||||
- Datto EDR Agent (Infocyte) v3.17.1.5409 — uninstall `"C:\Program Files\infocyte\agent\agent.exe" --uninstall`.
|
||||
- Endpoint Protection SDK / DattoAV v1.0.2510.6851 — quiet uninstall `"C:\Program Files\infocyte\agent\dattoav\Endpoint Protection SDK\endpointprotection.exe" uninstallSdk`.
|
||||
- **Bitdefender (ACG, keep):** Endpoint Security Tools v8.26.6.644; services EPSecurityService / EPProtectedService / EPRedline running. Defender in passive state (correct).
|
||||
- **Excel/Office:** Microsoft 365 Apps for business 16.0.20026.20168; `EXCEL.EXE` at `C:\Program Files\Microsoft Office\Root\Office16\`.
|
||||
- Network note: System log shows recurring Event 2505 — duplicate computer name on the network ("another computer on the network has the same name").
|
||||
|
||||
## Commands & Outputs
|
||||
|
||||
- `bash .claude/scripts/rmm-search.sh desktop-trcieja` -> 1 match, Cascades of Tucson, online.
|
||||
- Diagnostic dispatched cmd `c31c6552-1abd-41eb-b269-159cab42360d` (combined script) -> **failed exit -1** "Failed to execute command".
|
||||
- Sanity cmd `acb9eba4...` -> `PONG from DESKTOP-TRCIEJA as nt authority\system`; `quser` -> `lupesanchez console 1 Active`, logon 6/18/2026 9:27 AM.
|
||||
- Section scripts ran clean. Top processes by memory: EXCEL 540 MB, Memory Compression 427 MB, EPSecurityService (Bitdefender) 408 MB / 589 CPU-s, OUTLOOK 349 MB, endpointprotection (DattoAV) 333 MB / 611 CPU-s, AEMAgent (Datto RMM) 132 MB.
|
||||
- AV via SecurityCenter2: Datto AV (multiple states), Windows Defender (0x60100 passive), Bitdefender Endpoint Security Tools.
|
||||
- `Get-MpComputerStatus`: Defender RTP False, SigAge 65535 (Defender stood down — expected with Bitdefender present).
|
||||
- Excel add-ins (user LupeSanchez): Adobe PDFMaker LoadBehavior=3 (loaded), MicrosoftDataStreamerforExcel, PowerPivotExcelClientAddIn.
|
||||
- Disk/stability events (7d): Event 98 x13, Event 153 x4, Event 11 x2 — occasional SATA I/O retries (SMART healthy).
|
||||
- `log-skill-error.sh "rmm" "...large multi-line powershell body fails exit -1; split into <2KB section scripts" --friction` -> logged.
|
||||
- Two `[RMM]` bot alerts posted to #dev-alerts (dispatch + result summary).
|
||||
|
||||
## Pending / Incomplete Tasks
|
||||
|
||||
- **Order replacement workstation for Lupe Sanchez** (Howard's action). EOL i3-2120 / 8 GB.
|
||||
- On the new machine: provision GuruRMM + Bitdefender only; do **not** carry over the Datto stack (CentraStage + Infocyte/DattoAV).
|
||||
- Fleet-wide at Cascades: the leftover Datto (CentraStage + Infocyte) cleanup remains open per the wiki — DESKTOP-TRCIEJA is another instance of it.
|
||||
- Optional follow-ups (now moot if replacing): resolve duplicate computer name on the network (Event 2505); the Acrobat PDFMaker add-in remains enabled.
|
||||
- No wiki/ticket entry created this session (offered, not requested).
|
||||
|
||||
## Reference Information
|
||||
|
||||
- GuruRMM agent: `c9bf1a2d-bfdc-401e-9cc8-f9e90bb19587` (DESKTOP-TRCIEJA).
|
||||
- RMM API: `http://172.16.3.30:3001`; vault `infrastructure/gururmm-server.sops.yaml`.
|
||||
- Diagnostic command IDs: `c31c6552-...` (failed combined), `acb9eba4-...` (sanity), `bae771da-ce6e-4b14-be38-b66c07fd96bf` (recon).
|
||||
- Wiki: `wiki/clients/cascades-tucson.md` (line ~415: leftover Datto RMM/EDR cleanup item).
|
||||
@@ -0,0 +1,73 @@
|
||||
## User
|
||||
- **User:** Howard Enos (howard)
|
||||
- **Machine:** Howard-Home
|
||||
- **Role:** tech
|
||||
|
||||
## Session Summary
|
||||
|
||||
Howard reported that the Synology Drive sync from the Cascades Synology NAS (cascadesDS, 192.168.0.120) to CS-SERVER was not syncing all files/folders — some folders showed empty on the server, specifically the "Server" folder and its "ALdoc" content. Investigated end-to-end across both the CS-SERVER Drive Client (via GuruRMM) and the Synology Drive Server (via direct SSH over the site VPN).
|
||||
|
||||
Established the architecture: a Synology Drive **Client** (v7.5.0.16085) runs on CS-SERVER under the Windows `sysadmin` user, authenticating to the NAS as the DSM account **`Sync`** (uid 1116, home `/var/services/homes/Sync`), one-way **download** (mode:1) of that `Sync` user's "My Drive" (`/volume1/homes/Sync/Drive/`) into `D:\Shares\Main`. The folders Howard saw empty (`Server`, `homes`, `home`, `Managment`, `Sales Dept`, `Downloads`) are **empty stub directories at the source** inside the Sync user's My Drive — leftovers from the prior FreeFileSync/GoodSync migration (`sync.ffs_db`, `_gsdata_` are in the same folder). CS-SERVER was faithfully mirroring empty folders. The "ALdoc" data actually lives at `/volume1/homes/Sync/Drive/Documents/ALDocs` (under Documents, which syncs fine) and is already present on CS-SERVER at `D:\Shares\Main\Documents\ALDocs` (17,694 files). The real `/volume1/Server` share (2,486 files, 1.9 G) is a separate shared folder that is NOT in the Drive sync scope at all.
|
||||
|
||||
Ruled out several false leads: not encrypted/locked shares (data lists fine as admin), not a selective-sync blacklist (blacklist only excludes `/Backup`, `/Moments`, dotfiles, `.lnk/.pst/.swp/.tmp`), and not a dead Drive Server. `synopkg status` falsely reported the package "stopped" (status 263 "failed to get unit status"), but the systemd unit `pkgctl-SynologyDrive` is **active** with `cloud-daemon` running and 6690 listening; the `code: -2`/`bio error` lines in the client log are long-poll fallbacks, not a failed server.
|
||||
|
||||
Howard clarified intent: this is a **file-server migration** (Synology -> CS-SERVER); he needs all department-share data staged and kept current as users are moved, then the Synology retired. He accepted handling user home data separately (users move it into redirected profiles that already back up to the server), which removes `homes` from the bulk sync. With `homes` out of scope, a Synology Drive **Team Folder** sync of the department shares becomes viable. Produced a full setup walkthrough (Admin Console Team Folder enable + low versioning; Drive Client Download-only tasks into a clean `D:\Shares\_SynMigration\<share>` staging area; validation; per-share cutover choreography). No changes were made — diagnosis + plan only.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Use Synology Drive Team Folders (not robocopy) for the department shares**, now that `homes` is handled per-user. Earlier robocopy recommendation was driven by the homes 228 GB limitation; with homes excluded, the native real-time Team Folder path is acceptable and matches "permanent real-time sync."
|
||||
- **Exclude `homes` from the bulk sync** — Synology Drive cannot enable Sync on the `homes`/`home` system shares, and Howard will migrate user home data via redirected profiles instead.
|
||||
- **Sync into NEW empty staging folders (`D:\Shares\_SynMigration\<share>`), never into a live share** — a one-way download task makes the local folder mirror the NAS and could delete/overwrite an existing share's files. Keeps current shares + NTFS/SMB permissions untouched.
|
||||
- **Set Drive version history low/off** on the migration team folders to limit space + indexing load on the aging NAS.
|
||||
- **Per-share cutover rule:** a share is mirrored from Synology OR live on CS-SERVER, never both; disable the download task at cutover so the now-authoritative server copy is not overwritten.
|
||||
- **Pilot on the `Server` share first** (smallest real share, 1.9 G / 2,486 files, and the one that triggered the report) before templating the rest.
|
||||
|
||||
## Problems Encountered
|
||||
|
||||
- **`synopkg status SynologyDrive` falsely reported "stopped"** (status 263, "failed to get unit status") while the service was actually running. Resolved by checking the systemd unit directly: `systemctl is-active pkgctl-SynologyDrive` = active, with `cloud-daemon.exe`/`cloud-monitor` under the slice and 6690 listening.
|
||||
- **Initial RMM grep matched the wrong log** — searching for "Server" hit `auto_updater.log` ("update server"/"Windows Server"). Re-ran against `daemon.log` specifically to get the real sync connection + event lines.
|
||||
- **Local-vs-NAS folder-name mismatch** initially confused the mapping (local `Documents`/`Company Web Docs`/`FD`/`ITSvc` are not `/volume1` share names). Resolved by `find`-ing those folder names on the NAS, which located the real sync root at `/volume1/homes/Sync/Drive/`.
|
||||
- **Background NAS `du` jobs failed (exit 255)** after the temp askpass dir was cleaned mid-run; harmless (read-only) and the needed sizes were already captured.
|
||||
|
||||
## Configuration Changes
|
||||
|
||||
None. Diagnosis and planning only — no changes made to CS-SERVER, the Synology, or the repo (other than this session log).
|
||||
|
||||
## Credentials & Secrets
|
||||
|
||||
- No new credentials discovered or created. Synology admin creds already vaulted at `clients/cascades-tucson/synology-cascadesds.sops.yaml` (username `admin`); used read-only for SSH diagnostics via the unifi-wifi-style SSH_ASKPASS pattern (system OpenSSH).
|
||||
- The Drive Client authenticates to the NAS as DSM account **`Sync`** (uid 1116). Its password is stored inside the Drive Client config on CS-SERVER, not in the vault; not needed for this work.
|
||||
|
||||
## Infrastructure & Servers
|
||||
|
||||
- **CS-SERVER** (192.168.2.254) — GuruRMM agent `c39f1de7-d5b6-45ae-b132-e06977ab1713` (online). Synology Drive Client v7.5.0.16085 installed, runs as Windows `sysadmin`. Client config/logs: `C:\Users\sysadmin\AppData\Local\SynologyDrive\` (logs in `\log`, sync session in `\data\session\2`, filters in `\data\session\2\conf\*.filter`).
|
||||
- **cascadesDS Synology NAS** — 192.168.0.120. DSM 7.2.1-69057. Synology Drive Server **3.5.0-26088**, systemd unit `pkgctl-SynologyDrive` (active), protocol port **6690** (SSL). Reachable directly over the site VPN (ports 22/5000/5001/6690 open).
|
||||
- **Sync architecture:** Drive Client connection conn_id 2, `username:sync`, `mode:1` (one-way download), `is_index_home:0`, `user_is_admin:1`; source `/volume1/homes/Sync/Drive/` -> dest `D:\Shares\Main`.
|
||||
- **NAS shared folders (real data):** `/volume1/Server` (2,486 files, 1.9 G), `/volume1/Management` (13,712 files, 5.5 G), `/volume1/homes` (141,545 files, ~228 G), `/volume1/Public` (~50 G), `/volume1/SalesDept` (~12-23 G), `/volume1/chat`, `/volume1/Activities`, plus legacy `/volume1/Sandra Fish`, `/volume1/pacs`, `/volume1/web`.
|
||||
- **Sync user My Drive (`/volume1/homes/Sync/Drive/`) per-folder file counts (source truth):** Documents=519 top (18,193 recursive, incl. `ALDocs`), Public=49, SalesDept=102, chat=1, Company Web Docs=79, FD=154, ITSvc=1; EMPTY stubs: Server=0, homes=0, home=0, Managment=0, "Sales Dept"=0, Downloads=0.
|
||||
- **ALDocs:** source `/volume1/homes/Sync/Drive/Documents/ALDocs`; present on CS-SERVER at `D:\Shares\Main\Documents\ALDocs` (17,694 files). NOT under any "Server" folder.
|
||||
|
||||
## Commands & Outputs
|
||||
|
||||
- Locate CS-SERVER agent: `bash .claude/scripts/rmm-search.sh cs-server cascades` -> `c39f1de7-...` online.
|
||||
- RMM recon of `D:\Shares\Main` showed empty `Server`/`homes`/`Managment`/`Sales Dept`/`Downloads`/`home` vs populated `Documents`(18 GB)/`SalesDept`(23 GB)/`Public`(2.7 GB)/etc.
|
||||
- Drive Client connection line (daemon.log): `...server_ip:192.168.0.120, server_port:6690, ... username:sync, ... mode:1, ... is_index_home: 0, user_is_admin: 1 ...` and `... 'D:\Shares\Main/Server/New folder' ... is one-way downloading, ignore event.`
|
||||
- SSH askpass pattern (system OpenSSH, no sshpass): `SSH_ASKPASS=<helper> SSH_ASKPASS_REQUIRE=force DISPLAY=:0 ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no admin@192.168.0.120 'sh -s' <<heredoc`.
|
||||
- NAS source truth: `find /volume1/homes/Sync/Drive/Server -type f | wc -l` = 0; `find .../Documents/ALDocs -type f | wc -l` = 26 (top) with many subfolders.
|
||||
- Drive Server real state: `systemctl is-active pkgctl-SynologyDrive` = `active` (synopkg reported stop falsely); `netstat -tlnp | grep 6690` = LISTEN.
|
||||
|
||||
## Pending / Incomplete Tasks
|
||||
|
||||
1. **Confirm in-scope share list** — default `Server, Management, Public, SalesDept, chat, Activities`; decide on legacy `Sandra Fish`/`pacs`/`web`. Note `Culinary, IT, Receptionist, directoryshare` already exist as CS-SERVER shares (exclude unless re-pull wanted).
|
||||
2. **Confirm where ALDocs lives among the REAL shares** (not just the Sync user's My-Drive copy) so a Team Folder definitely captures it before the old My-Drive task is retired. Some curated folders (Company Web Docs, ITSvc, Quickbooks) were found to also exist under `/volume1/Public`.
|
||||
3. **Execute the Team Folder migration** (pending go-ahead): Admin Console enable Team Folders + low versioning; Drive Client Download-only tasks into `D:\Shares\_SynMigration\<share>`; pilot on `Server`, validate, template the rest.
|
||||
4. **Per-share cutover** later: create production SMB share + AD-group NTFS/SMB perms (per `phase2-server-prep.md`), repoint GPO drive maps, disable that share's download task.
|
||||
5. Optional: save the walkthrough as a migration runbook under `clients/cascades-tucson/docs/migration/`.
|
||||
|
||||
## Reference Information
|
||||
|
||||
- Migration plan docs: `clients/cascades-tucson/docs/migration/phase4-synology.md` (§6 Synology transition), `phase2-server-prep.md` (§4c sync, §4d share perms).
|
||||
- Synology DSM: `http://192.168.0.120:5000` (Synology Drive Admin Console for Team Folders + Version Control).
|
||||
- Vault: `clients/cascades-tucson/synology-cascadesds.sops.yaml`.
|
||||
- CS-SERVER Drive Client paths: `C:\Users\sysadmin\AppData\Local\SynologyDrive\{log,data\session\2}`.
|
||||
- Proposed staging root: `D:\Shares\_SynMigration\<share>` (new, separate from live shares and from the existing `D:\Shares\Main` Drive staging).
|
||||
@@ -0,0 +1,139 @@
|
||||
# Cascades — Voice VLAN 30 live migration (all Poly + desktop) + network-logging plan
|
||||
|
||||
## User
|
||||
- **User:** Howard Enos (howard)
|
||||
- **Machine:** Howard-Home
|
||||
- **Role:** tech
|
||||
|
||||
## Session Summary
|
||||
|
||||
Continuation of the 2026-06-17 VOICE VLAN 30 build (see `2026-06-17-howard-voice-vlan30-build.md`).
|
||||
This session executed the live device migration onto the isolated VLAN 30 and produced a spec for
|
||||
network observability. Work spanned the 06-17 -> 06-18 date boundary.
|
||||
|
||||
First, the Vertical-Remote management desktop was moved. Howard set USW-16-PoE port 16 native VLAN
|
||||
to VOICE, but the desktop kept its old `192.168.2.180` lease. Diagnosis (pfSense + UniFi
|
||||
controller) showed nothing misconfigured: re-VLANing a wired port does not bounce the NIC link, so
|
||||
Windows held its old lease and its unicast renewal to the old DHCP server was (correctly) blocked by
|
||||
the VOICE isolation rules. A UniFi client block/unblock is a MAC filter, not a link bounce, so it
|
||||
had no effect. Fixed by bouncing port 16 via the controller API (PUT rest/device port_overrides
|
||||
forward:disabled then restore, preserving ports 1-8) — the desktop re-DHCP'd to 10.0.30.201.
|
||||
|
||||
Second, Howard re-keyed the 22 Poly WiFi phones to the voice PPSK over ~2 hours. As each phone
|
||||
joined, the controller `/stat/sta` was polled to map the new 10.0.30.x lease to the phone's
|
||||
location/owner. A WiFi re-auth is itself a fresh DHCP, so the Poly phones needed no bounce. The
|
||||
first phone (Lauren Hasselman, Accounting Director) was validated end-to-end: dial tone + an
|
||||
outbound call to a cell phone. All 22 Poly phones plus the desktop (23 devices) ended up on VOICE,
|
||||
each pulling a clean lease and isolated from PHI/LAN/VLAN20/mgmt. A living inventory doc was created
|
||||
(`docs/network/voice-phone-inventory.md`) and the wiki Voice-VLAN entry flipped PLANNED -> IN
|
||||
PROGRESS.
|
||||
|
||||
Third, Howard raised the need for network logging to track devices that drop/get kicked and to
|
||||
root-cause the ongoing Cascades network issues. Investigation found the UniFi controller is
|
||||
retaining ZERO client events/alarms for the Cascades site over 7 days, and pfSense logs locally in
|
||||
tiny circular buffers — i.e., drop/kick history is not being captured at all. A "plan only" spec was
|
||||
written (`docs/network/network-logging-plan.md`) recommending the Synology cascadesDS (DSM Log
|
||||
Center syslog server) as the on-site collector (CS-SERVER ruled out as the fragile EOL DC), with
|
||||
pfSense + UniFi/AP syslog as sources and a 1-2 min client snapshotter to fill the controller's
|
||||
history gap.
|
||||
|
||||
Finally, a sync hit a rebase conflict because controller-query scratch files written to the repo
|
||||
CWD (.sta.json etc.) were swept into a commit by `git add -A`, and a stray locked curl.exe held the
|
||||
file. Killed the process, untracked .sta.json, gitignored the temp patterns, and pushed clean.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Desktop cutover via port bounce, not NIC change.** Confirmed desktop is DHCP; the fix for a
|
||||
stuck lease after re-VLAN is a link bounce (port disable/enable or PoE power-cycle), not a NIC
|
||||
reconfig and not a UniFi client block/unblock.
|
||||
- **Read drop/kick state from the UniFi controller, not pfSense SSH**, after pfSense sshd began
|
||||
rate-limiting following many rapid SSH calls. Controller API (`/stat/sta`) was the healthy path
|
||||
and also gives AP/location hints.
|
||||
- **Track phones by MAC + location, not IP** (leases are dynamic; a phone may renew to a different
|
||||
10.0.30.x).
|
||||
- **Network-logging collector = Synology Log Center, NOT CS-SERVER.** CS-SERVER is the fragile
|
||||
EOL/degraded-RAID single DC; adding syslog ingestion is unacceptable. pfSense/UniFi are sources,
|
||||
not retention/search stores. Synology keeps it on-site and off the DC.
|
||||
- **Plan only for logging** (per Howard) — spec written, build scheduled later.
|
||||
|
||||
## Problems Encountered
|
||||
|
||||
- **Desktop stuck on 192.168.2.180 after port moved to VLAN 30** — stale DHCP lease; renewal
|
||||
blocked by VOICE isolation. Resolved by bouncing port 16 via controller API -> re-DHCP to
|
||||
10.0.30.201.
|
||||
- **UniFi controller PUT returned HTTP 403** — UniFi OS requires a CSRF token on writes. Resolved by
|
||||
reading `x-updated-csrf-token` from the login response headers and sending `X-CSRF-Token`.
|
||||
- **pfSense SSH began failing (exit 255)** while ping still succeeded — sshd rate-limiting after many
|
||||
rapid `pfsense-ssh.sh` calls. Switched to the UniFi controller API for subsequent reads.
|
||||
- **Git-Bash `/tmp` path mismatch** — msys `curl -o /tmp/x.json` wrote where Windows python could not
|
||||
read (FileNotFoundError). Switched to CWD-relative scratch files.
|
||||
- **Scratch files committed + rebase blocked** — CWD-relative `.sta.json` got swept into a commit by
|
||||
sync's `git add -A`, and a stray locked `curl.exe` (PID 25252) held the file, blocking the rebase.
|
||||
Killed the process, `git rm --cached .sta.json`, gitignored `.sta.json`/`.dev.json`/`.q*`/etc.,
|
||||
committed, and pushed. (Lesson: write API scratch OUTSIDE the repo or use the ignored `.tmp-` prefix.)
|
||||
- **Earlier errorlog rebase conflict** (concurrent GURU-5070 entry) — resolved keeping both entries.
|
||||
|
||||
## Configuration Changes
|
||||
|
||||
- **Created** `clients/cascades-tucson/docs/network/voice-phone-inventory.md` — living inventory, 23
|
||||
devices on VOICE (desktop + 22 Poly) with MAC/IP/location.
|
||||
- **Created** `clients/cascades-tucson/docs/network/network-logging-plan.md` — observability spec
|
||||
(build later).
|
||||
- **Updated** `clients/cascades-tucson/docs/network/voice-vlan-cutover.md` — added the
|
||||
bounce-to-re-DHCP CRITICAL step; fixed stale NIC-change/OpenVPN/reservation references.
|
||||
- **Updated** `wiki/clients/cascades-tucson.md` — Voice VLAN PLANNED -> IN PROGRESS, two locations.
|
||||
- **Updated** `.claude/memory/MEMORY.md` + created `project_cascades_isolated_vlan_pattern.md` (prior
|
||||
session, synced).
|
||||
- **Updated** `.gitignore` — ignore controller-query scratch patterns; `git rm --cached .sta.json`.
|
||||
- **pfSense (prior session, this thread):** VOICE rule protocol TCP -> Any via PHP config API.
|
||||
- **UniFi:** bounced USW-16-PoE port 16 (disable/restore) via controller API — temporary, restored
|
||||
to exact original (native VOICE, forward:customize).
|
||||
|
||||
## Credentials & Secrets
|
||||
|
||||
- **VOICE PPSK key** `V0!c38863171` — vault `clients/cascades-tucson/wifi-voice-ppsk.sops.yaml`
|
||||
(created prior session, pushed). Entered on all 22 Poly phones this session.
|
||||
- UniFi controller RW: vault `infrastructure/uos-server-network-api-rw` (used for reads + the port
|
||||
bounce). pfSense admin: vault `clients/cascades-tucson/pfsense-firewall`. Synology: vault
|
||||
`clients/cascades-tucson/synology-cascadesds`.
|
||||
|
||||
## Infrastructure & Servers
|
||||
|
||||
- **VOICE VLAN 30:** `10.0.30.0/24`, gw `10.0.30.1` (pfSense `igc1.30`/opt241), DHCP `.100-.250`,
|
||||
DNS `8.8.8.8/1.1.1.1`. Isolation = Guest-clone (any-proto quick blocks to 192.168.0.0/22 +
|
||||
10.0.0.0/8 + 172.16.0.0/12, then pass any).
|
||||
- **UniFi VOICE network** id `6a32e0194e709ad31ad161e6` (VLAN Only). USW-16-PoE mac
|
||||
`d8:b3:70:21:94:5f`, device_id `685f39078e65331c46ef7e90`. UOS `172.16.3.29:11443`, site
|
||||
`va6iba3v`.
|
||||
- **Synology cascadesDS** `192.168.0.120` (DSM up on :5001) — proposed logging collector.
|
||||
- **Jupiter** `172.16.3.20` (Unraid/Docker, hosts UniFi VM) — fallback collector host.
|
||||
|
||||
## Commands & Outputs
|
||||
|
||||
- Controller client poll (mapping phones): `POST /api/auth/login` -> GET
|
||||
`/proxy/network/api/s/va6iba3v/stat/sta`, filter `network==VOICE or vlan==30`.
|
||||
- Port bounce: `PUT /proxy/network/api/s/va6iba3v/rest/device/<id>` body
|
||||
`{"port_overrides":[... port16 forward:disabled ...]}` with `X-CSRF-Token`, then restore.
|
||||
- Drop/kick history check: `POST .../stat/event {"within":168,"_limit":5000}` and `.../stat/alarm`
|
||||
both returned **0** records for Cascades -> controller not retaining client history.
|
||||
- Result: 23 devices on VOICE (`10.0.30.201` desktop + `.202`-`.223` the 22 Poly).
|
||||
|
||||
## Pending / Incomplete Tasks
|
||||
|
||||
- **8 wired AudioCodes** (USW-16-PoE ports 1-8) — flip port -> VOICE + **PoE power-cycle** each to
|
||||
re-DHCP. Not yet done.
|
||||
- **Christine's last name** (room 515, `10.0.30.220`, mac `48:25:67:64:95:6b`) — flagged VERIFY in
|
||||
the inventory (Howard unsure; "~Nyuda").
|
||||
- **Network logging build** — execute `network-logging-plan.md` (step 1: confirm Synology
|
||||
model/RAM/DSM -> Log Center-only vs Container Manager Graylog/Loki).
|
||||
- Confirm phones register to cloud PBX (assumed; dial-tone proven on one) — add Part A 5b pinhole
|
||||
only if a phone fails to register.
|
||||
|
||||
## Reference Information
|
||||
|
||||
- Runbook: `clients/cascades-tucson/docs/network/voice-vlan-cutover.md`
|
||||
- Inventory: `clients/cascades-tucson/docs/network/voice-phone-inventory.md`
|
||||
- Logging plan: `clients/cascades-tucson/docs/network/network-logging-plan.md`
|
||||
- Prior log: `clients/cascades-tucson/session-logs/2026-06/2026-06-17-howard-voice-vlan30-build.md`
|
||||
- Memory: `.claude/memory/project_cascades_isolated_vlan_pattern.md`
|
||||
- Vault PPSK: `clients/cascades-tucson/wifi-voice-ppsk.sops.yaml`
|
||||
@@ -0,0 +1,119 @@
|
||||
# 2026-06-18 — Darrell Delphen — Outlook email links failing (ISP SNI block)
|
||||
|
||||
## User
|
||||
- **User:** Mike Swanson (mike)
|
||||
- **Machine:** GURU-5070
|
||||
- **Role:** admin
|
||||
|
||||
## Session Summary
|
||||
|
||||
Darrell Delphen reported that links in Outlook email would not open on one workstation
|
||||
(DDDOffice072023), failing with "can't reach this page" / `ERR_CONNECTION_ABORTED` against a
|
||||
`url.emailprotection.link` URL, while links opened from Gmail worked fine. The failing host is
|
||||
Intermedia's Email Protection "Safe Link" rewriter — every link in Intermedia-protected mail is
|
||||
rewritten to `https://url.emailprotection.link/...`, so all Outlook-delivered links routed through it.
|
||||
|
||||
Diagnosis was done entirely over GuruRMM against agent `000ed57d-fd05-4001-871c-244f43155c16`. DNS
|
||||
resolved correctly and TCP 443 connected, but the TLS handshake died with SChannel `0x80090326`
|
||||
(SEC_E_ILLEGAL_MESSAGE — "message received was unexpected or badly formatted"). The endpoint's TLS
|
||||
stack was clean: FIPS off, no SCHANNEL protocol/cipher overrides, no cipher-suite GPO, only Windows
|
||||
Defender (no third-party AV/proxy/VPN/LSP/WFP callout). The same node `199.193.205.140` handshook
|
||||
successfully with the real SNI from GURU-5070 on a different network, proving the origin was healthy
|
||||
and the interference was on the endpoint's path. A blast-radius sweep showed only
|
||||
`url.emailprotection.link` failed while `google.com`, `microsoft.com`, `outlook.office365.com`,
|
||||
`cloudflare.com`, `badssl.com`, and even `login.serverdata.net` (same Intermedia infra) all succeeded;
|
||||
MTU was fine. An SNI-varied handshake to the same IP isolated it conclusively: `example.com` and
|
||||
`login.serverdata.net` SNIs succeeded 12/12 while `url.emailprotection.link` failed 12/12, across
|
||||
interleaved source ports — deterministic, SNI-keyed, not flow-hash/LAG. Root cause: a network device
|
||||
on the path performing SNI-based content inspection that corrupted the handshake for that one hostname.
|
||||
|
||||
The gateway turned out to be an ISP-provided Extreme **EXOS** device the client had no login for. The
|
||||
fix path was therefore ISP escalation. An escalation packet was drafted, and Cloudflare WARP was
|
||||
installed on the workstation as an interim workaround — tunneling past the SNI block. With WARP
|
||||
connected, egress moved to Cloudflare (104.28.152.216) and the real-SNI handshake succeeded (TLS 1.2,
|
||||
HTTP 200). The ISP then disabled a "NetIQ" web/URL-filtering feature on the gateway, which cleared the
|
||||
block at the source. After WARP was disconnected the native ISP path was verified working (5/5
|
||||
handshakes + HTTP 200, egress back to 167.89.210.225), so WARP was uninstalled and the machine
|
||||
returned to normal. Work was documented and billed on Syncro ticket #32437 — private technical note,
|
||||
customer-facing/emailed summary, and 1.0h remote labor ($150.00, invoice #1650728058).
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- Diagnosed exclusively via repeated GuruRMM `SslStream`/`Test-NetConnection` probes rather than
|
||||
asking the client to run tools — faster and reproducible.
|
||||
- Used an SNI-varied handshake to the *same fixed IP* as the decisive test. It separated
|
||||
destination/server problems from path interference and proved the block was keyed on the SNI string.
|
||||
- Ran a 12x repeatability test per SNI to rule out a faulty LAG/ECMP member (intermittent, 5-tuple
|
||||
keyed) vs deliberate content matching (deterministic). Result was 0/12 vs 12/12 — deterministic.
|
||||
- Chose Cloudflare WARP as the interim workaround because the block is SNI-based; any tunnel that
|
||||
encrypts egress past the EXOS hides the SNI. WARP is the lightest deploy.
|
||||
- Installed/connected WARP in stages (install, then connect, then verify) so each step could be
|
||||
confirmed before flipping the tunnel, given the agent bounces on network-stack changes.
|
||||
- Emailed the customer a plain-language summary (do_not_email:false) and kept the technical detail in
|
||||
a hidden note.
|
||||
|
||||
## Problems Encountered
|
||||
|
||||
- **Shell state not persisting between Bash calls** — `$TOKEN`/`$RMM` from `rmm-auth.sh` were gone on
|
||||
the next call (first dispatch produced no output). Fixed by `eval "$(rmm-auth.sh)"` inside every Bash
|
||||
invocation.
|
||||
- **PowerShell single-quotes collided with bash single-quoted `SCRIPT='...'`** — embedded
|
||||
`'C:\Program Files\...'` terminated the bash string (`FilesCloudflareCloudflare: command not found`).
|
||||
Fixed by defining the script via a quoted heredoc `SCRIPT=$(cat <<'PS' ... PS)`.
|
||||
- **WARP install/connect bounced the RMM agent** — commands returned `interrupted` ("Agent restarted
|
||||
during execution") because installing/connecting WARP resets the network stack/WFP. The agent
|
||||
auto-reconnected (through WARP after connect); verified state with follow-up commands.
|
||||
- **First WARP uninstall found nothing** — the product registers as **"Cloudflare One Client"**, not
|
||||
"Cloudflare WARP", so the DisplayName filter missed it. Found the GUID
|
||||
`{9E49837E-2971-413F-9587-119FA819E572}` and removed via `msiexec /x`.
|
||||
|
||||
## Configuration Changes
|
||||
|
||||
- **Endpoint DDDOffice072023:** Cloudflare WARP (Cloudflare One Client 2026.4.1390.0) installed,
|
||||
registered, connected, then later disconnected and **fully uninstalled**. Net change to the machine: none.
|
||||
- **ISP gateway (not us):** ISP disabled the "NetIQ" web/URL-filtering feature on the Extreme EXOS device.
|
||||
- No changes to the ClaudeTools repo code.
|
||||
|
||||
## Credentials & Secrets
|
||||
|
||||
None discovered, created, or rotated this session.
|
||||
|
||||
## Infrastructure & Servers
|
||||
|
||||
- **Endpoint:** DDDOffice072023 — Windows, GuruRMM agent `000ed57d-fd05-4001-871c-244f43155c16`
|
||||
(v0.6.66), RMM client "AZ Computer Guru" / site "Discovery test site". LAN gateway 192.168.1.1,
|
||||
ISP egress 167.89.210.225, WARP egress (while active) 104.28.152.216.
|
||||
- **ISP gateway:** Extreme Networks **EXOS** device, ISP-provided/managed (client has no login). Was
|
||||
running a "NetIQ" URL-filtering feature doing SNI inspection.
|
||||
- **Blocked destination:** `url.emailprotection.link` → CNAME `urlrs.gslb.serverdata.net` → A
|
||||
199.193.205.140 (Intermedia Email Protection link-rewriter). GSLB pool also advertises
|
||||
199.193.200.65 / 64.78.20.65 / 162.244.196.65, all TCP-dead from any network (not an ISP block).
|
||||
- **GuruRMM API:** http://172.16.3.30:3001 (JWT via `rmm-auth.sh`).
|
||||
|
||||
## Commands & Outputs
|
||||
|
||||
- Decisive SNI test (same IP, varied SNI), via RMM PowerShell:
|
||||
- SNI `url.emailprotection.link` → `0x80090326` SEC_E_ILLEGAL_MESSAGE (0/12 ok)
|
||||
- SNI `example.com` / `login.serverdata.net` → TLS 1.2 AES256 (12/12 ok)
|
||||
- Off-network control (GURU-5070) to 199.193.205.140 with real SNI → OK TLS 1.2.
|
||||
- Post-ISP-fix native verify: real-SNI handshake 5/5 OK + `Invoke-WebRequest` HEAD → HTTP 200,
|
||||
egress 167.89.210.225.
|
||||
- WARP removal: `msiexec /x {9E49837E-2971-413F-9587-119FA819E572} /qn /norestart` → exit 0;
|
||||
warp-svc/warp-cli/install-dir/uninstall-entry all gone.
|
||||
|
||||
## Pending / Incomplete Tasks
|
||||
|
||||
- None functional. Block is resolved at the ISP. If the issue recurs, suspect the EXOS "NetIQ"
|
||||
feature being re-enabled — re-run the SNI-varied handshake test to confirm.
|
||||
- Optional: if WARP is ever rolled to more machines as a workaround, harden it (force auto-connect,
|
||||
lock client) and note egress moves to Cloudflare (breaks office-static-IP allowlisting).
|
||||
|
||||
## Reference Information
|
||||
|
||||
- **Syncro ticket:** #32437 (id 112766479) — https://computerguru.syncromsp.com/tickets/112766479
|
||||
- Private note id 419714810; public/emailed summary id 419714813
|
||||
- Line item id 42925426 (Labor - Remote Business 1.0h @ $150)
|
||||
- Invoice #1650728058, total $150.00; status Invoiced
|
||||
- **Customer:** Darrell Delphen (Syncro customer_id 35996725), no prepaid block.
|
||||
- **SChannel error:** 0x80090326 = SEC_E_ILLEGAL_MESSAGE (handshake message malformed/unexpected) —
|
||||
signature of in-path TLS/SNI tampering when paired with same-IP success off-network.
|
||||
3
clients/dataforth/.gitignore
vendored
3
clients/dataforth/.gitignore
vendored
@@ -1,3 +1,6 @@
|
||||
|
||||
# plaintext credential note - never commit
|
||||
Oauth.txt
|
||||
|
||||
# Large local-only binary artifacts (WizTree disk scans, dumps) - keep on the machine, never commit
|
||||
local-artifacts/
|
||||
|
||||
246
clients/dataforth/CLAUDE.dataforth.md
Normal file
246
clients/dataforth/CLAUDE.dataforth.md
Normal file
@@ -0,0 +1,246 @@
|
||||
# ClaudeTools on AD2 (Dataforth Domain Controller)
|
||||
|
||||
## Identity
|
||||
|
||||
This is the AD2 workstation instance of ClaudeTools. This machine is a Windows Server on the Dataforth LAN (192.168.0.6). Your scope is Dataforth-only -- you do not need context about other clients.
|
||||
|
||||
## NO EMOJIS
|
||||
|
||||
Use ASCII markers: [OK], [ERROR], [WARNING], [SUCCESS], [INFO]
|
||||
|
||||
---
|
||||
|
||||
## Git & Sync
|
||||
|
||||
### Gitea Credentials (no 1Password on this machine)
|
||||
- URL: https://git.azcomputerguru.com
|
||||
- Username: mike@azcomputerguru.com
|
||||
- Password: Gptf*77ttb123!@#-git
|
||||
- URL-encoded password: Gptf%2A77ttb123%21%40%23-git
|
||||
- API Token: 9b1da4b79a38ef782268341d25a4b6880572063f
|
||||
- Remote: https://mike%40azcomputerguru.com:Gptf%2A77ttb123%21%40%23-git@git.azcomputerguru.com/azcomputerguru/claudetools.git
|
||||
|
||||
### Branch: ad2
|
||||
This machine operates on the `ad2` branch. The main workstation merges into main.
|
||||
|
||||
### /save behavior
|
||||
Save session logs to `session-logs/YYYY-MM-DD-session-ad2.md` (note the -ad2 suffix).
|
||||
After saving, commit and push to origin/ad2.
|
||||
|
||||
### /sync behavior
|
||||
```
|
||||
git fetch origin
|
||||
git rebase origin/main
|
||||
git push origin ad2
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Dataforth Network
|
||||
|
||||
| Host | IP | Role | Notes |
|
||||
|------|-----|------|-------|
|
||||
| AD1 | 192.168.0.27 | Primary DC | Disk at 90%, C:\Engineering = 787 GB |
|
||||
| **AD2** | **192.168.0.6** | **This machine** | Secondary DC, TestDataDB, file shares |
|
||||
| D2TESTNAS | 192.168.0.9 | SMB1 proxy for DOS | Debian 13, Samba, SSH root/Paper123!@#-nas |
|
||||
| UDM | 192.168.0.254 | Gateway/Router | UniFi Dream Machine |
|
||||
| ESXi-122 | 192.168.0.122 | Hypervisor | ESXi |
|
||||
| ESXi-124 | 192.168.0.124 | Hypervisor | ESXi |
|
||||
| DOS stations | TS-01 to TS-30+ | Test stations | DOS 6.22, QuickBASIC ATE software |
|
||||
|
||||
### Credentials
|
||||
- AD Sysadmin: INTRANET\sysadmin / Paper123!@#
|
||||
- D2TESTNAS SSH: root@192.168.0.9 / Paper123!@#-nas
|
||||
- D2TESTNAS Samba: guest access (no password)
|
||||
- WINS/NPS: 192.168.0.27:1812/1813
|
||||
- M365 Tenant: 7dfa3ce8-c496-4b51-ab8d-bd3dcd78b584
|
||||
- Rsync daemon (NAS): port 873, module "test", user rsync / IQ203s32119
|
||||
|
||||
---
|
||||
|
||||
## Local Resources
|
||||
|
||||
| Resource | Path |
|
||||
|----------|------|
|
||||
| TestDataDB app | C:\Shares\testdatadb\ |
|
||||
| Test database | C:\Shares\testdatadb\database\testdata.db (SQLite, 2.2M+ records) |
|
||||
| TestDataDB API | http://localhost:3000 |
|
||||
| Parsers | C:\Shares\testdatadb\parsers\ (multiline.js, csvline.js, shtfile.js, spec-reader.js) |
|
||||
| Templates | C:\Shares\testdatadb\templates\datasheet-exact.js |
|
||||
| Import script | C:\Shares\testdatadb\database\import.js |
|
||||
| Export script | C:\Shares\testdatadb\database\export-datasheets.js |
|
||||
| Stage import | C:\Shares\testdatadb\import-all-stage.js |
|
||||
| NAS share | \\D2TESTNAS\test (mapped as T:) |
|
||||
| Datasheets share | X:\For_Web |
|
||||
| ProdSW (BAT files) | C:\Shares\test\COMMON\ProdSW\ |
|
||||
| Sync script | C:\Shares\test\scripts\Sync-FromNAS.ps1 (bidirectional, 15-min schedule) |
|
||||
|
||||
---
|
||||
|
||||
## DOS Update System - Batch Files
|
||||
|
||||
### Boot Sequence on DOS Machines
|
||||
```
|
||||
AUTOEXEC.BAT (v4.1)
|
||||
-> STARTNET.BAT (v2.0) -- init network, map T: and X: drives
|
||||
-> ATESYNC.BAT
|
||||
-> CTONW.BAT (v5.0) -- upload test data to network
|
||||
-> CTONWTXT.BAT (v2.3) -- upload C:\STAGE\*.TXT to T:\STAGE\%MACHINE%
|
||||
-> NWTOC.BAT (v5.0) -- download updates from network
|
||||
```
|
||||
|
||||
### Current Production Versions (on AD2 & NAS)
|
||||
| File | Version | Last Update | Purpose |
|
||||
|------|---------|-------------|---------|
|
||||
| AUTOEXEC.BAT | v4.1 | 2026-03-12 | Startup config |
|
||||
| STARTNET.BAT | v2.0 | 2026-01-20 | Network init |
|
||||
| NWTOC.BAT | v5.0 | 2026-03-16 | Download updates from network |
|
||||
| CTONW.BAT | v5.0 | 2026-03-28 | Upload test data (5 steps with echo) |
|
||||
| CTONWTXT.BAT | v2.3 | 2026-03-28 | Upload Stage TXT files (no MD, dirs pre-created) |
|
||||
| CHECKUPD.BAT | v1.3 | 2026-01-20 | Check for updates |
|
||||
| UPDATE.BAT | v2.3 | 2026-01-20 | Full system backup |
|
||||
| STAGE.BAT | v1.0 | Original | Stage system file updates |
|
||||
| DEPLOY.BAT | v1.0 | 2026-01-20 | One-time deployment installer |
|
||||
|
||||
### DOS 6.22 Compatibility Rules
|
||||
- NO `IF NOT` -- unreliable on DOS 6.22. Use positive `IF EXIST` with GOTO
|
||||
- NO `IF /I` (case-insensitive compare)
|
||||
- NO `FOR /F` loops
|
||||
- NO `%COMPUTERNAME%` -- use `%MACHINE%` (set during DEPLOY)
|
||||
- `XCOPY /D` requires date parameter (`/D:mm-dd-yy`)
|
||||
- `MD` fails with error on existing directories -- pre-create dirs server-side
|
||||
- `COPY` without `/Y` hangs on overwrite prompts
|
||||
- All paths UPPERCASE for Samba compatibility
|
||||
- Line endings MUST be CRLF (0D 0A)
|
||||
|
||||
---
|
||||
|
||||
## Serial Number Encoding (DOS 8.3 filenames)
|
||||
|
||||
QuickBASIC ATE encodes long serial numbers for 8.3 filenames:
|
||||
```
|
||||
First 2 digits replaced with hex letter if serial too long:
|
||||
178236-12 -> H8236-12.TXT (17 -> H, charCode 72 - 55 = 17)
|
||||
10819-1 -> A819-1.TXT (10 -> A, charCode 65 - 55 = 10)
|
||||
|
||||
Decode: letter.charCodeAt(0) - 55 = numeric prefix
|
||||
Only applies when filename starts with [A-Z] followed by digits.
|
||||
|
||||
H-prefix files have decoded SN inside the file (SN: 178236-12)
|
||||
A-prefix files have encoded SN inside the file (SN: A819-1) -- must decode to 10819-1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Test Datasheet Pipeline
|
||||
|
||||
### 5-Stage Architecture
|
||||
1. **DOS Test Programs** -> Write DAT files to C:\ATE\*LOG\ and TXT to C:\STAGE\
|
||||
2. **Boot Upload** -> CTONW.BAT copies DAT to T:\%MACHINE%\LOGS\, CTONWTXT copies TXT to T:\STAGE\%MACHINE%
|
||||
3. **NAS <-> AD2 Sync** -> Rsync every 15 min (Sync-FromNAS.ps1 scheduled task)
|
||||
4. **TestDataDB Import** -> import.js parses DAT into SQLite; export-datasheets.js generates TXT to X:\For_Web
|
||||
5. **Web Share** -> X:\For_Web\ holds validated datasheets (501K+ files)
|
||||
|
||||
### import-all-stage.js (ready to run)
|
||||
Located at `C:\Shares\testdatadb\import-all-stage.js`. Processes ~8,100 TXT files:
|
||||
- Scans \\D2TESTNAS\test\STAGE\TS-*\*.TXT
|
||||
- Decodes hex-prefix serial numbers
|
||||
- Cross-references testdata.db by (serial_number, model_number)
|
||||
- Inserts missing records as log_type='SHT'
|
||||
- Copies to X:\For_Web\{decoded_serial}.TXT
|
||||
|
||||
```
|
||||
cd C:\Shares\testdatadb
|
||||
node import-all-stage.js
|
||||
```
|
||||
|
||||
### Machine data volumes in STAGE
|
||||
| Machine | Files |
|
||||
|---------|-------|
|
||||
| TS-4L | 3,082 |
|
||||
| TS-4R | 2,741 |
|
||||
| TS-1R | 509 |
|
||||
| TS-8R | 478 |
|
||||
| TS-3R | 435 |
|
||||
| TS-11R | 325 |
|
||||
| TS-8L | 285 |
|
||||
| TS-11L | 248 |
|
||||
| TS-27 | 10 (already imported) |
|
||||
| TS-1L | 1 |
|
||||
|
||||
### Web Share Layout (X:\)
|
||||
- X:\For_Web -- Validated datasheets (production)
|
||||
- X:\For_Web_PDF -- PDF versions (4.7K files)
|
||||
- X:\Test_Datasheets -- Incoming/staging
|
||||
- X:\Bad_Datasheets -- Invalid files (18K)
|
||||
- X:\Datasheets_Log -- Processing logs
|
||||
|
||||
---
|
||||
|
||||
## Known Issues & Pending Work
|
||||
|
||||
### HIGH PRIORITY
|
||||
1. **Run import-all-stage.js** -- 8,100 TXT files need cross-referencing and ingestion
|
||||
2. **Website Upload Replacement** -- Old ASP.NET endpoints (Uploader.aspx) return 404. Need new approach.
|
||||
3. **7B Series Datasheets** -- ~830K records can't generate datasheets (missing 7BMAIN.DAT spec file). Check ENGR share.
|
||||
4. **Service Permissions** -- testdatadb runs as SYSTEM, causing file permission issues. Change to INTRANET\sysadmin.
|
||||
|
||||
### MEDIUM PRIORITY
|
||||
5. **C2 IP Blocking** -- iptables rules added to UDM for 80.76.49.18 and 45.88.91.99. Need permanent rules in UniFi UI.
|
||||
6. **MFA Enforcement** -- 19/38 users ready. Report-only until April 4, 2026. Monitor registration.
|
||||
7. **Joel Lohr Account** -- Retiring March 31. Disable account post-retirement. Auto-reply set to Dan Center.
|
||||
|
||||
---
|
||||
|
||||
## Security Incident (2026-03-27)
|
||||
|
||||
**DF-JOEL2 (192.168.0.143) compromised via phishing:**
|
||||
- Joel Lohr clicked phishing link in personal Yahoo email
|
||||
- ScreenConnect C2 installed, "Angel Raya" connected remotely
|
||||
- Two C2 backdoors deployed via PowerShell
|
||||
- C2 IPs: 80.76.49.18, 45.88.91.99 (AS399486, suspended by host)
|
||||
- IC3 Complaint: 1c32ade367084be9acd548f23705736f
|
||||
- ConnectWise Case: 03464184
|
||||
- **Remediation complete:** IPs blocked, 3 rogue clients removed, password reset, sessions revoked
|
||||
- **No lateral movement detected** (32 machines scanned clean)
|
||||
|
||||
---
|
||||
|
||||
## Key Contacts
|
||||
|
||||
| Person | Email | Role |
|
||||
|--------|-------|------|
|
||||
| John Lehman | jlehman@dataforth.com | Engineering, QB code, test specs |
|
||||
| Dan Center | dcenter@dataforth.com | Operations (replacing Joel) |
|
||||
| Peter Iliya | pIliya@dataforth.com | Applications Engineer |
|
||||
| AJ | dataforthgit@... | Engineering contact |
|
||||
| Ken Hoffman | (unresponsive) | TestDataSheetUploader author |
|
||||
| Georg Haubner | ghaubner@dataforth.com | Has pre-crypto backup on D: drive |
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference Commands
|
||||
|
||||
```powershell
|
||||
# Check BAT files on NAS
|
||||
ssh root@192.168.0.9 'ls -la /data/test/COMMON/ProdSW/'
|
||||
|
||||
# Trigger NAS sync
|
||||
Start-ScheduledTask -TaskName 'Sync-FromNAS'
|
||||
|
||||
# Check sync log
|
||||
Get-Content 'C:\Shares\test\scripts\sync-from-nas.log' -Tail 20
|
||||
|
||||
# Check TestDataDB health
|
||||
curl http://localhost:3000/health
|
||||
|
||||
# Query test records
|
||||
node -e "const db=require('better-sqlite3')('C:\\Shares\\testdatadb\\database\\testdata.db',{readonly:true});console.log(db.prepare('SELECT COUNT(*) as cnt FROM test_records').get())"
|
||||
|
||||
# Check Stage files on NAS
|
||||
ssh root@192.168.0.9 'find /data/test/STAGE -name "*.TXT" | wc -l'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** 2026-03-29
|
||||
18
errorlog.md
18
errorlog.md
@@ -17,8 +17,26 @@ Categories (the `[type]` tag): _(none)_ = skill/command execution failure ·
|
||||
|
||||
<!-- Append entries below this line -->
|
||||
|
||||
2026-06-18 | Howard-Home | git/sync-temp-files | [friction] controller-query scratch (.sta.json/.dev.json/.q*) written to repo CWD got swept into the commit by sync.sh 'git add -A', then a stray locked .sta.json blocked the rebase. Fix: write API scratch OUTSIDE the repo (or use the already-ignored .tmp- prefix); gitignored the patterns [ctx: ref=howard-home /tmp friction family]
|
||||
|
||||
2026-06-18 | Howard-Home | rmm | [friction] agent returns exit -1 'Failed to execute command' on a ~7KB multi-line powershell body sent as one command; split into <2KB section scripts and each ran fine [ctx: host=DESKTOP-TRCIEJA agent=0.6.66]
|
||||
|
||||
2026-06-18 | GURU-5070 | coord/ad2-comms | [correction] tried to coordinate with the AD2 session via coord API msg+lock; AD2 is network-isolated (Gitea only, no coord API) so those were no-ops. ALL inter-session comms with AD2 must go via git /sync (committed notes/docs).
|
||||
|
||||
2026-06-18 | GURU-5070 | syncro | comment POST piped straight to jq failed with 'jq: parse error: Invalid numeric literal at line 1 col 10' and left it AMBIGUOUS whether the note posted (GET-verify showed it had NOT); per no-retry rule had to GET first, then re-post. Robust pattern that worked: jq -n payload to a file, POST with --data-binary @file, capture response to a file, then GET-verify by subject. Skill's curl|jq comment pattern should adopt this. [ctx: ticket=32441 skill=syncro pattern=curl-pipe-jq]
|
||||
|
||||
2026-06-18 | GURU-5070 | post-bot-alert | Discord POST failed (non-200/unreachable) [ctx: channel=#bot-alerts http=400 resp={"message": "The request body contains invalid JSON.", "code": 50109}]
|
||||
|
||||
2026-06-18 | GURU-5070 | ssh/ad2 | [correction] attributed AD2 SSH timeouts to a flaky VPN tunnel + my rapid scp/ssh bursts; real cause = OpenVPN adapter MTU 1500 vs tunnel PMTU ~1424 -> TCP MSS blackhole that drops bulk/scp segments (DF set) while small cmds pass. Fix: tunnel adapter MTU 1400 [ctx: ref=feedback_prefer_ssh_over_rmm]
|
||||
|
||||
2026-06-18 | GURU-5070 | bash/env | [friction] /tmp curl-write then Windows-python read mismatch; wrote .claude/tmp + absolute path fixed it [ctx: ref=feedback_tmp_path_windows]
|
||||
|
||||
2026-06-18 | Howard-Home | pfsense-ssh/logs | [friction] used clog on pfSense 25.07 logs (now plain-text ASCII) -> empty output -> wrongly concluded DHCP log was empty / dhcpd not serving; cost a hypothesis. Read pfSense 25.07 logs with tail/grep/cat directly, NOT clog [ctx: ref=reference_pfsense_25_07_ops client=cascades-tucson]
|
||||
|
||||
2026-06-18 | AD2 | vault | real vault.sh not found at resolved vault_path; vault read failed [ctx: path=D:/vault/scripts/vault.sh]
|
||||
|
||||
2026-06-18 | AD2 | vault | real vault.sh not found at resolved vault_path; vault read failed [ctx: path=D:/vault/scripts/vault.sh]
|
||||
|
||||
2026-06-17 | GURU-5070 | mailbox/365-mail | [correction] claimed in a prior session that /mailbox skill + memories were repointed off the deleted fabb3421 to the 365-mail suite, but mailbox.md still hardwired fabb3421 (token 401 AADSTS700016). Correct app is the dedicated ComputerGuru Mailbox app 1873b1b0 via get-token.sh 'mailbox' tier (cert auth); repointed mailbox.md + feedback_365_remediation_tool.md 2026-06-17. Lesson: verify the edit actually landed before reporting it done.
|
||||
|
||||
2026-06-17 | Howard-Home | wiki-compile/coord | [friction] skill doc Phase 6 shows 'lock release claudetools wiki/<type>/<slug>' but coord.py takes 'lock release <id>'; wasted a round-trip. Capture the lock id from claim output and release by id. [ctx: ref=wiki-compile-skill]
|
||||
|
||||
54
generated-129093-16.TXT
Normal file
54
generated-129093-16.TXT
Normal file
@@ -0,0 +1,54 @@
|
||||
DATAFORTH CORPORATION Phone: (520) 741-1404
|
||||
3331 E. Hemisphere Loop Fax: (520) 741-0762
|
||||
Tucson, AZ 85706 USA email: info@dataforth.com
|
||||
|
||||
TEST DATA SHEET
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
Date: 05-15-2018
|
||||
Model: SCM5B38-05
|
||||
SN: 129093-16
|
||||
|
||||
ACCURACY TEST
|
||||
|
||||
Calculated Measured
|
||||
Vin (mV) Vout (V) Vout (V)* Error (%) Status
|
||||
========== ========== ========== ========= ========
|
||||
-20.000 -5.000 -5.000 -0.005 PASS
|
||||
-9.999 -2.500 -2.499 +0.002 PASS
|
||||
-0.001 -0.000 +0.001 +0.016 PASS
|
||||
+10.001 +2.500 +2.502 +0.018 PASS
|
||||
+20.000 +5.000 +5.000 +0.003 PASS
|
||||
|
||||
FINAL TEST RESULTS
|
||||
|
||||
Parameter Measured Value Specification Status
|
||||
======================== =============== ===================== ======
|
||||
Supply Current, Nom 29.0 mA < 75 mA PASS
|
||||
Supply Current, Max 130.6 mA < 194 mA PASS
|
||||
Output Resistance 22 ohms < 55 ohms PASS
|
||||
Exc. Voltage 10.001 V 10.0+/-0.003 V PASS
|
||||
Exc. Load Reg. 5 ppm/mA +/-11 ppm/mA PASS
|
||||
Vout Reg. w/ Load 0.0 % +/-0.1 % PASS
|
||||
Exc. Current Limit 54.4 mA < 63 mA PASS
|
||||
Linearity 0.010 % +/- 0.03 % PASS
|
||||
Accuracy 0.018 % +/- 0.08 % PASS
|
||||
Supply Sensitivity 0.8 uV/% +/- 2.4 uV/% PASS
|
||||
Frequency Response 29.2 dB 27+/-7 dB PASS
|
||||
Output Noise 566 uVrms < 4000 uVrms PASS
|
||||
240 VAC Withstand PASS
|
||||
Hi-Pot PASS
|
||||
_______________________________________________________________________
|
||||
Check List
|
||||
|
||||
Module Appearance: __X__ Mounting Screw: __X__
|
||||
|
||||
Pins Straight: __X__ Module Header: __X__
|
||||
|
||||
|
||||
It is hereby certified that the above product is in conformance with
|
||||
all requirements to the extent specified. This product is not
|
||||
authorized or warranted for use in life support devices and/or systems.
|
||||
|
||||
* NIST traceable calibration certificates support Measured Value data.
|
||||
Calibration services are available through ANSI/NCSL Z540-1 and
|
||||
ISO Guide 25 Certified Metrology Labs.
|
||||
53
generated-v2-129093-16.TXT
Normal file
53
generated-v2-129093-16.TXT
Normal file
@@ -0,0 +1,53 @@
|
||||
DATAFORTH CORPORATION Phone: (520) 741-1404
|
||||
3331 E. Hemisphere Loop Fax: (520) 741-0762
|
||||
Tucson, AZ 85706 USA email: info@dataforth.com
|
||||
|
||||
TEST DATA SHEET
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
Date: 05-15-2018
|
||||
Model: SCM5B38-05
|
||||
SN: 129093-16
|
||||
|
||||
ACCURACY TEST
|
||||
|
||||
Calculated Measured
|
||||
Vin (mV) Vout (V) Vout (V)* Error (%) Status
|
||||
========== ========== ========== ========= ========
|
||||
-20.000 -5.000 -5.000 -0.005 PASS
|
||||
-9.999 -2.500 -2.499 +0.002 PASS
|
||||
-0.001 -0.000 +0.001 +0.016 PASS
|
||||
+10.001 +2.500 +2.502 +0.018 PASS
|
||||
+20.000 +5.000 +5.000 +0.003 PASS
|
||||
|
||||
FINAL TEST RESULTS
|
||||
|
||||
Parameter Measured Value Specification Status
|
||||
======================= =============== ===================== ======
|
||||
Supply Current, Nom 29.0 mA < 75 mA PASS
|
||||
Supply Current, Max 130.6 mA < 194 mA PASS
|
||||
Output Resistance 22 ohms < 55 ohms PASS
|
||||
Exc. Voltage 10.001 V 10.0+/-0.003 V PASS
|
||||
Exc. Load Reg. 5 ppm/mA +/-11 ppm/mA PASS
|
||||
Vout Reg. w/ Load 0.0 % +/-0.1 % PASS
|
||||
Exc. Current Limit 54.4 mA < 63 mA PASS
|
||||
Linearity 0.010 % +/- 0.03 % PASS
|
||||
Accuracy 0.018 % +/- 0.08 % PASS
|
||||
Supply Sensitivity 0.8 uV/% +/- 2.4 uV/% PASS
|
||||
Frequency Response 29.2 dB 27+/-7 dB PASS
|
||||
Output Noise 566 uVrms < 4000 uVrms PASS
|
||||
240 VAC Withstand PASS
|
||||
Hi-Pot PASS
|
||||
_______________________________________________________________________
|
||||
Check List
|
||||
|
||||
Module Appearance: __X__ Mounting Screw: __X__
|
||||
|
||||
Pins Straight: __X__ Module Header: __X__
|
||||
|
||||
It is hereby certified that the above product is in conformance with
|
||||
all requirements to the extent specified. This product is not
|
||||
authorized or warranted for use in life support devices and/or systems.
|
||||
|
||||
* NIST traceable calibration certificates support Measured Value data.
|
||||
Calibration services are available through ANSI/NCSL Z540-1 and
|
||||
ISO Guide 25 Certified Metrology Labs.
|
||||
42
projects/dataforth-dos/8B5BSCM-RENDER-VERIFY-2026-06-18.md
Normal file
42
projects/dataforth-dos/8B5BSCM-RENDER-VERIFY-2026-06-18.md
Normal file
@@ -0,0 +1,42 @@
|
||||
# 8B/5B/SCM Render Fix — Diagnosis + Stage/Verify Results (2026-06-18)
|
||||
|
||||
Driving the ~5,148 unpublished 8B/5B/SCM PASS records (render-coverage gap, same class as DSCA).
|
||||
|
||||
## Root cause (validated)
|
||||
1. **parseRawData bug (general):** a PASS/FAIL line is wrongly consumed as the step-response line
|
||||
for non-DSCA families that omit the `"0","0",v` line (8B45/8B49/5B39/SCM5B33...) -> drops the
|
||||
first Final-Test group -> measurement-count mismatch -> null. Fix: change the step-skip guard to
|
||||
`if (!((family === 'DSCA' || skipStepIfStatus) && looksLikeStatus))` and pass `skipStepIfStatus`
|
||||
only for templated models (so non-templated models are byte-unchanged).
|
||||
2. **No per-model Final-Test template:** one hardcoded `DATA_LINES['8B']` (RTD-shaped) -> wrong
|
||||
param names/specs for non-RTD models (8B45 is frequency-input == DSCA45). Needs per-model
|
||||
templates, same as DSCA. The "published" siblings render null too (legacy content on Hoffman;
|
||||
`api_uploaded_at` was back-populated from Hoffman inventory, NOT from our render).
|
||||
|
||||
## Artifacts
|
||||
- `8b5bscm-templates.json` — 136 models mined from Hoffman originals (accOut, accHeader, rows, _srcSerial).
|
||||
- Verify harness pattern: patch a TEMP copy of datasheet-exact.js (parse-fix + template-gated
|
||||
Final-Test, footer skip), render each model's published `_srcSerial`, content-normalized compare
|
||||
to its Hoffman original. (Never touches the live renderer.)
|
||||
|
||||
## Verify results (template-gated Final-Test only; NO slotmaps/precision/accuracy-label work yet)
|
||||
Content-only (cosmetic header/spacing deferred, per the DSCA byte-fidelity decision):
|
||||
- 15 models content-perfect (>=0.99) — publishable as-is.
|
||||
- 17 within precision-tuning distance (0.93-0.97) — single value-row rounding diffs.
|
||||
- 27 @ 0.85-0.93, 7 < 0.85.
|
||||
- 70 NULL — render-count mismatch -> need per-model slotmaps.
|
||||
- By family: 8B avg 0.97, SCM5B 0.93 (strong); 8B38 0.78; 7B ~0.88 (separate parse path).
|
||||
|
||||
## Remaining work = the SAME machinery AD2 built for DSCA
|
||||
1. Per-model **slotmaps** for the 70 nulls (AD2 `_derive_slotmaps.js`) — derive by matching rendered
|
||||
measured values to the Hoffman original's displayed values.
|
||||
2. **QB single-precision rounding** (AD2 `Math.fround`) — closes the 0.93-0.97 precision diffs.
|
||||
3. **Frequency/AAC accuracy-block labels + stimulus formatting** — the still-open DSCA45/33 piece
|
||||
(8B45/8B49/5B45 are frequency-input; SCM5B33 is AAC). Solving once covers BOTH 8B/5B AND DSCA45/33.
|
||||
4. 8B38 (0.78) + 7B family (separate SCM7B path) — family-specific.
|
||||
|
||||
## Recommendation
|
||||
Converge with AD2's mature DSCA machinery rather than re-implement in parallel in the shared
|
||||
`datasheet-exact.js`: feed the mined `8b5bscm-templates.json` into AD2's template/slotmap/rounding
|
||||
path and finish the frequency/AAC accuracy renderer once for DSCA + 8B/5B/SCM together.
|
||||
The 15 content-clean models can be published immediately if a partial win is wanted.
|
||||
7474
projects/dataforth-dos/8b5bscm-templates.json
Normal file
7474
projects/dataforth-dos/8b5bscm-templates.json
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,83 @@
|
||||
# Proposal — make the DB hold the LATEST test run (same-day retest fix)
|
||||
|
||||
**Date:** 2026-06-17 · **Host:** AD2 · **Status:** PROPOSAL — diagnose-only, review before deploying
|
||||
**File to change:** `C:\Shares\testdatadb\database\import.js` (repo: `projects/dataforth-dos/database/import.js`)
|
||||
**Evidence:** `PARSING-FIDELITY-VERDICT-2026-06-17.md`, `SAMEDAY-RETEST-EXPOSURE-2026-06-17.txt`
|
||||
|
||||
## Problem
|
||||
|
||||
`test_records` is one row per serial number. On re-import, the `INSERT ... ON CONFLICT (serial_number)` updates only when:
|
||||
|
||||
```sql
|
||||
WHERE test_records.overall_result = 'FAIL'
|
||||
OR (EXCLUDED.overall_result = 'PASS' AND EXCLUDED.test_date > test_records.test_date)
|
||||
```
|
||||
|
||||
The date comparison is **strictly greater**, and the `.DAT` serial/date line carries **date only** (no time). So when a unit is tested two or more times on the **same date**, the first same-day run to be imported wins and no later same-day run can replace it. The DB — and therefore the website datasheet — can show a **non-final** run.
|
||||
|
||||
This is the documented audit failure mode: same-day runs are usually trim / re-test iterations, and the **last** run is the accepted certificate result.
|
||||
|
||||
## Exposure (whole-source sweep, 2026-06-17)
|
||||
|
||||
981,716 records parsed across 26,815 `.DAT` files (406,549 serials):
|
||||
|
||||
- Same-day multi-run events (distinct values): **6,515** across **5,977 serials**
|
||||
- DB already on the latest same-day run: 3,803
|
||||
- Superseded by a later-date retest (fine): 984
|
||||
- **DB on a non-latest run (the defect): 311**
|
||||
- Serial absent from DB (collisions/completeness): 1,417
|
||||
|
||||
## Root cause
|
||||
|
||||
1. **Strictly-greater date** (`>`) in the conflict `WHERE` — rejects all same-date updates.
|
||||
2. **Date-only granularity** — no intra-day timestamp in the `.DAT` to order same-day runs.
|
||||
|
||||
## Proposed fix (minimal, guarded)
|
||||
|
||||
Allow a same-date PASS to overwrite **only when the data actually differs**, so the last differing same-day run processed wins (imports run in chronological append order, and the live station logs are scanned last — so the last-processed run is the latest):
|
||||
|
||||
```sql
|
||||
ON CONFLICT (serial_number) DO UPDATE SET
|
||||
log_type = EXCLUDED.log_type,
|
||||
model_number = EXCLUDED.model_number,
|
||||
test_date = EXCLUDED.test_date,
|
||||
test_station = EXCLUDED.test_station,
|
||||
overall_result = EXCLUDED.overall_result,
|
||||
raw_data = EXCLUDED.raw_data,
|
||||
source_file = EXCLUDED.source_file,
|
||||
api_uploaded_at = NULL,
|
||||
forweb_exported_at = NULL
|
||||
WHERE test_records.overall_result = 'FAIL'
|
||||
OR (EXCLUDED.overall_result = 'PASS' AND EXCLUDED.test_date > test_records.test_date)
|
||||
OR (EXCLUDED.overall_result = 'PASS' AND EXCLUDED.test_date = test_records.test_date
|
||||
AND EXCLUDED.raw_data IS DISTINCT FROM test_records.raw_data) -- NEW: latest same-day run wins
|
||||
```
|
||||
|
||||
The added clause only fires on a genuine same-date data change, so identical re-imports do **not** needlessly clear `api_uploaded_at` (avoids re-push churn).
|
||||
|
||||
### Behavior after fix
|
||||
|
||||
| Existing | Incoming | Before | After |
|
||||
|---|---|---|---|
|
||||
| PASS date D | PASS date D, different data | ignored (stale) | **updated → latest run** |
|
||||
| PASS date D | PASS date D, identical | ignored | ignored (no churn) |
|
||||
| PASS date D | PASS date D+1 | updated | updated (unchanged) |
|
||||
| PASS date D+1 | PASS date D | ignored | ignored (unchanged) |
|
||||
| FAIL | PASS (any date) | updated | updated (unchanged) |
|
||||
|
||||
## Caveats / assumptions
|
||||
|
||||
- **Relies on chronological append order** within a `.DAT` and on the live station logs being scanned **last** (they are: `runImport` does HISTLOGS → Recovery → station `TEST_PATH`). If a serial's latest run existed only in HISTLOGS (scanned first) and an older copy in a station log (scanned last), the older copy would win. Rare, but possible. For a hard guarantee, add a monotonic tiebreaker (ingest sequence, or a per-run timestamp if the test program can emit one) — a larger change.
|
||||
- **Re-push impact:** the 311 corrected rows (plus any future same-day retests) will clear `api_uploaded_at` and re-upload to Hoffman on the next run. Expected and desired (the website gets the final result), but it is outbound API traffic — run deliberately.
|
||||
- **Does NOT fix** generic reused serials (`1-1`, `1-2`, …) that collide across different products, nor the 608 units absent from the DB. Those are separate items (serial-uniqueness model / ingestion completeness).
|
||||
|
||||
## Stronger alternative (larger migration)
|
||||
|
||||
If full per-run archival is required (every test sheet reproducible), replace the `UNIQUE (serial_number)` model with a composite key **`(serial_number, test_date, run_sequence)`** (or store all runs and select the latest at render time). This preserves every run and removes the same-day ambiguity entirely, but is a schema migration + dedupe + render/upload changes — propose separately if desired.
|
||||
|
||||
## Rollout (after approval)
|
||||
|
||||
1. Apply the `WHERE`-clause change to `database/import.js` (repo copy first, review, then deploy).
|
||||
2. Re-run the import so the 311 same-day cases settle on the latest run.
|
||||
3. Let the upload path re-push the cleared rows; confirm counts.
|
||||
4. Re-run `tools/validate-parsing.js` to confirm same-day violations drop to ~0.
|
||||
285
projects/dataforth-dos/DATASHEET-RTD-BUG-DIAGNOSIS-2026-06-17.md
Normal file
285
projects/dataforth-dos/DATASHEET-RTD-BUG-DIAGNOSIS-2026-06-17.md
Normal file
@@ -0,0 +1,285 @@
|
||||
# Test-Datasheet Bug — End-to-End Trace & Diagnosis
|
||||
|
||||
**Date:** 2026-06-17
|
||||
**Host:** AD2 (192.168.0.6) — testdatadb generator + PostgreSQL 18
|
||||
**Author:** Mike Swanson / AZ Computer Guru
|
||||
**Status:** DIAGNOSIS ONLY — no code or DB changes made
|
||||
**Reported by:** John Lehman / Peter Iliya (Dataforth) — customer Wellbore Integrity (Joseph Swinehart) cal-cert audit on 8B35 4-wire RTD certs
|
||||
|
||||
---
|
||||
|
||||
## 0. TL;DR
|
||||
|
||||
There are **two independent defects, both in the datasheet *renderer*** (`templates/datasheet-exact.js`). Ingestion (.DAT parsing), the database contents, and the spec files are all **correct** — the raw test data in the DB holds the right values; the renderer mislabels and mis-maps them.
|
||||
|
||||
| # | Defect | Symptom on cert | Scope | Severity |
|
||||
|---|--------|-----------------|-------|----------|
|
||||
| **A** | RTD input column rendered as **resistance** instead of **temperature** | Header reads `Rin (ohms)`, should read `Temp. (C)`; positive input values lost their leading `+` | ~24,000 RTD certs (8B35, DSCA34, SCM5B34/35, and any RTD variant) | **HIGH** — this is the audit finding |
|
||||
| **B** | **Entire DSCA Final-Test parameter list is wrong** | Wrong parameter names, garbage specs (`< 0 mA`, `+/- 0 %`), values aligned to the wrong rows, output column mislabeled (`Vout (V)` vs `Output (mA)`), lines missing/added | up to 78,343 DSCA certs (all DSCLOG) | **HIGH** |
|
||||
|
||||
Defect A is a small, surgical fix. Defect B requires rebuilding the DSCA template against the legacy spec (`DSCFIN.DAT`) and is a larger effort.
|
||||
|
||||
**Root cause is confirmed against the original ground-truth files** (the DOS-station-generated staged `.TXT`), not assumptions.
|
||||
|
||||
---
|
||||
|
||||
## 1. The Original File IS the Ground Truth — and We Found It
|
||||
|
||||
Mike's key point was correct: **the rendered datasheet exists as a file BEFORE it ever reaches the database.** That original file is produced by the DOS test station itself and is the source of truth. Our DB-based regeneration is what introduces the error.
|
||||
|
||||
### Where the original files physically live
|
||||
|
||||
The DOS station's QuickBASIC ATE program writes a fully-rendered `.TXT` datasheet to `C:\STAGE\` on the station, then `CTONWTXT.BAT` uploads it to the NAS `STAGE` share. It is mirrored to AD2:
|
||||
|
||||
```
|
||||
ORIGINAL (ground truth) staged datasheets:
|
||||
AD2: C:\Shares\test\STAGE\<TS-station>\<encoded-SN>.TXT
|
||||
NAS: /data/test/STAGE/<TS-station>/<encoded-SN>.TXT (\\192.168.0.9\test\STAGE)
|
||||
```
|
||||
|
||||
Filenames use the 8.3 hex-prefix serial encoding (first two digits → letter, `55 + n`):
|
||||
`179553-13` → `17`→`H` → **`H9553-13.TXT`**; `180224-7` → `18`→`I` → **`I0224-7.TXT`**.
|
||||
|
||||
**Confirmed files for this investigation:**
|
||||
|
||||
| Module | SN | Original staged file (ground truth) | Source `.DAT` |
|
||||
|--------|----|--------------------------------------|---------------|
|
||||
| 8B35-04 (4-wire RTD) | 179553-13 | `C:\Shares\test\STAGE\TS-4L\H9553-13.TXT` | `C:\Shares\test\TS-4L\LOGS\8BLOG\35-04.DAT` |
|
||||
| DSCA38-05 (full bridge) | 180224-7 | `C:\Shares\test\STAGE\TS-11R\I0224-7.TXT` | `C:\Shares\test\TS-11R\LOGS\DSCLOG\38-05.DAT` |
|
||||
| DSCA34-05C (3-wire RTD) | 180007-8 | `C:\Shares\test\STAGE\TS-4R\I0007-8.TXT` | `C:\Shares\test\TS-4R\LOGS\DSCLOG\34-05C.DAT` |
|
||||
|
||||
> Note re Peter: `179553-13` cannot be found on X: / DFWDS / For_Web because the **website copy is regenerated from the DB at upload time** — it is never written to disk. The on-disk ground truth is the **STAGE** copy above, not For_Web. (For_Web on AD2 holds only ~7,500 legacy files and does not contain this SN.)
|
||||
|
||||
---
|
||||
|
||||
## 2. End-to-End Pipeline (upstream emphasis)
|
||||
|
||||
```
|
||||
[1] DOS Test Station (TS-xx, DOS 6.22, QuickBASIC ATE)
|
||||
- Runs the unit test, measures everything.
|
||||
- Writes TWO artifacts:
|
||||
(a) C:\ATE\...\<model>.DAT raw CSV-ish multi-line test log (-> network LOGS)
|
||||
(b) C:\STAGE\<encSN>.TXT FULLY RENDERED datasheet <-- GROUND TRUTH
|
||||
- The station ALREADY knows the sensor type, so it prints the correct
|
||||
input column ("Temp. (C)" for RTD) and the correct Final-Test parameter
|
||||
list for that exact model. This is the format we must reproduce.
|
||||
|
||||
[2] Boot upload (CTONW.BAT / CTONWTXT.BAT)
|
||||
- .DAT -> \\NAS\test\<TS>\LOGS\<logtype>\<model>.DAT
|
||||
- .TXT -> \\NAS\test\STAGE\<TS>\<encSN>.TXT
|
||||
|
||||
[3] NAS <-> AD2 sync (Sync-FromNAS.ps1, 15 min)
|
||||
- Mirrors to C:\Shares\test\... on AD2.
|
||||
|
||||
[4] testdatadb ingest (THIS host) ----- the original .TXT is IGNORED here -----
|
||||
- import.js scans .DAT files (NOT the staged .TXT).
|
||||
- parsers/multiline.js parses the .DAT into a record:
|
||||
{ log_type, model_number, serial_number, test_date, test_station,
|
||||
overall_result, raw_data (the verbatim .DAT block), source_file }
|
||||
- INSERT ... ON CONFLICT(serial_number) into PostgreSQL test_records.
|
||||
raw_data = the exact .DAT text block (this is faithful & correct).
|
||||
|
||||
[5] Render + upload (the bug lives here)
|
||||
- render-datasheet.js -> templates/datasheet-exact.js regenerates the
|
||||
datasheet text FROM raw_data + spec files, in memory.
|
||||
- upload-to-api.js POSTs {SerialNumber, Content} to Hoffman bulk API.
|
||||
- Hoffman serves it on the public product page.
|
||||
```
|
||||
|
||||
**The pivotal architectural fact:** step [4] throws away the already-correct rendered `.TXT` and keeps only the raw `.DAT` block (`raw_data`). Step [5] then *re-renders* from scratch. Every rendering defect is introduced in step [5]; the upstream data is fine.
|
||||
|
||||
### 2.1 What the `.DAT` / `raw_data` actually contains (8B35-04, SN 179553-13)
|
||||
|
||||
```
|
||||
"8B35-04 " <- model
|
||||
-1.461694,-1.218078E-02,-.014174,-3.986431E-02,"PASS" <- accuracy pt 1: stim,calc,meas,err,status
|
||||
151.5394,1.262828,1.26273,-1.966953E-03,"PASS" <- pt 2
|
||||
303.6477,2.530397,2.531,1.204967E-02,"PASS" <- pt 3
|
||||
448.7633,3.739694,3.7414,.0341177,"PASS" <- pt 4
|
||||
598.0475,4.983729,4.9824,-2.658844E-02,"PASS" <- pt 5
|
||||
"0","0",0 <- step-response placeholder
|
||||
"PASS 28.424741","PASS","PASS 252.21681","PASS","PASS" <- final-test STATUS groups (5 per line)
|
||||
"PASS","","PASS","PASS","PASS"
|
||||
"PASS","PASS 3.478871E-023","PASS 3.986431E-023","PASS","PASS 26.328911"
|
||||
"PASS","PASS","PASS 40.429370","PASS","PASS 140.50"
|
||||
```
|
||||
|
||||
The **first column of each accuracy point is the stimulus**. For SN 179553-13 it is `-1.46, 151.5, 303.6, 448.8, 598.0` — clearly **temperatures in °C** (model MAXIN = 600 °C), **not** ohms. The spec record confirms `SENTYPE = "P1RTD4W"`, `MAXIN = 600`. So the DB has the right numbers and the right sensor type. Nothing upstream is wrong.
|
||||
|
||||
---
|
||||
|
||||
## 3. Diffs — Original (correct) vs DB-Generated (wrong)
|
||||
|
||||
### 3.1 8B35-04 RTD (SN 179553-13) — Defect A only
|
||||
|
||||
ACCURACY block header + first/last rows:
|
||||
|
||||
```
|
||||
ORIGINAL (H9553-13.TXT, ground truth) GENERATED (current testdatadb)
|
||||
----------------------------------- ------------------------------
|
||||
Temp. (C) Vout (V) ... Rin (ohms) Vout (V) ... <-- WRONG LABEL
|
||||
-1.46 ... -1.46 ... (same)
|
||||
+151.54 ... 151.54 ... <-- lost '+'
|
||||
+598.05 ... 598.05 ... <-- lost '+'
|
||||
```
|
||||
|
||||
- **Header:** `Temp. (C)` → rendered as `Rin (ohms)`. **This is the audit discrepancy.**
|
||||
- **Values:** numerically identical (correct temperatures). Only difference: positive values lose the leading `+` because the resistance formatter omits the sign.
|
||||
- **Final Test Results: byte-for-byte IDENTICAL** to the original (7 lines: Supply Current Nom, Exc. Current #1, Linearity, Accuracy, Supply Sensitivity, Frequency Response, Output Noise). **No missing lines for 8B35.** The 8B/5B Final-Test rendering is correct.
|
||||
|
||||
So for the Wellbore Integrity audit, the **only** defect on the 8B35 cert is the input column header label (and the cosmetic `+` sign). The measured data is right.
|
||||
|
||||
### 3.2 DSCA38-05 full-bridge (SN 180224-7) — Defect B
|
||||
|
||||
This is not a label tweak — the **whole Final Test table is wrong**:
|
||||
|
||||
```
|
||||
ORIGINAL (I0224-7.TXT) GENERATED (current)
|
||||
Supply Current 23.8 mA < 30 mA Supply Current, Nom 23.8 mA < 0 mA <- spec garbage
|
||||
Supply Curr. w/ EXC Load 53.8 mA < 80 mA Supply Current @ Max Load 53.8 mA < 0 mA
|
||||
Excitation Voltage 10.000 V 10+/-.003V Linearity, 50mA Load 10.000 % +/- 0 % <- wrong name+unit
|
||||
Exc. Load Regulation -6 ppm/mA ... Accuracy, 50mA Load 6 % +/- 0 % <- wrong row
|
||||
Output Reg. w/ EXC Load 0.00 % +/- .05 % Positive Current Limit 0.0 mA < 0 mA
|
||||
Excitation Current Limit 54 mA < 65 mA Negative Current Limit 54 mA > 0 mA
|
||||
Linearity 0.002 % +/- .02 % Overrange 0.002 % > 0 %
|
||||
Accuracy -0.008 % +/- .05 % Power Supply Sensitivity 0.008 %/% +/-.0006
|
||||
Power Supply Sens. 0.0000 %/% +/-.0006 Frequency Response 0.0000 dB 25+/-5 dB <- value=0
|
||||
Frequency Response 25.0 dB 25+/-5 dB Compliance 25.0 % +/- 0 % <- 25.0 is freq!
|
||||
Output Noise 1205 uVrms <=2000 (Output Noise line dropped entirely)
|
||||
```
|
||||
|
||||
Also in the ACCURACY block: original column titles are `Output (V)` and separator dashes `----------`; the generator emits `Vout (V)` and `==========`. The input header happens to read `Vin (mV)` in both (bridge module), so the bridge input label is OK, but everything below it is misaligned.
|
||||
|
||||
### 3.3 DSCA34-05C 3-wire RTD (SN 180007-8) — Defect A **and** B together
|
||||
|
||||
```
|
||||
ORIGINAL (I0007-8.TXT) GENERATED (current)
|
||||
Temp. (C) Output (mA) ... Rin (ohms) Vout (V) ... <-- A: label + wrong out-unit
|
||||
Supply Current 50.7 mA < 65 mA Supply Current, Nom 50.7 mA < 0 mA
|
||||
Exc. Current @ -f.s. 264.0 uA 261 uA Linearity, 0mA Load 264.0 % +/- 0 % <-- 264.0 is uA, not %
|
||||
Exc. Current @ +f.s. 281.0 uA 278 uA Accuracy, 0mA Load 281.0 % +/- 0 %
|
||||
Linearity 0.017 % +/-.03% Overrange 0.017 % > 0 %
|
||||
Accuracy -0.034 % +/-.05% Power Supply Sens. 0.034 %/% +/-.0005
|
||||
... (9 real params) ... (wrong names, garbage specs, lines dropped)
|
||||
```
|
||||
|
||||
The excitation currents (264.0 uA, 281.0 uA) get printed under the labels "Linearity, 0mA Load" / "Accuracy, 0mA Load" as **percentages** — visibly nonsensical.
|
||||
|
||||
---
|
||||
|
||||
## 4. Root-Cause Localization
|
||||
|
||||
Tested against the original files, the defect is **(c) the renderer** — `templates/datasheet-exact.js`. Ingestion/parsing (a) and the DB data (b) are correct.
|
||||
|
||||
### Defect A — RTD treated as resistance
|
||||
|
||||
```js
|
||||
// getSensorNum(): RTD maps to 7
|
||||
if (s.includes('RTD')) return 7; // line ~150
|
||||
|
||||
// Accuracy input-column header (generateExactDatasheet):
|
||||
} else if (sensorNum === 7) {
|
||||
inputHeader = ' Rin (ohms)'; // line ~564 <-- WRONG for Dataforth RTD
|
||||
}
|
||||
|
||||
// Accuracy value formatting (formatAccuracyLine):
|
||||
} else if (sensorNum === 7) {
|
||||
stimStr = point.stim.toFixed(2).padStart(8); // line ~447 <-- resistance format, no sign
|
||||
}
|
||||
```
|
||||
|
||||
Dataforth RTD modules always express the input as **temperature** on the datasheet (the RTD curve converts resistance→°C; the `.DAT` already stores °C). `sensorNum === 7` is reached **only** by RTD sentypes (`P1RTD3W`, `P1RTD4W`, `NIRTD3W`, …). There is currently **no module for which `Rin (ohms)` is correct** — true resistance/potentiometer inputs are not routed to 7 (they fall through to the voltage default). So fixing the `7` branch is safe and will not over-correct.
|
||||
|
||||
### Defect B — DSCA parameter list mismatch
|
||||
|
||||
`DATA_LINES['DSCA']` is a **single hardcoded list** (Supply Current Nom / @ Max Load / Linearity 0mA / Accuracy 0mA / Linearity 5mA / … / Compliance / Accuracy @ 5 ohm). Real DSCA modules use **different Final-Test layouts per subtype** (bridge/excitation modules list Excitation Voltage / Exc. Load Reg. / Output Reg.; RTD/TC list Exc. Current @ ±f.s.; etc.). The hardcoded list does not match, so:
|
||||
|
||||
- `raw_data` STATUS groups are mapped positionally onto the **wrong** parameter names;
|
||||
- `buildTSpecs()` for DSCA reads spec fields (`ILIMIT`, `PERCOVER`, `COMPLIANCE`, `ACCURACY1/2/3`, `LINEAR1/2/3`) that are zero/absent for these modules → specs print as `< 0`, `+/- 0`;
|
||||
- the skip rule `if (status.length <= 4) continue` drops every bare-`PASS` slot, but because the list is misaligned the *wrong* lines drop and the survivors land on wrong rows;
|
||||
- the ACCURACY block also uses the 5B/8B titles (`Vout (V)` / `==========`) instead of DSCA's (`Output (V|mA)` / `----------`).
|
||||
|
||||
The "missing Final Test lines" complaint is a **symptom of Defect B** (DSCA misalignment + skip rule), **not** a separate bug. The 8B/5B skip rule is correct — the legacy station also prints only tested parameters (verified: 8B35 matches exactly).
|
||||
|
||||
> Clarification for the thread: **DSCA38 is a bridge/strain-gauge module (FBRIDGE/HBRIDGE), not an RTD.** The DSCA RTD analog is **DSCA34**. So DSCA38's problem is Defect B; DSCA34's problem is A+B. If the audit specifically concerns RTD resistance-vs-temperature, the DSCA part to verify with the customer is **DSCA34**, while DSCA38 demonstrates the broader DSCA table breakage.
|
||||
|
||||
---
|
||||
|
||||
## 5. Correct Output & Proposed Fix
|
||||
|
||||
### 5.1 Correct RTD output (Defect A)
|
||||
|
||||
Per the originals, RTD modules must render:
|
||||
- **Input column header:** `Temp. (C)` (same column/format as thermocouples, `sensorNum` 3–6).
|
||||
- **Input values:** the stimulus value straight from `raw_data` (already °C — **no conversion needed**), signed format (`+598.05`, `-1.46`).
|
||||
|
||||
Surgical change in `templates/datasheet-exact.js` (fold RTD into the temperature path):
|
||||
|
||||
```js
|
||||
// (1) Header — replace the sensorNum===7 branch:
|
||||
if ((sensorNum >= 3 && sensorNum <= 6) || sensorNum === 7) {
|
||||
inputHeader = ' Temp. (C)';
|
||||
} else if (sensorNum === 2 || sensorNum === 9) {
|
||||
inputHeader = ' Iin (mA)';
|
||||
} else {
|
||||
inputHeader = (maxIn != null && maxIn < 1) ? ' Vin (mV)' : ' Vin (V)';
|
||||
}
|
||||
|
||||
// (2) Value format — in formatAccuracyLine, treat 7 like 3–6:
|
||||
if ((sensorNum >= 3 && sensorNum <= 6) || sensorNum === 7) {
|
||||
stimStr = formatSigned(point.stim, 2, 8); // temperature, signed
|
||||
} else { ... }
|
||||
```
|
||||
|
||||
Leave the existing `i===13 && sensorNum===7 → 'ohm/ohm'` unit override (Lead-R-Effect) as-is; it is a separate, correct detail. **Verify exact leading-space alignment** against a known-good thermocouple original before pushing (the header column should byte-match; this is the same polish already in progress in `generated-v2-*.TXT`).
|
||||
|
||||
This fix corrects **header + values** for all RTD modules (8B35, DSCA34, SCM5B34/35, etc.) at once. **Do not** add any resistance→temperature math — the data is already temperature.
|
||||
|
||||
### 5.2 DSCA template (Defect B)
|
||||
|
||||
Not a one-liner. The fix is to drive the DSCA Final-Test parameter list (and ACCURACY column titles/units) **per module subtype**, matching the legacy QuickBASIC DSC writer. The legacy parameter selection lives in **`specdata\DSCFIN.DAT`** (DSC final-test definitions) alongside `DSCMAIN4.DAT`/`DSCOUT.DAT`. Recommended approach:
|
||||
1. Reverse the DSCFIN.DAT layout (or read the QB DSC datasheet source) to get the per-subtype parameter name/unit/spec list.
|
||||
2. Replace the single `DATA_LINES['DSCA']` + DSCA branch of `buildTSpecs()` with subtype-aware selection keyed on SENTYPE / output-signal type.
|
||||
3. Fix the DSCA ACCURACY block to use `Output (V|mA)` (per `OUTSIGTYPE`) and dash separators.
|
||||
4. Validate byte-for-byte against staged originals across DSCA subtypes (bridge, RTD, TC, current-out, voltage-out).
|
||||
|
||||
Until B is fixed, **all DSCA (DSCLOG) website datasheets should be treated as unreliable** for Final-Test content.
|
||||
|
||||
---
|
||||
|
||||
## 6. Impact (records currently on the website)
|
||||
|
||||
| Group | On-web count | Defect |
|
||||
|-------|-------------:|--------|
|
||||
| 8B35* | 5,476 | A |
|
||||
| DSCA34* (RTD) | 3,573 | A + B |
|
||||
| SCM5B34/35* (RTD) | 14,887 | A |
|
||||
| All DSCA (DSCLOG) | 78,343 | B (RTD subset also A) |
|
||||
| **Total on website** | 464,671 | — |
|
||||
|
||||
RTD-label exposure (Defect A) ≈ **24,000 certs**; DSCA table exposure (Defect B) ≈ **78,000 certs**.
|
||||
|
||||
After fixes are reviewed and deployed, affected records can be re-pushed by clearing `api_uploaded_at` for the affected models and letting the upload path re-render (RE-PUSH is idempotent; Hoffman returns `Unchanged` when content matches).
|
||||
|
||||
---
|
||||
|
||||
## 7. How to Reproduce / Verify (read-only)
|
||||
|
||||
```powershell
|
||||
# Render current generator output for a SN and compare to the staged original:
|
||||
cd C:\Shares\testdatadb
|
||||
node -e "const db=require('./database/db');const {renderContent}=require('./database/render-datasheet');(async()=>{const r=await db.queryOne('SELECT * FROM test_records WHERE serial_number=$1',['179553-13']);if(r.test_date&&r.test_date.toISOString)r.test_date=r.test_date.toISOString().slice(0,10);console.log(renderContent(r));await db.close();})()"
|
||||
|
||||
# Ground truth:
|
||||
type C:\Shares\test\STAGE\TS-4L\H9553-13.TXT
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Files Referenced
|
||||
|
||||
- Generator: `C:\Shares\testdatadb\templates\datasheet-exact.js` (repo: `projects/dataforth-dos/datasheet-pipeline/implementation/templates/datasheet-exact.js`)
|
||||
- Render glue: `C:\Shares\testdatadb\database\render-datasheet.js`
|
||||
- Specs: `C:\Shares\testdatadb\parsers\spec-reader.js`, `C:\Shares\testdatadb\specdata\*.DAT` (incl. `DSCFIN.DAT`)
|
||||
- Ingest: `C:\Shares\testdatadb\database\import.js`, `C:\Shares\testdatadb\parsers\multiline.js`
|
||||
- Ground-truth originals: `C:\Shares\test\STAGE\<TS>\<encSN>.TXT`
|
||||
58
projects/dataforth-dos/DSCA-STAGE3-REPORT-2026-06-18.txt
Normal file
58
projects/dataforth-dos/DSCA-STAGE3-REPORT-2026-06-18.txt
Normal file
@@ -0,0 +1,58 @@
|
||||
Fix 2 STAGE 3 — DSCA Final-Test render vs staged-original content validation
|
||||
GATE = FINAL TEST RESULTS section, content-strict (rule lines canonicalized,
|
||||
whitespace collapsed). Accuracy-section diffs reported separately (deferred
|
||||
cosmetic spacing + any pre-existing calc rounding — NOT a Fix 2 gate).
|
||||
Corpus: 2806 staged DSCA originals across 126 models.
|
||||
==============================================================================
|
||||
|
||||
SUMMARY
|
||||
models with staged originals: 126
|
||||
models FINAL-TEST CLEAN (>=1 compared, 0 mismatch): 92
|
||||
models with FINAL-TEST mismatches: 6
|
||||
certs compared: 2450
|
||||
Final-Test match: 2412
|
||||
Final-Test mismatch: 38
|
||||
(certs with accuracy-section diffs: 1369 — informational)
|
||||
staged serials not in DB: 40
|
||||
in DB but not rendered (skipped/null): 316
|
||||
|
||||
MODELS WITH FINAL-TEST CONTENT MISMATCHES (investigate before re-push):
|
||||
DSCA38-05 compared=129 ftMatch=97 ftMismatch=32
|
||||
[finaltest L3] SN 180257-1
|
||||
render: "Supply Current 19.8 mA < 30 mA PASS"
|
||||
golden: "Supply Current 20.0 mA < 30 mA PASS"
|
||||
[finaltest L3] SN 180257-10
|
||||
render: "Supply Current 19.6 mA < 30 mA PASS"
|
||||
golden: "Supply Current 20.3 mA < 30 mA PASS"
|
||||
[finaltest L3] SN 180257-11
|
||||
render: "Supply Current 20.1 mA < 30 mA PASS"
|
||||
golden: "Supply Current 19.8 mA < 30 mA PASS"
|
||||
DSCA38-1793 compared=33 ftMatch=32 ftMismatch=1
|
||||
[finaltest L3] SN 176923-11
|
||||
render: "Supply Current 22.3 mA < 25 mA PASS"
|
||||
golden: "Supply Current 23.1 mA < 25 mA PASS"
|
||||
DSCA38-19C compared=248 ftMatch=247 ftMismatch=1
|
||||
[finaltest L4] SN 180036-8
|
||||
render: "Supply Curr. w/ EXC Load 54.5 mA < 100 mA PASS"
|
||||
golden: "Supply Curr. w/ EXC Load 55.0 mA < 100 mA PASS"
|
||||
DSCA38-19E compared=28 ftMatch=27 ftMismatch=1
|
||||
[finaltest L3] SN 179698-16
|
||||
render: "Supply Current 42.7 mA < 60 mA PASS"
|
||||
golden: "Supply Current 45.7 mA < 60 mA PASS"
|
||||
DSCA39-05 compared=17 ftMatch=16 ftMismatch=1
|
||||
[finaltest L3] SN A276-2
|
||||
render: "Supply Current, Nom 61.7 mA < 75 mA PASS"
|
||||
golden: "Supply Current, Nom 60.0 mA < 75 mA PASS"
|
||||
DSCA39-1950 compared=17 ftMatch=15 ftMismatch=2
|
||||
[finaltest L4] SN 178236-12
|
||||
render: "Linearity 0.013 % +/- .05 % PASS"
|
||||
golden: "Linearity 0.009 % +/- .05 % PASS"
|
||||
[finaltest L4] SN 178236-13
|
||||
render: "Linearity 0.017 % +/- .05 % PASS"
|
||||
golden: "Linearity 0.012 % +/- .05 % PASS"
|
||||
|
||||
FINAL-TEST CLEAN MODELS (92):
|
||||
DSCA30-01, DSCA30-02, DSCA30-03, DSCA30-06, DSCA30-07, DSCA30-08, DSCA30-08C, DSCA30-09, DSCA30-09C, DSCA30-1944, DSCA30-1945, DSCA30-1946, DSCA31-02, DSCA31-03, DSCA31-06, DSCA31-07, DSCA31-11, DSCA31-12, DSCA31-1273, DSCA31-12C, DSCA31-13, DSCA31-13C, DSCA31-15, DSCA31-1918, DSCA32-01, DSCA32-01C, DSCA32-01E, DSCA34-01, DSCA34-02C, DSCA34-04, DSCA34-04C, DSCA34-05, DSCA34-05C, DSCA34-1858, DSCA36-01, DSCA36-02, DSCA36-03, DSCA36-04, DSCA36-04C, DSCA36-1949, DSCA38-02, DSCA38-03, DSCA38-07, DSCA38-08C, DSCA38-09, DSCA38-09E, DSCA38-12C, DSCA38-12E, DSCA38-1468, DSCA38-1544, DSCA38-15C, DSCA38-16, DSCA38-16C, DSCA38-18C, DSCA38-19, DSCA39-01, DSCA39-02, DSCA39-07, DSCA40-03, DSCA40-05, DSCA40-05C, DSCA40-06, DSCA40-1951, DSCA40-1952, DSCA41-01, DSCA41-02, DSCA41-03, DSCA41-05C, DSCA41-06, DSCA41-09, DSCA41-13, DSCA41-14, DSCA41-15, DSCA41-15E, DSCA42-01, DSCA42-01C, DSCA42-02, DSCA43-10, DSCA43-20E, DSCA47E-08C, DSCA47J-01C, DSCA47J-03, DSCA47K-05, DSCA47K-13, DSCA47K-14, DSCA47N-15, DSCA47T-06, DSCA47T-1928, DSCA49-04, DSCA49-05, DSCA49-1601, DSCA49-1895
|
||||
|
||||
MODELS WITH NO COMPARABLE CERT (no staged serial in DB, or all skipped/null):
|
||||
DSCA31-1947(null), DSCA33-01(null), DSCA33-01A(null), DSCA33-02(null), DSCA33-02C(null), DSCA33-03(null), DSCA33-03A(null), DSCA33-03C(null), DSCA33-04(null), DSCA33-04C(null), DSCA33-05(null), DSCA33-05C(null), DSCA33-07C(null), DSCA33-1917(null), DSCA33-1919(null), DSCA33-1948(null), DSCA45-01(null), DSCA45-01C(null), DSCA45-02(null), DSCA45-03(null), DSCA45-03C(null), DSCA45-04(null), DSCA45-04C(null), DSCA45-05C(null), DSCA45-06(null), DSCA45-07(null), DSCA45-08(null), DSCA47N-15C(null)
|
||||
106
projects/dataforth-dos/DSCA33-45-HOFFMAN-RECOVERY-2026-06-18.md
Normal file
106
projects/dataforth-dos/DSCA33-45-HOFFMAN-RECOVERY-2026-06-18.md
Normal file
@@ -0,0 +1,106 @@
|
||||
# DSCA33 / DSCA45 — Recover the "lost" specs from Hoffman (Handoff to AD2)
|
||||
|
||||
**For:** the Claude session on AD2 (`C:\Shares\testdatadb`). **Ref:** Syncro #32441.
|
||||
**Supersedes** the FIX2-5 handoff's *TODO 2 (request DSCA33/45 main spec files from John)* —
|
||||
**we do not need John.** The original specs are recoverable from the Hoffman API.
|
||||
|
||||
---
|
||||
|
||||
## The finding (why this changes the plan)
|
||||
|
||||
The DSCA33/DSCA45 "main spec" records (DSCMAIN/DSCOUT with SENTYPE/MAXIN/input-type) were
|
||||
lost in the cryptolocker wipe, so `render-datasheet.js` bails (null render) and the pipeline
|
||||
**skips** those models. BUT the **original software published correct DSCA33/45 certs to the
|
||||
Hoffman API before the wipe**, and our broken renderer never overwrote them (it skips null
|
||||
renders). They are still there, pristine.
|
||||
|
||||
Verified from GURU-5070 against the public API
|
||||
(`GET https://www.dataforth.com/api/v1/TestReportDataFiles/{serial}`, OAuth client-creds,
|
||||
vaulted `clients/dataforth/hoffman-product-api`):
|
||||
|
||||
| Family | Models | Mineable from Hoffman | Units already correct+live on Hoffman | No original anywhere |
|
||||
|---|---|---|---|---|
|
||||
| DSCA33 | 35 | **34** | 2,633 / 3,397 | **DSCA33-1948 (16 units)** |
|
||||
| DSCA45 | 23 | **22** | 4,524 / 5,413 | **DSCA45-1746 (8 units)** |
|
||||
|
||||
So of the ~8,763 "blocked" certs: **~7,157 are already correct and live on the public site**
|
||||
(no action needed), **~1,580 not-yet-uploaded units** just need rendering, and only
|
||||
**24 units across 2 niche models** have no original to recover.
|
||||
|
||||
---
|
||||
|
||||
## The artifact (already built — use it, don't re-derive)
|
||||
|
||||
`projects/dataforth-dos/dsca33-45-templates.json` — **56 models**, mined from the Hoffman
|
||||
originals with the same `===`-rule column-span extractor as STAGE 1. Schema is a **superset of
|
||||
`dsca-templates.json`**:
|
||||
|
||||
```json
|
||||
"DSCA45-05E": {
|
||||
"accOut": "Output (mA)",
|
||||
"accHeader": [
|
||||
" Frequency Calculated Measured",
|
||||
" (Hz) Output (mA) Output (mA)* Error (%) Status"
|
||||
],
|
||||
"rows": [ {"name":"Supply Current","spec":"< 105 mA"}, ... ],
|
||||
"_srcSerial": "176326-2"
|
||||
}
|
||||
```
|
||||
|
||||
- `rows` = the Final-Test `Parameter | Specification` list (incl. the spec-less rows like
|
||||
`240 VAC Withstand`, `Hi-Pot`, and section sub-heads `Zero-Crossing Input` / `TTL Input` —
|
||||
**kept**, same skip-rule reconciliation as STAGE 2).
|
||||
- `accHeader` = the **verbatim 2-line accuracy header** from the original. Use it — DSCA33/45
|
||||
introduce header tokens the 92-model set never had (see flags) and the frequency-input layout.
|
||||
- `_srcSerial` = a known already-uploaded serial for that model → your validation oracle.
|
||||
|
||||
Spot-checked DSCA33-07C and DSCA45-05E row-for-row against their live Hoffman originals: exact.
|
||||
|
||||
Regenerate if needed: `python projects/dataforth-dos/tools/mine-hoffman-dsca.py <map.json> <out.json>`.
|
||||
|
||||
---
|
||||
|
||||
## Flags (DSCA33/45 differ from the 92 you already did)
|
||||
|
||||
1. **New accOut tokens:** `Output (VDC)` and `Output (mADC)` (DSCA33 current/voltage DC outputs),
|
||||
not just `Output (V)`/`Output (mA)`. Your accuracy-block must emit the verbatim token.
|
||||
2. **Model-specific accuracy input label:** DSCA33 uses `Vin (mVAC)` / `Vin (VAC)` / `Iin (AAC)` /
|
||||
`Iin (mAAC)`. **Use the `accHeader` lines** rather than synthesizing from a (missing) spec field.
|
||||
3. **DSCA45 is frequency-input:** two-line super-header `Frequency` / `(Hz)` and a frequency
|
||||
sweep in the accuracy block — structurally unlike the voltage/current-input models. Confirm the
|
||||
accuracy renderer reproduces it (the `accHeader` gives you the exact text).
|
||||
4. Your **DSCA33/45 slotMaps are already derived** and come from the same original layout as these
|
||||
rows, so order should align — verify during validation.
|
||||
|
||||
---
|
||||
|
||||
## Plan
|
||||
|
||||
1. **Backup first** (fresh `pg_dump` + VSS) per FIX2-5 discipline. Save-state `datasheet-exact.js`.
|
||||
2. **Load** `dsca33-45-templates.json` for `family in (DSCA33, DSCA45)`, same wiring as STAGE 2
|
||||
(template `rows` drive names+specs; map raw_data STATUS groups positionally via the slotMaps;
|
||||
QB `Math.fround` rounding; data-driven loadNote). For the accuracy block, drive the header from
|
||||
`accHeader` / `accOut`.
|
||||
3. **VALIDATE against Hoffman (the gate — stronger than STAGE 3):** for each model, render its
|
||||
`_srcSerial` (an already-uploaded unit) and **content-normalized byte-compare against
|
||||
`GET /api/v1/TestReportDataFiles/{_srcSerial}`**. Require a clean match **per model** before that
|
||||
model is allowed to render/publish.
|
||||
> **CRITICAL — do not enable live DSCA33/45 rendering until each model passes.** The moment the
|
||||
> renderer returns non-null for DSCA33/45, the pipeline stops skipping them and will **re-push and
|
||||
> UPDATE the ~7,157 already-correct originals** on the next cycle. That is only safe if the render
|
||||
> byte-matches the original — which the per-model gate proves. A mismatched model would overwrite
|
||||
> good customer certs. Gate hard.
|
||||
4. **Publish the gap:** for validated models, render + `uploadBySerialNumbers` the **not-yet-uploaded**
|
||||
units (~1,580; `api_uploaded_at IS NULL`). Already-uploaded units return `Unchanged` (idempotent).
|
||||
5. **Leave blocked (24 units):** `DSCA33-1948` (16), `DSCA45-1746` (8) — no Hoffman original. Low
|
||||
priority; only these would ever need John, and they look like one-off custom part numbers.
|
||||
|
||||
Commit to `ad2`; update #32441 (hidden notes). The remote operator sees it on sync.
|
||||
|
||||
---
|
||||
|
||||
## Reference
|
||||
- Templates: `projects/dataforth-dos/dsca33-45-templates.json` (56 models)
|
||||
- Miner: `projects/dataforth-dos/tools/mine-hoffman-dsca.py`
|
||||
- Hoffman API creds: vault `clients/dataforth/hoffman-product-api` (read = `GET .../TestReportDataFiles/{serial}`)
|
||||
- Memory: `project_dsca33_45_spec_gap` (updated — resolved via Hoffman, not John)
|
||||
@@ -0,0 +1,70 @@
|
||||
# Email draft — to John Lehman, 2026-06-17
|
||||
|
||||
**To:** John Lehman <jlehman@dataforth.com>
|
||||
**From:** Mike Swanson <mike@azcomputerguru.com>
|
||||
**Subject:** Test Datasheet Problems — What's Going On and How We're Fixing It
|
||||
**Status:** DRAFT (not yet sent)
|
||||
|
||||
---
|
||||
|
||||
John,
|
||||
|
||||
I dug all the way into the datasheet issues you and Peter flagged (the 8B35 RTD certs the Wellbore Integrity audit caught), and I found the original problem plus a few related things worth knowing about. Good news up front: **your test data and original datasheets are accurate — nothing is corrupting your measurements.** The problems are all on our side, in the system that loads the data and re-creates the datasheets for the website, and **we'll be making the fixes** — that piece is our responsibility.
|
||||
|
||||
**First, how the system works (so the rest makes sense)**
|
||||
|
||||
When a unit is tested, the DOS test station produces **two** things:
|
||||
1. The **datasheet** you're used to seeing — a text file the station writes at test time. This is the original, and it's correct.
|
||||
2. A **raw data log** (the `.DAT` file) — the underlying numbers.
|
||||
|
||||
The website doesn't store that original datasheet. Our system loads the raw log into a database and then **rebuilds** the datasheet from the database when it's needed. That rebuild step is where the errors are. So the unit you tested is fine and the original sheet is fine — the *regenerated* copy on the website is what's off.
|
||||
|
||||
**Problem 1 — RTD modules say "resistance" when they should say "temperature" (this is the audit finding)**
|
||||
|
||||
An RTD senses temperature by changing its resistance, but on a Dataforth datasheet the input column is shown as **Temperature (°C)** — exactly what your original sheets do. Our rebuild has a bug: for RTD modules it labels that column **"Rin (ohms)"** even though it's printing the temperature numbers underneath, so the header and the values disagree. That's precisely what the auditor caught on the 8B35.
|
||||
|
||||
Important: on the 8B35 cert the **measured values are correct** — it's the column label that's wrong (and the positive numbers lost their leading "+"). This affects every RTD product, not just the 8B35 — roughly 24,000 certificates (8B35, DSCA34, the SCM5B34/35 family). We've already written and tested the fix against your original sheets; it produces the correct "Temp. (°C)" column with no effect on any non-RTD product.
|
||||
|
||||
**Problem 2 — DSCA datasheets have the wrong Final Test lines (the "missing lines / wrong headers")**
|
||||
|
||||
For the DSCA line, our rebuild is using the **wrong master list of test parameters** — so you get wrong parameter names, wrong spec limits, values on the wrong rows, and some lines dropping off. This is the "Final Test lines are missing" part of your report, and it affects the DSCA line broadly (up to ~78,000 certs). One clarification: DSCA38 is a bridge/strain-gauge module — its issue is this one; the DSCA *RTD* module is DSCA34, which has both this and Problem 1. This one's a real rebuild on our end (matching each DSCA sub-type to its spec file), so it'll take longer than the others.
|
||||
|
||||
**Problem 3 — some units show an earlier same-day test instead of the final one**
|
||||
|
||||
If a unit was tested more than once on the **same day** (a re-trim, say), our system can keep an *earlier* run instead of the final accepted one, so the website cert may show non-final numbers. It affects about **311 units** across your history. It matters for audits because the **last** same-day run is the one that belongs on the cert. We'll change our database rule so the latest same-day run wins.
|
||||
|
||||
**Problem 4 — two batches of units never made it into the system**
|
||||
|
||||
I also found units that were tested and have an original datasheet but **no** record in the database/website. Two separate reasons:
|
||||
|
||||
- **A serial-number format our importer can't read (~9,500 records).** In the DOS days, when a serial was too long for the old 8-character filename limit, the software shortened it with a letter code — e.g. `10243-1` is written as `A243-1`. Our importer only recognizes serials that start with a number, so it **silently skips** every letter-coded unit. The data is sitting right there in the logs; we just aren't reading it. Across all your logs that's about **840 units / 9,500 records / 141 models** never imported. We'll update the importer to recognize and decode those serials, which recovers the whole backlog.
|
||||
|
||||
- **A one-time casualty of the cryptolocker incident (~379 units).** This is *not* an ongoing gap — we import every 15 minutes and everything since has come through cleanly. Here's what happened: the DOS stations append all of a given model's results into a single, shared log file. During the incident, stations were failing to sync, and in the recovery we didn't yet realize they were appending to one filename — so for a couple of weeks, fresh logs coming off the DOS machines overwrote the accumulated server-side logs, wiping the earlier history those files held. The data backs this up: the affected units are confined to **October 2025 through January 2026** (most in December 2025) and **stop cold after January**, on three stations (TS-4L, TS-4R, TS-1R) — the exact fingerprint of a one-time overwrite during the incident, not a recurring problem. The good news: the **original datasheet text files for these units still exist** on the server, so we can backfill them from those.
|
||||
|
||||
**The bottom line**
|
||||
|
||||
- Your **test data and original datasheets are intact and accurate** — I verified the database matches the originals to the number across ~11,000 units, with zero cases of data being read in wrong.
|
||||
- Everything above is in **our** load-and-rebuild software, or was a one-time incident casualty — not a measurement or test problem.
|
||||
- For the **audit specifically**: the 8B35 measured values are right; it's the column heading, and that fix is ready to go.
|
||||
|
||||
**What we're doing next**
|
||||
|
||||
We'll tackle them in this order unless you'd rather reprioritize:
|
||||
1. RTD column-label fix — clears the audit issue (ready now).
|
||||
2. Missing-units importer fix — recovers the ~9,500-record backlog.
|
||||
3. Same-day-run rule — so certs show the final run.
|
||||
4. DSCA Final Test rebuild — the larger one.
|
||||
5. Backfill the ~379 incident-era units from their original datasheet files.
|
||||
|
||||
The only thing I need from you is a thumbs-up on that priority order (and on backfilling the 379 from their original sheets). Everything else is on us.
|
||||
|
||||
I've documented all of this in detail and am glad to walk your team through any of it on a call.
|
||||
|
||||
Thanks,
|
||||
Mike Swanson
|
||||
AZ Computer Guru
|
||||
(520) 304-8300
|
||||
|
||||
---
|
||||
|
||||
*Supporting detail: `DATASHEET-RTD-BUG-DIAGNOSIS-2026-06-17.md` (Problems 1 & 2), `PARSING-FIDELITY-VERDICT-2026-06-17.md` + `CONFLICT-RULE-FIX-PROPOSAL-2026-06-17.md` (Problem 3), `MISSING-UNITS-REPORT-FOR-JOHN-2026-06-17.md` (Problem 4).*
|
||||
@@ -0,0 +1,99 @@
|
||||
# Test Datasheets Missing From the Database/Website — Findings
|
||||
|
||||
**To:** John Lehman (Engineering)
|
||||
**From:** Mike Swanson, AZ Computer Guru
|
||||
**Date:** 2026-06-17
|
||||
**Scope:** Why some tested units have a staged datasheet but no record in testdatadb / on the website.
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
I cross-checked **all 11,921 staged datasheet files** (the `.TXT` the test stations produce) against the database. **608 had no matching database record.** They fall into two distinct causes:
|
||||
|
||||
| Cause | Units | Recoverable? |
|
||||
|---|---:|---|
|
||||
| **1. Encoded / non-standard serial numbers the importer skips** | **229** | **Yes** — the data exists, the importer just doesn't read it |
|
||||
| **2. One-time log loss during the cryptolocker incident/recovery** | **379** | Yes — from the staged `.TXT`, which still exists on disk |
|
||||
|
||||
The first cause is the important one: it is a **software limitation we can fix**, and its true reach is far larger than these 608 — see "Full scope" below.
|
||||
|
||||
---
|
||||
|
||||
## Cause 1 — Encoded serial numbers are silently skipped (229 units, fixable)
|
||||
|
||||
**What happens:** When a serial number is too long for the DOS 8.3 filename, the test program encodes the first two digits as a letter (e.g. `10243-1` is written as `A243-1`; `10` -> `A`). For these units, the serial is stored **with a leading letter** inside the log file:
|
||||
|
||||
```
|
||||
"A243-1","01-21-2025" <- real serial 10243-1, model 5B45-25D
|
||||
```
|
||||
|
||||
The database importer recognizes a record only when the serial **starts with a digit**. A serial that starts with a letter never matches, so the whole record is **silently dropped** — it is never imported, never rendered, never sent to the website.
|
||||
|
||||
**Confirmed:** the encoded serials are present in the `.DAT` logs (e.g. `5BLOG\45-25D.DAT` contains `"A243-1","01-21-2025"`), and the decoded form (`10243-1`) appears in no log and in no database row. So the data exists; the importer simply can't read it.
|
||||
|
||||
**Of the 608 missing, 229 are this case:**
|
||||
- 212 are hex-encoded serials (all `A`-prefix, i.e. `10xxx` serials)
|
||||
- 17 are other non-standard serial formats the same rule rejects (e.g. `TEST-1`, `178540-A1`, `A-1`)
|
||||
- Stations: TS-11R (142), TS-11L (59), TS-8R (11), TS-8L (17)
|
||||
- Dates: late 2025 through 2026
|
||||
- Example models: SCM5B47K-05, SCM5B38-04, SCM5B34-01, 8B45-02, SCM5B36-02, 8B32-01, 5B45-25D, DSCA45-01
|
||||
|
||||
**Examples (encoded -> real serial):**
|
||||
```
|
||||
A243-1 -> 10243-1 5B45-25D 02-03-2026 TS-11L
|
||||
A244-1 -> 10244-1 SCM5B30-02 03-26-2026 TS-11L
|
||||
A276-1 -> 10276-1 DSCA39-05 05-07-2026 TS-11L
|
||||
A328-1 -> 10328-1 DSCA45-08 02-17-2026 TS-11L
|
||||
```
|
||||
|
||||
### Full scope of this bug (beyond the 608)
|
||||
|
||||
The 229 above are only the units that *also* still have a staged `.TXT` on disk. Scanning **every** `.DAT` log across all stations and the central history logs, the importer is dropping:
|
||||
|
||||
- **840 distinct encoded serial numbers**
|
||||
- **9,510 individual test records**
|
||||
- across **141 models**
|
||||
- of which **831 of 840 serials are absent from the database**
|
||||
|
||||
So this single serial-format limitation is keeping on the order of **~9,500 test results out of the database and off the website.**
|
||||
|
||||
**Fix:** teach the importer to (a) accept a serial that starts with a letter and (b) decode it back to its real number (`A243-1` -> `10243-1`) before storing — matching how the longer serials (the `H`-prefix range) are already handled. This is a one-function change to the import parser. It would recover the 229 units here plus the ~9,500-record backlog. (I will write this up as a separate proposed change for review; no code has been changed.)
|
||||
|
||||
---
|
||||
|
||||
## Cause 2 — One-time log loss during the cryptolocker incident/recovery (379 units)
|
||||
|
||||
These have ordinary numeric serials (no encoding issue), but their raw test data is **no longer in any log file** we import from. **This is not an ongoing gap** — the import runs every 15 minutes and everything since has come through cleanly.
|
||||
|
||||
**What happened:** the DOS stations append all of a given model's results into a single shared per-model `.DAT` file. During the crypto incident, stations were failing to sync, and in the recovery the appended-to-one-filename behavior wasn't yet understood — so for ~2 weeks, freshly-created logs coming off the DOS machines overwrote the accumulated server-side logs, wiping the earlier history those files held.
|
||||
|
||||
**Evidence (date/station fingerprint of a one-time overwrite):** the 379 affected units are confined to **2025-10 → 2026-01 and stop cold after January 2026**, concentrated on three stations:
|
||||
|
||||
| Test month | Units | | Station | Units |
|
||||
|---|---:|---|---|---:|
|
||||
| 2025-10 | 95 | | TS-4L | 185 |
|
||||
| 2025-11 | 66 | | TS-4R | 171 |
|
||||
| 2025-12 | 210 | | TS-1R | 23 |
|
||||
| 2026-01 | 8 | | | |
|
||||
|
||||
**Confirmed example:** units `177097-1 ... 177097-16` (model DSCA33-05, tested 10-17-2025, TS-1R) appear in **no** log file anywhere under the test share. Their model's log (`DSCLOG\33-05.DAT`) now holds a later work order (`178644-*`, tested 02-26-2026) — the older appended history was overwritten; only the staged `.TXT` datasheet survived.
|
||||
|
||||
**Recovery:** the rendered datasheet `.TXT` still exists on disk for these units, so they can be backfilled directly from the `.TXT` (store the existing sheet as-is). The raw `.DAT` history is gone (the `Recovery-TEST` backup area the importer references does not exist on this server). Backfilling from the staged `.TXT` is the practical path and recovers all 379.
|
||||
|
||||
---
|
||||
|
||||
## Recommended actions
|
||||
|
||||
1. **Fix the importer's serial handling** (Cause 1) — recovers 229 staged units and ~9,500 total dropped records across 141 models. Highest value, single code change. Proposal to follow for review.
|
||||
2. **Backfill the 379 incident-era units (Cause 2) from their staged `.TXT` files** — recovers the datasheets that still exist on disk.
|
||||
3. **Recurrence:** Cause 2 was a one-time incident artifact (the 15-minute import has captured everything since January 2026), so no ongoing process change is required. Worth confirming the current sync no longer overwrites accumulated server-side logs with fresh DOS-side copies.
|
||||
|
||||
---
|
||||
|
||||
## How this was determined (for the record)
|
||||
|
||||
- Compared every `C:\Shares\test\STAGE\**\*.TXT` against `test_records` (serial, model, date, measured results).
|
||||
- For each missing unit, searched all `.DAT` sources (central HISTLOGS + every station's LOGS, 26,815 files) for the encoded and decoded serial.
|
||||
- Confirmed encoded serials are present in the logs but skipped by the import regex; confirmed overwritten units are absent from all logs and their model log now holds a newer work order.
|
||||
- Tools (read-only) committed under `projects/dataforth-dos/datasheet-pipeline/implementation/tools/`; raw data in `MISSING-UNITS-ROOTCAUSE-2026-06-17.txt`.
|
||||
@@ -0,0 +1,57 @@
|
||||
========== MISSING-UNITS ROOT CAUSE & SCOPE ==========
|
||||
Staged .TXT with SN : 11921
|
||||
Staged units MISSING from DB : 608
|
||||
|
||||
ROOT-CAUSE CATEGORIES (of the missing):
|
||||
Parser-drop (encoded serial w/ leading letter present in .DAT, regex rejects): 212
|
||||
Source .DAT has no record for this unit (data absent) : 379
|
||||
Other (decoded present, still missing - investigate) : 17
|
||||
|
||||
|
||||
by leading letter: A=212
|
||||
by station : TS-11R=142, TS-11L=59, TS-8R=11
|
||||
by model (top 12): SCM5B47K-05=18, SCM5B38-04=14, SCM5B34-01=13, 8B45-02=12, SCM5B36-02=11, 8B32-01=10, 5B45-25D=9, SCM5B49-05=8, 5B45-01=6, SCM5B39-05=6, DSCA45-01=5, SCM5B36-04=5
|
||||
date range : 01-13-2026 .. 12-11-2025
|
||||
samples :
|
||||
A243-1 -> 10243-1 5B45-25D 02-03-2026 TS-11L
|
||||
A243-2 -> 10243-2 5B45-25D 02-03-2026 TS-11L
|
||||
A244-1 -> 10244-1 SCM5B30-02 03-26-2026 TS-11L
|
||||
A255-1 -> 10255-1 5B45-25D 02-03-2026 TS-11L
|
||||
A255-2 -> 10255-2 5B45-25D 02-03-2026 TS-11L
|
||||
A276-1 -> 10276-1 DSCA39-05 05-07-2026 TS-11L
|
||||
A276-2 -> 10276-2 DSCA39-05 05-07-2026 TS-11L
|
||||
A328-1 -> 10328-1 DSCA45-08 02-17-2026 TS-11L
|
||||
A328-2 -> 10328-2 DSCA45-01C 02-17-2026 TS-11L
|
||||
A376-1 -> 10376-1 DSCA45-08 02-17-2026 TS-11L
|
||||
A376-2 -> 10376-2 DSCA45-08 02-17-2026 TS-11L
|
||||
A376-3 -> 10376-3 DSCA45-02 02-17-2026 TS-11L
|
||||
|
||||
DATA-ABSENT samples:
|
||||
177097-1 -> 177097-1 DSCA33-05 10-17-2025 TS-1R
|
||||
177097-10 -> 177097-10 DSCA33-05 10-17-2025 TS-1R
|
||||
177097-11 -> 177097-11 DSCA33-05 10-17-2025 TS-1R
|
||||
177097-12 -> 177097-12 DSCA33-05 10-17-2025 TS-1R
|
||||
177097-13 -> 177097-13 DSCA33-05 10-17-2025 TS-1R
|
||||
177097-14 -> 177097-14 DSCA33-05 10-17-2025 TS-1R
|
||||
177097-16 -> 177097-16 DSCA33-05 10-17-2025 TS-1R
|
||||
177097-2 -> 177097-2 DSCA33-05 10-17-2025 TS-1R
|
||||
177097-3 -> 177097-3 DSCA33-05 10-17-2025 TS-1R
|
||||
177097-4 -> 177097-4 DSCA33-05 10-17-2025 TS-1R
|
||||
|
||||
OTHER samples:
|
||||
A-1 -> A-1 SCM5B38-37 11-20-2025 TS-11R
|
||||
TEST-1 -> TEST-1 SCM5B392-04 11-12-2025 TS-11R
|
||||
TEST-2 -> TEST-2 SCM5B392-04 11-12-2025 TS-11R
|
||||
178540-A1 -> 178540-A1 SCM5B40-03 02-26-2026 TS-8L
|
||||
178540-A2 -> 178540-A2 SCM5B40-03 02-26-2026 TS-8L
|
||||
178540-A3 -> 178540-A3 SCM5B40-03 02-26-2026 TS-8L
|
||||
178540-A4 -> 178540-A4 SCM5B40-03 02-26-2026 TS-8L
|
||||
178540-B1 -> 178540-B1 SCM5B40-03 02-26-2026 TS-8L
|
||||
178540-B2 -> 178540-B2 SCM5B40-03 02-26-2026 TS-8L
|
||||
178540-B3 -> 178540-B3 SCM5B40-03 02-26-2026 TS-8L
|
||||
|
||||
FULL .DAT BLIND SPOT (all letter-prefixed serials the importer skips, not just staged):
|
||||
distinct letter-prefixed serials in .DAT : 840
|
||||
total letter-prefixed records (dropped) : 9510
|
||||
distinct models affected : 141
|
||||
of those serials, DECODED form absent from DB: 831 / 840
|
||||
@@ -0,0 +1,34 @@
|
||||
========== PARSING FIDELITY REPORT ==========
|
||||
Staged .TXT files scanned : 11922
|
||||
- no SN line (non-standard fmt): 1
|
||||
- SN found / compared : 11921
|
||||
- .TXT w/o 5 accuracy rows : 239
|
||||
Unique SNs looked up in DB : 11811
|
||||
SNs present in DB : 11239
|
||||
|
||||
EXPLAINED (not parsing faults):
|
||||
Consistent (SN+model+date+5 error% match) : 11226
|
||||
Retest, DB newer date than .TXT : 35
|
||||
Retest same-day (stim matches, run differs): 42
|
||||
VAS/single-point fmt (no 5-row block) : 5
|
||||
Serial collision (generic SN, diff family): 2
|
||||
|
||||
NEEDS REVIEW (potential genuine issues):
|
||||
Missing from DB (after hex-decode) : 608
|
||||
Model variant mismatch (same family) : 2
|
||||
DB OLDER than .TXT (stale DB?) : 1
|
||||
GENUINE error% fault (stim ALSO differs) : 0
|
||||
Accuracy-row-count diff : 0
|
||||
|
||||
COLLISION (informational) (first 20):
|
||||
1-1: txt=SCM5B34-02 db=SCMVAS-M300
|
||||
1-2: txt=SCM5B34-02 db=SCMVAS-M300
|
||||
|
||||
MODEL VARIANT MISMATCH (first 20):
|
||||
A819-1: txt=8B35-01 db=8B36-04
|
||||
A821-2: txt=8B35-04 db=8B36-01
|
||||
|
||||
DB OLDER THAN .TXT (first 20):
|
||||
A821-1: txt=02-25-2026 db=2026-01-13
|
||||
|
||||
MISSING-FROM-DB (first 30): A243-1 (dec 10243-1), A243-2 (dec 10243-2), A244-1 (dec 10244-1), A255-1 (dec 10255-1), A255-2 (dec 10255-2), A276-1 (dec 10276-1), A276-2 (dec 10276-2), A328-1 (dec 10328-1), A328-2 (dec 10328-2), A376-1 (dec 10376-1), A376-2 (dec 10376-2), A376-3 (dec 10376-3), A377-1 (dec 10377-1), A377-2 (dec 10377-2), A377-3 (dec 10377-3), A405-1 (dec 10405-1), A405-2 (dec 10405-2), A405-3 (dec 10405-3), A405-4 (dec 10405-4), A417-1 (dec 10417-1), A417-2 (dec 10417-2), A561-1 (dec 10561-1), A561-2 (dec 10561-2), A561-3 (dec 10561-3), A561-4 (dec 10561-4), A561-5 (dec 10561-5), A561-6 (dec 10561-6), A601-1 (dec 10601-1), A601-2 (dec 10601-2), A602-1 (dec 10602-1)
|
||||
@@ -0,0 +1,50 @@
|
||||
# Parsing Fidelity Verdict — testdatadb ingestion vs original staged datasheets
|
||||
|
||||
**Date:** 2026-06-17 · **Host:** AD2 · **Scope:** all 11,922 staged original `.TXT` datasheets vs the PostgreSQL `test_records`
|
||||
**Raw report:** `PARSING-FIDELITY-REPORT-2026-06-17.txt` · **Tool:** `datasheet-pipeline/implementation/tools/validate-parsing.js`
|
||||
|
||||
## Verdict
|
||||
|
||||
**Two distinct questions, two answers:**
|
||||
|
||||
1. **Is the parser faithful to the `.DAT` record it reads?** YES — 0 genuine parse faults across 11,239 comparable records. Every value the importer stores is byte-exact; no misreads, no mis-segmentation.
|
||||
2. **Does each DB row faithfully reproduce the unit's *final* test sheet?** NOT always. The DB is one-row-per-serial, and for units re-tested **on the same calendar date** the conflict rule (strictly-greater date) freezes on an arbitrary same-day run instead of the latest. Whole-source sweep: **311 (serial,date) groups where the DB holds a non-latest same-day run** (see `SAMEDAY-RETEST-EXPOSURE-2026-06-17.txt`). This is a data-model / conflict-rule defect, not a parser fault — fix proposed in `CONFLICT-RULE-FIX-PROPOSAL-2026-06-17.md`.
|
||||
|
||||
The remaining staged-sample "mismatches" were explained by legitimate later-date retests (latest-wins working), reused generic serials, VAS format, or legacy out-of-scope units.
|
||||
|
||||
## Method
|
||||
|
||||
Compared each staged original `.TXT` (the DOS-station ground truth, written *before* ingestion) against the DB record's `raw_data` (parsed from the `.DAT`). The cross-check keyed on **scale-invariant data**:
|
||||
- **Error (%)** — dimensionless, identical in `.DAT` and `.TXT` for every family (immune to mV scaling and current-output V→mA conversion). The primary fidelity signal.
|
||||
- **Stim setpoints** (scale-aware) — used to confirm the *same unit/test* when error values differed (retest vs wrong-record).
|
||||
- Serial (with hex-prefix decode), model (SCM-prefix normalized), date.
|
||||
|
||||
## Results (11,921 staged files with an SN)
|
||||
|
||||
| Outcome | Count | Meaning |
|
||||
|---|---:|---|
|
||||
| **Consistent** (SN+model+date+5×error%) | **11,226** | Faithful parse, confirmed |
|
||||
| Retest — DB date newer than `.TXT` | 35 | ON-CONFLICT updated DB to a later test (expected) |
|
||||
| Retest — same date, stim matches, run differs | 42 | **FAITHFULNESS VIOLATION** — unit tested 2+ times same day; DB froze on a non-latest run (strictly-greater-date rule). Staged subset of the 311 whole-source cases. |
|
||||
| VAS/single-point format | 5 | No 5-row accuracy block (SCMVAS) — not comparable by this method |
|
||||
| Serial collision (generic SN, diff family) | 2 | `1-1`/`1-2` reused across products; unique-on-serial keeps one |
|
||||
| **Genuine parse fault** | **0** | — |
|
||||
| Model variant mismatch (same family) | 2 | `A819-1`/`A821-2` — reused serial across 8B35/8B36 (collision) |
|
||||
| DB older than `.TXT` | 1 | `A821-1` — same collision pair |
|
||||
| Accuracy-row-count diff | 0 | — |
|
||||
|
||||
### Why the "genuine" bucket collapsed to 0
|
||||
The last 16 suspects were all `SCM5B37K-1530` (K-thermocouple). Their stim values matched the same 5 nominal setpoints (-50/112.5/275/437.5/600 °C) but differed by ~0.06 °C run-to-run — because the thermocouple input is a *measured analog value*, not an exact setpoint. A scale+relative stim tolerance correctly classifies them as same-day retests. A real segmentation fault would show a different setpoint *structure*; none did.
|
||||
|
||||
## Two follow-up items (NOT parsing-correctness bugs)
|
||||
|
||||
1. **608 staged originals have no DB record** (mostly A-prefix `10xxx` serials, e.g. `A243-1` = `10243-1`, model `5B45-25D`). These exist as staged `.TXT` but are absent from the DB under both decoded and encoded serial. This is an **ingestion-completeness** question (the source `.DAT` for these units appears to be out of the import scan scope, or these are custom `-NND` variants), separate from parsing fidelity. Worth a completeness pass: confirm which `.DAT` paths the importer scans and whether these models' `.DAT` files are present.
|
||||
2. **Same-day retests don't apply "latest wins" (PRIMARY DEFECT).** The `ON CONFLICT` rule updates only when `EXCLUDED.test_date > test_records.test_date` (strictly greater). For a unit tested 2+ times on one date, the rule freezes on whichever same-day run the import processed first and never advances to the latest — so the DB (and the website cert) can show non-final measured values. Whole-source exposure (981,716 records / 406,549 serials): **6,515 same-day multi-run events across 5,977 serials; the DB holds a non-latest run for 311 of them** (3,803 already on latest, 984 superseded by a later-date retest, 1,417 serial absent). Fix proposed in `CONFLICT-RULE-FIX-PROPOSAL-2026-06-17.md`. Directly audit-relevant: same-day runs are typically trim/re-test iterations and the **last** run is the accepted cert result.
|
||||
|
||||
## How to re-run
|
||||
|
||||
```bash
|
||||
cd C:\Shares\testdatadb
|
||||
node <path-to>/validate-parsing.js [optional-report-path]
|
||||
# Reads C:\Shares\test\STAGE\**\*.TXT and compares to test_records. Read-only.
|
||||
```
|
||||
@@ -0,0 +1,29 @@
|
||||
|
||||
========== SAME-DAY RETEST EXPOSURE (whole source) ==========
|
||||
Records parsed : 981716
|
||||
Distinct serials in source : 406549
|
||||
Serial+date with same-day multi-runs : 6515
|
||||
Distinct serials affected : 5977
|
||||
|
||||
Of those same-day multi-run (serial,date) groups, the DB row:
|
||||
matches the LATEST same-day run : 3803
|
||||
does NOT hold the latest run : 311 <-- faithfulness violations
|
||||
holds an even newer-date test (ok) : 984
|
||||
serial absent from DB : 1417
|
||||
|
||||
Examples (not-latest):
|
||||
4321-1 (2020-07-16, 2 runs): DB sig != latest
|
||||
82001-1 (2012-09-05, 6 runs): DB sig != latest
|
||||
608-55 (2018-01-28, 2 runs): DB sig != latest
|
||||
610-7 (2020-03-05, 2 runs): DB sig != latest
|
||||
1-2: DB date 2017-02-06 < multirun 2021-07-07
|
||||
1-2: DB date 2017-02-06 < multirun 2021-08-19
|
||||
1-2: DB date 2017-02-06 < multirun 2021-08-23
|
||||
1-2: DB date 2017-02-06 < multirun 2021-08-16
|
||||
1-2: DB date 2017-02-06 < multirun 2021-08-22
|
||||
1-2: DB date 2017-02-06 < multirun 2021-08-26
|
||||
1-2: DB date 2017-02-06 < multirun 2017-08-31
|
||||
1-2: DB date 2017-02-06 < multirun 2021-06-23
|
||||
1-2: DB date 2017-02-06 < multirun 2021-08-02
|
||||
1-2: DB date 2017-02-06 < multirun 2021-08-03
|
||||
1-2: DB date 2017-02-06 < multirun 2022-06-22
|
||||
File diff suppressed because one or more lines are too long
@@ -7,6 +7,17 @@
|
||||
|
||||
const { getFamily } = require('../parsers/spec-reader');
|
||||
|
||||
// DSCA per-model Final-Test templates (Fix 2 STAGE 1 output). Each entry is
|
||||
// { accOut: 'Output (V)'|'Output (mA)', rows: [{name, spec}, ...] }, extracted
|
||||
// byte-accurately from the staged originals. This is the AUTHORITATIVE source
|
||||
// of DSCA parameter names + specs + accuracy output label; loaded once.
|
||||
let DSCA_TEMPLATES = {};
|
||||
try {
|
||||
DSCA_TEMPLATES = require('../dsca-templates.json');
|
||||
} catch (e) {
|
||||
DSCA_TEMPLATES = {};
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// DATA LINES: parameter names and units per family
|
||||
// -------------------------------------------------------------------------
|
||||
@@ -189,12 +200,18 @@ function parseRawData(rawData, family) {
|
||||
}
|
||||
}
|
||||
|
||||
// Next line: step response / placeholders
|
||||
// Next line: step response / placeholders.
|
||||
// SCM5B/8B: "0","0",value DSCT: just value. Many DSCA models OMIT this bare
|
||||
// line and go straight to the STATUS groups; consuming a STATUS group here
|
||||
// drops a Final-Test row (the "lines drop" defect). For DSCA, skip consuming
|
||||
// when the line is actually a STATUS group (starts with PASS/FAIL).
|
||||
if (lineIdx < lines.length) {
|
||||
const parts = parseCSVLine(lines[lineIdx++]);
|
||||
// SCM5B/8B: "0","0",value DSCT: just value
|
||||
const lastVal = parts[parts.length - 1];
|
||||
result.stepResponse = parseFloat(lastVal) || 0;
|
||||
const looksLikeStatus = /^"?(PASS|FAIL)/i.test(lines[lineIdx].trim());
|
||||
if (!(family === 'DSCA' && looksLikeStatus)) {
|
||||
const parts = parseCSVLine(lines[lineIdx++]);
|
||||
const lastVal = parts[parts.length - 1];
|
||||
result.stepResponse = parseFloat(lastVal) || 0;
|
||||
}
|
||||
}
|
||||
|
||||
// Remaining lines: STATUS groups
|
||||
@@ -297,6 +314,45 @@ function formatMeasured(statusStr) {
|
||||
return { passFail, formatted, value };
|
||||
}
|
||||
|
||||
/**
|
||||
* DSCA measured-value formatter. The DSCA Final-Test STATUS$ entries use the
|
||||
* trailing digit as a literal decimal-place count (code N -> toFixed(N)),
|
||||
* UNLIKE the 5B/8B QB format strings where code 2 means 1 decimal. Returns the
|
||||
* trimmed value string (caller column-aligns it) plus the PASS/FAIL prefix.
|
||||
*/
|
||||
function formatMeasuredExact(statusStr) {
|
||||
if (!statusStr || statusStr.length <= 4) return null;
|
||||
const passFail = statusStr.substring(0, 4);
|
||||
const decimalDigit = statusStr[statusStr.length - 1];
|
||||
// char at index 4 is either a space (positive) or '-' (negative); start there
|
||||
// so negative signs survive (e.g. "PASS-4.2424060" -> "-4", not "4").
|
||||
const valueStr = statusStr.substring(4, statusStr.length - 1).trim();
|
||||
const parsed = parseFloat(valueStr);
|
||||
if (isNaN(parsed)) return { passFail, valStr: valueStr, value: NaN };
|
||||
// The DOS QuickBASIC stored/computed these as single-precision floats, so the
|
||||
// value at the half-rounding boundary rounds the way single precision rounds,
|
||||
// not double. Recover the single (Math.fround) before formatting — without it,
|
||||
// double-precision toFixed flips last-digit boundaries (e.g. 9.9995 -> "9.999"
|
||||
// here but "10.000" in the original; 46.85 -> "46.9" vs "46.8").
|
||||
const value = Math.fround(parsed);
|
||||
const d = parseInt(decimalDigit, 10);
|
||||
const valStr = isNaN(d) ? value.toFixed(1) : value.toFixed(d);
|
||||
return { passFail, valStr, value };
|
||||
}
|
||||
|
||||
/**
|
||||
* Split a DSCA template spec string into its value part and trailing unit.
|
||||
* e.g. "< 30 mA" -> { valuePart: "< 30", unit: "mA" } (internal spacing kept,
|
||||
* so the value part can be right-aligned to match the staged column layout).
|
||||
* "+/- 11 ppm/mA" -> { valuePart: "+/- 11", unit: "ppm/mA" }.
|
||||
*/
|
||||
function splitSpecUnit(spec) {
|
||||
const s = String(spec);
|
||||
const m = s.match(/^(.*\S)\s+(\S+)$/);
|
||||
if (m) return { valuePart: m[1], unit: m[2] };
|
||||
return { valuePart: s.trim(), unit: '' };
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// Format TSPEC display string from spec values
|
||||
// -------------------------------------------------------------------------
|
||||
@@ -438,12 +494,9 @@ function buildTSpecs(specs, family, stepResponse) {
|
||||
|
||||
function formatAccuracyLine(point, sensorNum, maxIn) {
|
||||
let stimStr;
|
||||
if (sensorNum >= 3 && sensorNum <= 6) {
|
||||
if ((sensorNum >= 3 && sensorNum <= 6) || sensorNum === 7) {
|
||||
// Temperature: +####.##
|
||||
stimStr = formatSigned(point.stim, 2, 8);
|
||||
} else if (sensorNum === 7) {
|
||||
// Resistance: #####.##
|
||||
stimStr = point.stim.toFixed(2).padStart(8);
|
||||
} else {
|
||||
// Voltage/Current: +###.###
|
||||
const scale = (maxIn != null && maxIn < 1) ? 1000 : 1;
|
||||
@@ -499,6 +552,14 @@ function generateExactDatasheet(record, specs) {
|
||||
return generateSCMVASDatasheet(record);
|
||||
}
|
||||
|
||||
// DSCA STAGE 2: the per-model staged template is the authoritative Final-Test
|
||||
// layout (names + specs + accuracy output label). No template means no staged
|
||||
// original was available -> do not guess the layout; skip this cert.
|
||||
const dscaTpl = (family === 'DSCA')
|
||||
? (DSCA_TEMPLATES[(record.model_number || '').trim()] || null)
|
||||
: null;
|
||||
if (family === 'DSCA' && !dscaTpl) return null;
|
||||
|
||||
const parsed = (family === 'SCM7B')
|
||||
? parse7BRawData(record.raw_data)
|
||||
: parseRawData(record.raw_data, family);
|
||||
@@ -557,17 +618,19 @@ function generateExactDatasheet(record, specs) {
|
||||
|
||||
// Input column header based on sensor type
|
||||
let inputHeader;
|
||||
if (sensorNum >= 3 && sensorNum <= 6) {
|
||||
if ((sensorNum >= 3 && sensorNum <= 6) || sensorNum === 7) {
|
||||
inputHeader = ' Temp. (C)';
|
||||
} else if (sensorNum === 2 || sensorNum === 9) {
|
||||
inputHeader = ' Iin (mA)';
|
||||
} else if (sensorNum === 7) {
|
||||
inputHeader = ' Rin (ohms)';
|
||||
} else {
|
||||
inputHeader = (maxIn != null && maxIn < 1) ? ' Vin (mV)' : ' Vin (V)';
|
||||
}
|
||||
lines.push(' ' + inputHeader + ' Vout (V) Vout (V)* Error (%) Status');
|
||||
lines.push(TAB5 + '========== ========== ========== ========= ========');
|
||||
// DSCA labels its accuracy output column "Output (V)"/"Output (mA)" (from the
|
||||
// template) with '-' rule separators; 5B/8B/etc. use "Vout (V)" with '='.
|
||||
const accOut = (family === 'DSCA' && dscaTpl) ? dscaTpl.accOut : 'Vout (V)';
|
||||
const accSep = (family === 'DSCA') ? '-' : '=';
|
||||
lines.push(' ' + inputHeader + ' ' + accOut + ' ' + accOut + '* Error (%) Status');
|
||||
lines.push(TAB5 + accSep.repeat(10) + ' ' + accSep.repeat(10) + ' ' + accSep.repeat(10) + ' ' + accSep.repeat(9) + ' ' + accSep.repeat(8));
|
||||
|
||||
for (const point of parsed.accuracy) {
|
||||
lines.push(formatAccuracyLine(point, sensorNum, maxIn));
|
||||
@@ -579,6 +642,74 @@ function generateExactDatasheet(record, specs) {
|
||||
// QB column positions (1-indexed): TAB(31), TAB(47), TAB(60-speclen), TAB(61), TAB(71)
|
||||
lines.push(' FINAL TEST RESULTS');
|
||||
lines.push('');
|
||||
if (family === 'DSCA') {
|
||||
// DSCA Final-Test renders from the per-model staged template: the rows give
|
||||
// the parameter names + specs (and accuracy label) directly; the value-bearing
|
||||
// raw_data STATUS groups map positionally onto the spec-bearing rows. Rows with
|
||||
// an empty spec (240VAC Withstand / Hi-Pot) carry no measured value and render
|
||||
// as PASS. Header/column scheme matches the staged originals.
|
||||
|
||||
// Value-bearing measurements in source order (drop "PASS"/"" padding entries).
|
||||
const measurements = [];
|
||||
for (const s of parsed.statusEntries) {
|
||||
const m = formatMeasuredExact(s);
|
||||
if (m) measurements.push(m);
|
||||
}
|
||||
const specRowCount = dscaTpl.rows.filter(r => (r.spec || '').trim()).length;
|
||||
// The simple positional zip is sound only when there is exactly one measured
|
||||
// value per spec-bearing row. When counts differ, this subtype measures slots
|
||||
// the template omits (e.g. an extra load pair); use the per-model slotMap
|
||||
// (absolute statusEntries index per spec-bearing row, derived from the staged
|
||||
// originals) to pull the right value. With no usable slotMap, skip rather than
|
||||
// misalign ("do not guess").
|
||||
const useSlot = (measurements.length !== specRowCount)
|
||||
&& Array.isArray(dscaTpl.slotMap) && dscaTpl.slotMap.length === specRowCount;
|
||||
if (measurements.length !== specRowCount && !useSlot) return null;
|
||||
|
||||
let h1 = setCol('', 12, 'Parameter');
|
||||
h1 = setCol(h1, 31, 'Measured Value*');
|
||||
h1 = setCol(h1, 51, 'Specification');
|
||||
h1 = setCol(h1, 69, 'Status');
|
||||
lines.push(h1);
|
||||
let h2 = setCol('', 4, '='.repeat(25));
|
||||
h2 = setCol(h2, 31, '='.repeat(15));
|
||||
h2 = setCol(h2, 48, '='.repeat(19));
|
||||
h2 = setCol(h2, 69, '='.repeat(6));
|
||||
lines.push(h2);
|
||||
|
||||
let mi = 0, si = 0;
|
||||
for (const row of dscaTpl.rows) {
|
||||
const spec = (row.spec || '').trim();
|
||||
let line = setCol('', 4, row.name);
|
||||
if (spec) {
|
||||
const su = splitSpecUnit(spec);
|
||||
const m = useSlot
|
||||
? formatMeasuredExact(parsed.statusEntries[dscaTpl.slotMap[si++]])
|
||||
: measurements[mi++];
|
||||
if (m) {
|
||||
// measured value right-justified ending at col 38, unit at col 40
|
||||
const v = String(m.valStr);
|
||||
line = setCol(line, 39 - v.length, v);
|
||||
if (su.unit) line = setCol(line, 40, su.unit);
|
||||
}
|
||||
// spec value-part right-justified ending at col 58, unit at col 60
|
||||
line = setCol(line, 59 - su.valuePart.length, su.valuePart);
|
||||
if (su.unit) line = setCol(line, 60, su.unit);
|
||||
line = setCol(line, 70, m ? m.passFail : 'PASS');
|
||||
} else {
|
||||
// no spec => 240VAC Withstand / Hi-Pot style row: blank measured + PASS
|
||||
line = setCol(line, 70, 'PASS');
|
||||
}
|
||||
lines.push(line);
|
||||
}
|
||||
// Footer load note ("Standard output load for test is ... ohms.") — printed
|
||||
// before the underline, only by the models whose staged original had it
|
||||
// (captured per-model in STAGE 1; not all current-output models print it).
|
||||
if (dscaTpl.loadNote) {
|
||||
lines.push('');
|
||||
lines.push(TAB5 + dscaTpl.loadNote);
|
||||
}
|
||||
} else {
|
||||
// QB: TAB(12); "Parameter"; TAB(30); "Measured Value"; TAB(51); "Specification "; TAB(70); "Status"
|
||||
let hdr1 = setCol('', 11, 'Parameter');
|
||||
hdr1 = setCol(hdr1, 29, 'Measured Value');
|
||||
@@ -628,6 +759,7 @@ function generateExactDatasheet(record, specs) {
|
||||
|
||||
lines.push(line);
|
||||
}
|
||||
} // end non-DSCA Final Test Results
|
||||
|
||||
// ---- Footer ----
|
||||
// 240 VAC / Hi-Pot (conditional by family/model)
|
||||
@@ -651,9 +783,6 @@ function generateExactDatasheet(record, specs) {
|
||||
let hp = setCol(TAB5 + 'Hi-Pot', 70, 'PASS');
|
||||
lines.push(hp);
|
||||
}
|
||||
} else if (family === 'DSCA') {
|
||||
lines.push(TAB5 + '240VAC Withstand' + ''.padEnd(50) + 'PASS');
|
||||
lines.push(TAB5 + 'Hi-Pot' + ''.padEnd(60) + 'PASS');
|
||||
} else if (family === 'DSCT') {
|
||||
lines.push(TAB5 + '240 VAC Withstand' + ''.padEnd(49) + 'PASS');
|
||||
lines.push(TAB5 + 'Hi-Pot' + ''.padEnd(60) + 'PASS');
|
||||
@@ -677,11 +806,6 @@ function generateExactDatasheet(record, specs) {
|
||||
lines.push(setCol(TAB5 + 'Pins Straight: __X__', 44, 'Module Header: __X__'));
|
||||
}
|
||||
|
||||
// DSCA current output load note
|
||||
if (family === 'DSCA' && specs && specs.OUTSIGTYPE && specs.OUTSIGTYPE.trim().toUpperCase() === 'CURRENT') {
|
||||
lines.push(TAB5 + 'Standard output load for test is 250 ohms.');
|
||||
}
|
||||
|
||||
lines.push('');
|
||||
lines.push(TAB5 + 'It is hereby certified that the above product is in conformance with');
|
||||
lines.push(TAB5 + 'all requirements to the extent specified. This product is not');
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
// Root-cause + scope of the 608 missing staged units (READ-ONLY) for the report to John.
|
||||
// Hypothesis: importer serial/date regex requires a leading digit, so hex-encoded
|
||||
// (leading-letter) serials in the .DAT are never matched -> records dropped.
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const db = require('./database/db');
|
||||
|
||||
const STAGE = 'C:/Shares/test/STAGE';
|
||||
const STRICT = /^"(\d+-\d+[A-Za-z]?)","(\d{2}-\d{2}-\d{4})"$/; // current importer regex
|
||||
const LOOSE = /^"([^"]+)","(\d{2}-\d{2}-\d{4})"$/; // any serial before a date
|
||||
const decode = sn => /^[A-Za-z]\d/.test(sn) ? String(sn.toUpperCase().charCodeAt(0) - 55) + sn.slice(1) : sn;
|
||||
|
||||
function walk(dir, re, out) { let it=[]; try{it=fs.readdirSync(dir,{withFileTypes:true})}catch{return out;}
|
||||
for(const e of it){const p=path.join(dir,e.name); if(e.isDirectory()) walk(p,re,out); else if(re.test(e.name)) out.push(p);} return out; }
|
||||
|
||||
(async () => {
|
||||
// ---- staged .TXT inventory ----
|
||||
const txts = walk(STAGE, /\.txt$/i, []);
|
||||
const staged = [];
|
||||
for (const f of txts) { let t; try{t=fs.readFileSync(f,'utf8')}catch{continue;}
|
||||
const sn=(t.match(/^\s*SN:\s*(\S+)/m)||[])[1]; if(!sn) continue;
|
||||
const model=(t.match(/^\s*Model:\s*(\S+)/m)||[])[1]||'';
|
||||
const date=(t.match(/^\s*Date:\s*(\d{2}-\d{2}-\d{4})/m)||[])[1]||'';
|
||||
const station=(f.match(/STAGE[\\\/]([^\\\/]+)/)||[])[1]||'';
|
||||
staged.push({ sn, dec: decode(sn), model, date, station, file: f });
|
||||
}
|
||||
|
||||
// ---- which staged decoded serials are in DB ----
|
||||
const decs=[...new Set(staged.map(s=>s.dec))]; const inDb=new Set();
|
||||
for(let i=0;i<decs.length;i+=1000){ const rows=await db.query('SELECT serial_number FROM test_records WHERE serial_number = ANY($1)',[decs.slice(i,i+1000)]); for(const r of rows) inDb.add(r.serial_number); }
|
||||
const missing = staged.filter(s=>!inDb.has(s.dec));
|
||||
|
||||
// ---- scan ALL .DAT sources: which serial tokens appear, strict vs letter-prefixed ----
|
||||
let dats=[]; walk('C:/Shares/test/Ate/HISTLOGS', /\.dat$/i, dats);
|
||||
let stations=[]; try{stations=fs.readdirSync('C:/Shares/test',{withFileTypes:true}).filter(d=>d.isDirectory()&&/^TS-\d+[LR]?$/i.test(d.name)).map(d=>d.name);}catch{}
|
||||
for(const s of stations) walk(path.join('C:/Shares/test',s,'LOGS'), /\.dat$/i, dats);
|
||||
|
||||
const looseSet=new Set(); const letterSet=new Set(); let letterRecs=0; const letterModels=new Set();
|
||||
let fi=0;
|
||||
for(const f of dats){ fi++; if(fi%5000===0) console.log(' scan '+fi+'/'+dats.length);
|
||||
let lines; try{lines=fs.readFileSync(f,'utf8').split('\n')}catch{continue;}
|
||||
let lastModel='';
|
||||
for(const l of lines){ const t=l.trim();
|
||||
const mm=t.match(/^"([A-Z0-9][A-Z0-9 \-]*)"$/i); if(mm && !/PASS|FAIL/.test(t) && !t.includes(',')) { lastModel=mm[1].trim(); continue; }
|
||||
const m=t.match(LOOSE); if(m){ const sn=m[1]; looseSet.add(sn);
|
||||
if(/^[A-Za-z]\d/.test(sn) && !STRICT.test(t)){ letterSet.add(sn); letterRecs++; if(lastModel) letterModels.add(lastModel); } }
|
||||
}
|
||||
}
|
||||
|
||||
// ---- categorize the missing ----
|
||||
const cat = { parserDrop: [], absent: [], decInDbButMiss: [] };
|
||||
for(const s of missing){
|
||||
if(letterSet.has(s.sn) || (/^[A-Za-z]\d/.test(s.sn) && looseSet.has(s.sn))) cat.parserDrop.push(s);
|
||||
else if(!looseSet.has(s.sn) && !looseSet.has(s.dec)) cat.absent.push(s);
|
||||
else cat.decInDbButMiss.push(s);
|
||||
}
|
||||
const by=(arr,k)=>{const m={};for(const x of arr){const v=(x[k]||'?');m[v]=(m[v]||0)+1;}return Object.entries(m).sort((a,b)=>b[1]-a[1]);};
|
||||
|
||||
// ---- full letter-prefixed population in .DAT and how much is absent from DB ----
|
||||
const letterDecs=[...letterSet].map(decode); const letterInDb=new Set();
|
||||
for(let i=0;i<letterDecs.length;i+=1000){ const rows=await db.query('SELECT serial_number FROM test_records WHERE serial_number = ANY($1)',[letterDecs.slice(i,i+1000)]); for(const r of rows) letterInDb.add(r.serial_number); }
|
||||
const letterMissingDistinct=letterDecs.filter(d=>!letterInDb.has(d)).length;
|
||||
|
||||
const out=[]; const L=s=>{out.push(s);console.log(s);};
|
||||
L('========== MISSING-UNITS ROOT CAUSE & SCOPE ==========');
|
||||
L('Staged .TXT with SN : '+staged.length);
|
||||
L('Staged units MISSING from DB : '+missing.length);
|
||||
L('');
|
||||
L('ROOT-CAUSE CATEGORIES (of the missing):');
|
||||
L(' Parser-drop (encoded serial w/ leading letter present in .DAT, regex rejects): '+cat.parserDrop.length);
|
||||
L(' Source .DAT has no record for this unit (data absent) : '+cat.absent.length);
|
||||
L(' Other (decoded present, still missing - investigate) : '+cat.decInDbButMiss.length);
|
||||
L('');
|
||||
L('PARSER-DROP breakdown by leading char: '+by(cat.parserDrop, 'sn').slice(0,1).length? '' : '');
|
||||
const lead=k=>{const m={};for(const x of cat.parserDrop){const c=x.sn[0].toUpperCase();m[c]=(m[c]||0)+1;}return Object.entries(m).sort((a,b)=>b[1]-a[1]);};
|
||||
L(' by leading letter: '+lead().map(([c,n])=>c+'='+n).join(', '));
|
||||
L(' by station : '+by(cat.parserDrop,'station').map(([c,n])=>c+'='+n).join(', '));
|
||||
L(' by model (top 12): '+by(cat.parserDrop,'model').slice(0,12).map(([c,n])=>c+'='+n).join(', '));
|
||||
L(' date range : '+(()=>{const ds=cat.parserDrop.map(s=>s.date).filter(Boolean).sort();return ds[0]+' .. '+ds[ds.length-1];})());
|
||||
L(' samples :'); cat.parserDrop.slice(0,12).forEach(s=>L(' '+s.sn+' -> '+s.dec+' '+s.model+' '+s.date+' '+s.station));
|
||||
if(cat.absent.length){ L(''); L('DATA-ABSENT samples:'); cat.absent.slice(0,10).forEach(s=>L(' '+s.sn+' -> '+s.dec+' '+s.model+' '+s.date+' '+s.station)); }
|
||||
if(cat.decInDbButMiss.length){ L(''); L('OTHER samples:'); cat.decInDbButMiss.slice(0,10).forEach(s=>L(' '+s.sn+' -> '+s.dec+' '+s.model+' '+s.date+' '+s.station)); }
|
||||
L('');
|
||||
L('FULL .DAT BLIND SPOT (all letter-prefixed serials the importer skips, not just staged):');
|
||||
L(' distinct letter-prefixed serials in .DAT : '+letterSet.size);
|
||||
L(' total letter-prefixed records (dropped) : '+letterRecs);
|
||||
L(' distinct models affected : '+letterModels.size);
|
||||
L(' of those serials, DECODED form absent from DB: '+letterMissingDistinct+' / '+letterSet.size);
|
||||
|
||||
if(process.argv[2]) fs.writeFileSync(process.argv[2], out.join('\n')+'\n');
|
||||
await db.close();
|
||||
})().catch(e=>{console.error(e);process.exit(1);});
|
||||
@@ -0,0 +1,104 @@
|
||||
// Whole-source sweep (READ-ONLY): find serials with same-day multi-runs (distinct values)
|
||||
// and measure how many the DB does NOT hold the latest run for. Scans the import's .DAT sources.
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const db = require('./database/db');
|
||||
|
||||
const ROOTS = ['C:/Shares/test/Ate/HISTLOGS']; // central combined logs first
|
||||
const STATION_BASE = 'C:/Shares/test';
|
||||
|
||||
function datFiles(dir, out) {
|
||||
let it = []; try { it = fs.readdirSync(dir, { withFileTypes: true }); } catch { return out; }
|
||||
for (const e of it) { const p = path.join(dir, e.name);
|
||||
if (e.isDirectory()) datFiles(p, out);
|
||||
else if (/\.dat$/i.test(e.name)) out.push(p);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
// signature of a record = the 5 Error(%) columns joined (distinguishes runs)
|
||||
function recSig(block) {
|
||||
const errs = [];
|
||||
for (const l of block) {
|
||||
if (/,"(PASS|FAIL)"/.test(l)) { const f = l.split(','); if (f.length >= 5) { errs.push(f[3].trim()); if (errs.length === 5) break; } }
|
||||
}
|
||||
return errs.length === 5 ? errs.join('|') : null;
|
||||
}
|
||||
|
||||
(async () => {
|
||||
// gather files: HISTLOGS, then station LOGS (mirrors import order; station = latest)
|
||||
let files = [];
|
||||
for (const r of ROOTS) datFiles(r, files);
|
||||
let stations = [];
|
||||
try { stations = fs.readdirSync(STATION_BASE, { withFileTypes: true }).filter(d => d.isDirectory() && /^TS-\d+[LR]?$/i.test(d.name)).map(d => d.name); } catch {}
|
||||
for (const s of stations) datFiles(path.join(STATION_BASE, s, 'LOGS'), files);
|
||||
console.log('Scanning ' + files.length + ' .DAT files (' + stations.length + ' stations + HISTLOGS)...');
|
||||
|
||||
// serial -> date -> { sigs:Set, last:sig }
|
||||
const map = new Map();
|
||||
let recCount = 0, fi = 0;
|
||||
for (const f of files) {
|
||||
fi++; if (fi % 3000 === 0) console.log(' ...' + fi + '/' + files.length + ' files, ' + recCount + ' records');
|
||||
let lines; try { lines = fs.readFileSync(f, 'utf8').split('\n'); } catch { continue; }
|
||||
let block = [];
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
const t = lines[i].trim();
|
||||
const sd = t.match(/^"(\d+-\d+[A-Za-z]?)","(\d{2}-\d{2}-\d{4})"$/);
|
||||
if (sd) {
|
||||
const sig = recSig(block);
|
||||
if (sig) {
|
||||
recCount++;
|
||||
const sn = sd[1]; const [mm,dd,yy] = sd[2].split('-'); const date = `${yy}-${mm}-${dd}`;
|
||||
let dm = map.get(sn); if (!dm) { dm = new Map(); map.set(sn, dm); }
|
||||
let e = dm.get(date); if (!e) { e = { sigs: new Set(), last: null }; dm.set(date, e); }
|
||||
e.sigs.add(sig); e.last = sig;
|
||||
}
|
||||
block = [];
|
||||
} else if (t) block.push(t);
|
||||
}
|
||||
}
|
||||
console.log('Parsed ' + recCount + ' records, ' + map.size + ' distinct serials.');
|
||||
|
||||
// find serials with same-day multi-runs (>=2 distinct sigs on one date)
|
||||
const multi = []; // { sn, date, runs, lastSig }
|
||||
for (const [sn, dm] of map) for (const [date, e] of dm) if (e.sigs.size >= 2) multi.push({ sn, date, runs: e.sigs.size, lastSig: e.last });
|
||||
console.log('Serials*date with same-day multi-runs (distinct values): ' + multi.length);
|
||||
const multiSerials = new Set(multi.map(m => m.sn));
|
||||
console.log('Distinct serials affected: ' + multiSerials.size);
|
||||
|
||||
// For each, check what the DB holds vs the latest same-day run
|
||||
const sns = [...multiSerials];
|
||||
const dbMap = new Map();
|
||||
for (let i = 0; i < sns.length; i += 1000) {
|
||||
const rows = await db.query('SELECT serial_number, test_date, raw_data FROM test_records WHERE serial_number = ANY($1)', [sns.slice(i, i+1000)]);
|
||||
for (const r of rows) dbMap.set(r.serial_number, r);
|
||||
}
|
||||
let notLatest = 0, dbNewer = 0, dbAbsent = 0, dbMatches = 0, examples = [];
|
||||
for (const m of multi) {
|
||||
const d = dbMap.get(m.sn);
|
||||
if (!d) { dbAbsent++; continue; }
|
||||
const dbDate = d.test_date && d.test_date.toISOString ? d.test_date.toISOString().slice(0,10) : String(d.test_date);
|
||||
if (dbDate > m.date) { dbNewer++; continue; } // DB has an even later test -> fine
|
||||
if (dbDate < m.date) { notLatest++; if (examples.length<15) examples.push(`${m.sn}: DB date ${dbDate} < multirun ${m.date}`); continue; }
|
||||
const dbSig = recSig((d.raw_data||'').split('\n').map(s=>s.trim()));
|
||||
if (dbSig === m.lastSig) dbMatches++;
|
||||
else { notLatest++; if (examples.length<15) examples.push(`${m.sn} (${m.date}, ${m.runs} runs): DB sig != latest`); }
|
||||
}
|
||||
|
||||
const out = [];
|
||||
const L = s => { out.push(s); console.log(s); };
|
||||
L('\n========== SAME-DAY RETEST EXPOSURE (whole source) ==========');
|
||||
L('Records parsed : ' + recCount);
|
||||
L('Distinct serials in source : ' + map.size);
|
||||
L('Serial+date with same-day multi-runs : ' + multi.length);
|
||||
L('Distinct serials affected : ' + multiSerials.size);
|
||||
L('');
|
||||
L('Of those same-day multi-run (serial,date) groups, the DB row:');
|
||||
L(' matches the LATEST same-day run : ' + dbMatches);
|
||||
L(' does NOT hold the latest run : ' + notLatest + ' <-- faithfulness violations');
|
||||
L(' holds an even newer-date test (ok) : ' + dbNewer);
|
||||
L(' serial absent from DB : ' + dbAbsent);
|
||||
if (examples.length) { L(''); L('Examples (not-latest):'); examples.forEach(x=>L(' '+x)); }
|
||||
if (process.argv[2]) fs.writeFileSync(process.argv[2], out.join('\n')+'\n');
|
||||
await db.close();
|
||||
})().catch(e => { console.error(e); process.exit(1); });
|
||||
@@ -0,0 +1,177 @@
|
||||
// Parsing-fidelity validation (READ-ONLY): every staged original .TXT vs the DB record.
|
||||
// Compares scale-invariant data: SN, model, date, and the 5 Error(%) accuracy values
|
||||
// (error% is dimensionless -> immune to mV scaling / current-output conversion, so a
|
||||
// mismatch means a real parsing/segmentation/identity fault, not a rendering transform).
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const db = require('./database/db');
|
||||
|
||||
const STAGE = 'C:/Shares/test/STAGE';
|
||||
const ERR_TOL = 0.003; // half-unit of 3-decimal display + margin
|
||||
const REPORT = process.argv[2] || null;
|
||||
|
||||
function walk(dir, out) {
|
||||
let items = [];
|
||||
try { items = fs.readdirSync(dir, { withFileTypes: true }); } catch { return out; }
|
||||
for (const it of items) {
|
||||
const p = path.join(dir, it.name);
|
||||
if (it.isDirectory()) walk(p, out);
|
||||
else if (/\.txt$/i.test(it.name)) out.push(p);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function parseTxt(txt) {
|
||||
const lines = txt.split(/\r?\n/);
|
||||
const get = re => { for (const l of lines) { const m = l.match(re); if (m) return m[1]; } return null; };
|
||||
const sn = get(/^\s*SN:\s*(\S+)/);
|
||||
const model = get(/^\s*Model:\s*(\S+)/);
|
||||
const date = get(/^\s*Date:\s*(\d{2}-\d{2}-\d{4})/);
|
||||
// accuracy rows: lines ending in PASS/FAIL with >=4 numeric tokens, before FINAL TEST
|
||||
const errs = [];
|
||||
const stims = [];
|
||||
for (const l of lines) {
|
||||
if (/FINAL TEST/i.test(l)) break;
|
||||
if (!/\b(PASS|FAIL)\b/.test(l)) continue;
|
||||
const nums = (l.match(/[+-]?\d*\.\d+|[+-]?\d+/g) || []).map(Number);
|
||||
if (nums.length >= 4) { errs.push(nums[3]); stims.push(nums[0]); } // [0]=stim [3]=Error(%)
|
||||
if (errs.length === 5) break;
|
||||
}
|
||||
return { sn, model, date, errs, stims };
|
||||
}
|
||||
|
||||
// Decode hex-prefix encoded serial (A-prefix files store the ENCODED SN inside):
|
||||
// leading [A-Z] -> (charCode-55) numeric prefix. H9553-13-style files already store
|
||||
// the decoded SN, which is numeric, so they don't match and pass through unchanged.
|
||||
function decodeSn(sn) {
|
||||
if (/^[A-Za-z]\d/.test(sn)) {
|
||||
const n = sn.toUpperCase().charCodeAt(0) - 55;
|
||||
return String(n) + sn.slice(1);
|
||||
}
|
||||
return sn;
|
||||
}
|
||||
const normModel = m => (m || '').toUpperCase().replace(/^SCM/, '');
|
||||
|
||||
function parseRawAcc(raw) {
|
||||
if (!raw) return { errs: [], stims: [] };
|
||||
const lines = raw.split('\n').map(s => s.trim()).filter(Boolean);
|
||||
const errs = [], stims = [];
|
||||
for (let i = 1; i < lines.length && errs.length < 5; i++) {
|
||||
const f = lines[i].split(',');
|
||||
if (f.length >= 5 && /"(PASS|FAIL)"/.test(lines[i])) {
|
||||
const e = parseFloat(f[3]), s = parseFloat(f[0]);
|
||||
if (!isNaN(e)) { errs.push(e); stims.push(s); }
|
||||
}
|
||||
}
|
||||
return { errs, stims };
|
||||
}
|
||||
// scale-aware + relative stim match (mV display = V*1000; analog inputs vary run-to-run).
|
||||
// Matching the 5-point setpoint pattern proves same unit/test -> correct segmentation.
|
||||
function stimMatch1(t, r) {
|
||||
return [r, r * 1000, r / 1000].some(c => Math.abs(t - c) <= Math.max(0.3, 0.005 * Math.abs(c)));
|
||||
}
|
||||
function stimsMatch(txt, raw) {
|
||||
return txt.length === 5 && raw.length === 5 && txt.every((t, i) => stimMatch1(t, raw[i]));
|
||||
}
|
||||
|
||||
(async () => {
|
||||
console.log('Scanning staged .TXT files...');
|
||||
const files = walk(STAGE, []);
|
||||
console.log('Found ' + files.length + ' staged .TXT files');
|
||||
|
||||
// Parse all files, collect SNs
|
||||
const recs = [];
|
||||
let noSn = 0, noAcc = 0;
|
||||
for (const f of files) {
|
||||
let t; try { t = fs.readFileSync(f, 'utf8'); } catch { continue; }
|
||||
const p = parseTxt(t);
|
||||
if (!p.sn) { noSn++; continue; }
|
||||
if (p.errs.length < 5) noAcc++;
|
||||
p.key = decodeSn(p.sn); // DB lookup key (decoded)
|
||||
recs.push({ file: f, ...p });
|
||||
}
|
||||
|
||||
// Bulk-load DB rows for these SNs (decoded keys)
|
||||
const sns = [...new Set(recs.map(r => r.key))];
|
||||
const dbMap = new Map();
|
||||
for (let i = 0; i < sns.length; i += 1000) {
|
||||
const chunk = sns.slice(i, i + 1000);
|
||||
const rows = await db.query(
|
||||
'SELECT serial_number, model_number, test_date, raw_data FROM test_records WHERE serial_number = ANY($1)', [chunk]);
|
||||
for (const r of rows) dbMap.set(r.serial_number, r);
|
||||
}
|
||||
|
||||
const out = { missing: [], collision: [], model: [], dbOlder: [], err: [], errRowCount: [], retest: 0, retestSameDay: 0, vasFmt: 0, ok: 0 };
|
||||
for (const r of recs) {
|
||||
const d = dbMap.get(r.key);
|
||||
if (!d) { out.missing.push(r.sn + (r.key !== r.sn ? ' (dec ' + r.key + ')' : '')); continue; }
|
||||
const dbDate = d.test_date && d.test_date.toISOString ? d.test_date.toISOString().slice(0,10) : String(d.test_date);
|
||||
let txtDate = null;
|
||||
if (r.date) { const [mm,dd,yy] = r.date.split('-'); txtDate = `${yy}-${mm}-${dd}`; }
|
||||
|
||||
// Collision: same SN but a genuinely different product family in DB (generic serials like 1-1 reused)
|
||||
if (r.model && d.model_number && normModel(r.model) !== normModel(d.model_number)) {
|
||||
const famTxt = normModel(r.model).replace(/[-0-9].*$/, '');
|
||||
const famDb = normModel(d.model_number).replace(/[-0-9].*$/, '');
|
||||
if (famTxt !== famDb) { out.collision.push(`${r.sn}: txt=${r.model} db=${d.model_number}`); continue; }
|
||||
out.model.push(`${r.sn}: txt=${r.model} db=${d.model_number}`); continue; // same family, diff variant
|
||||
}
|
||||
|
||||
// Retest: DB date newer than the staged file -> ON-CONFLICT updated DB to a later test. Expected.
|
||||
if (txtDate && dbDate > txtDate) { out.retest++; continue; }
|
||||
if (txtDate && dbDate < txtDate) { out.dbOlder.push(`${r.sn}: txt=${r.date} db=${dbDate}`); continue; }
|
||||
|
||||
// Same test run -> error% must match
|
||||
const acc = parseRawAcc(d.raw_data);
|
||||
const de = acc.errs;
|
||||
if (r.errs.length === 5 && de.length === 5) {
|
||||
const maxd = Math.max(...r.errs.map((e,i) => Math.abs(e - de[i])));
|
||||
if (maxd > ERR_TOL) {
|
||||
// Same SN+model+date but error% differs. If the STIM SETPOINTS match, it's the
|
||||
// same unit/test points -> a same-day retest (DB kept a different run). If stim
|
||||
// does NOT match, the wrong record's data is in raw_data -> genuine parse fault.
|
||||
if (stimsMatch(r.stims, acc.stims)) { out.retestSameDay++; continue; }
|
||||
out.err.push(`${r.sn} (${d.model_number}): STIM txt=[${r.stims.join(',')}] raw=[${acc.stims.map(x=>x.toFixed(4)).join(',')}] | err txt=[${r.errs.join(',')}] db=[${de.map(x=>x.toFixed(4)).join(',')}]`); continue;
|
||||
}
|
||||
} else if (r.errs.length === 5 && de.length === 0) {
|
||||
out.vasFmt++; continue; // VAS/single-point format, no 5-row accuracy block in raw_data
|
||||
} else if (r.errs.length === 5 && de.length !== 5) {
|
||||
out.errRowCount.push(`${r.sn} (${d.model_number}): txt 5 rows, raw_data ${de.length}`); continue;
|
||||
}
|
||||
out.ok++;
|
||||
}
|
||||
|
||||
const lines = [];
|
||||
const L = s => { lines.push(s); console.log(s); };
|
||||
L('========== PARSING FIDELITY REPORT ==========');
|
||||
L('Staged .TXT files scanned : ' + files.length);
|
||||
L(' - no SN line (non-standard fmt): ' + noSn);
|
||||
L(' - SN found / compared : ' + recs.length);
|
||||
L(' - .TXT w/o 5 accuracy rows : ' + noAcc);
|
||||
L('Unique SNs looked up in DB : ' + sns.length);
|
||||
L('SNs present in DB : ' + (sns.length - new Set(out.missing).size));
|
||||
L('');
|
||||
L('EXPLAINED (not parsing faults):');
|
||||
L(' Consistent (SN+model+date+5 error% match) : ' + out.ok);
|
||||
L(' Retest, DB newer date than .TXT : ' + out.retest);
|
||||
L(' Retest same-day (stim matches, run differs): ' + out.retestSameDay);
|
||||
L(' VAS/single-point fmt (no 5-row block) : ' + out.vasFmt);
|
||||
L(' Serial collision (generic SN, diff family): ' + out.collision.length);
|
||||
L('');
|
||||
L('NEEDS REVIEW (potential genuine issues):');
|
||||
L(' Missing from DB (after hex-decode) : ' + out.missing.length);
|
||||
L(' Model variant mismatch (same family) : ' + out.model.length);
|
||||
L(' DB OLDER than .TXT (stale DB?) : ' + out.dbOlder.length);
|
||||
L(' GENUINE error% fault (stim ALSO differs) : ' + out.err.length);
|
||||
L(' Accuracy-row-count diff : ' + out.errRowCount.length);
|
||||
const sample = (label, arr) => { if (arr.length) { L(''); L(label + ' (first 20):'); arr.slice(0,20).forEach(x => L(' ' + x)); } };
|
||||
sample('COLLISION (informational)', out.collision);
|
||||
sample('MODEL VARIANT MISMATCH', out.model);
|
||||
sample('DB OLDER THAN .TXT', out.dbOlder);
|
||||
sample('GENUINE FAULT (stim+error differ)', out.err);
|
||||
sample('ROW-COUNT DIFF', out.errRowCount);
|
||||
if (out.missing.length) { L(''); L('MISSING-FROM-DB (first 30): ' + out.missing.slice(0,30).join(', ')); }
|
||||
|
||||
if (REPORT) { fs.writeFileSync(REPORT, lines.join('\n') + '\n'); console.log('\n[written] ' + REPORT); }
|
||||
await db.close();
|
||||
})().catch(e => { console.error(e); process.exit(1); });
|
||||
1
projects/dataforth-dos/dsca-clean-models.json
Normal file
1
projects/dataforth-dos/dsca-clean-models.json
Normal file
@@ -0,0 +1 @@
|
||||
["DSCA30-01","DSCA30-02","DSCA30-03","DSCA30-06","DSCA30-07","DSCA30-08","DSCA30-08C","DSCA30-09","DSCA30-09C","DSCA30-1944","DSCA30-1945","DSCA30-1946","DSCA31-02","DSCA31-03","DSCA31-06","DSCA31-07","DSCA31-11","DSCA31-12","DSCA31-1273","DSCA31-12C","DSCA31-13","DSCA31-13C","DSCA31-15","DSCA31-1918","DSCA32-01","DSCA32-01C","DSCA32-01E","DSCA34-01","DSCA34-02C","DSCA34-04","DSCA34-04C","DSCA34-05","DSCA34-05C","DSCA34-1858","DSCA36-01","DSCA36-02","DSCA36-03","DSCA36-04","DSCA36-04C","DSCA36-1949","DSCA38-02","DSCA38-03","DSCA38-07","DSCA38-08C","DSCA38-09","DSCA38-09E","DSCA38-12C","DSCA38-12E","DSCA38-1468","DSCA38-1544","DSCA38-15C","DSCA38-16","DSCA38-16C","DSCA38-18C","DSCA38-19","DSCA39-01","DSCA39-02","DSCA39-07","DSCA40-03","DSCA40-05","DSCA40-05C","DSCA40-06","DSCA40-1951","DSCA40-1952","DSCA41-01","DSCA41-02","DSCA41-03","DSCA41-05C","DSCA41-06","DSCA41-09","DSCA41-13","DSCA41-14","DSCA41-15","DSCA41-15E","DSCA42-01","DSCA42-01C","DSCA42-02","DSCA43-10","DSCA43-20E","DSCA47E-08C","DSCA47J-01C","DSCA47J-03","DSCA47K-05","DSCA47K-13","DSCA47K-14","DSCA47N-15","DSCA47T-06","DSCA47T-1928","DSCA49-04","DSCA49-05","DSCA49-1601","DSCA49-1895"]
|
||||
3650
projects/dataforth-dos/dsca33-45-templates.json
Normal file
3650
projects/dataforth-dos/dsca33-45-templates.json
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,111 @@
|
||||
# 2026-06-17 — Dataforth datasheet RTD bug diagnosis + AD2 harness onboarding
|
||||
|
||||
## User
|
||||
- **User:** Mike Swanson (mike)
|
||||
- **Machine:** AD2
|
||||
- **Role:** admin
|
||||
|
||||
## Session Summary
|
||||
|
||||
Diagnosed a Dataforth test-datasheet defect reported 2026-06-17 by John Lehman / Peter Iliya, triggered by a Wellbore Integrity (Joseph Swinehart) cal-cert audit on 8B35 4-wire RTD certificates. The complaint: column headers wrong and some Final Test lines missing. Worked on AD2 (the testdatadb generator + PostgreSQL host), tracing the pipeline upstream from the website copy to the original ground-truth file. Diagnosis only — no changes to the generator or DB.
|
||||
|
||||
Confirmed two independent defects, both in the datasheet renderer `templates/datasheet-exact.js` (ingestion, DB data, and spec files are all correct). Defect A: RTD modules render the input column as resistance (`Rin (ohms)`) when the original/ground-truth file shows temperature (`Temp. (C)`); the stimulus values in `raw_data` are already temperatures (°C), so the numbers are right but the label is wrong and positive values lose their leading `+`. Defect B: the entire DSCA Final-Test parameter list is hardcoded as a single layout that does not match real DSCA module subtypes, producing wrong parameter names, garbage specs (`< 0 mA`, `+/- 0 %`), values mapped to the wrong rows, a mislabeled output column, and dropped lines. The "missing Final Test lines" complaint is a symptom of Defect B, not a separate bug.
|
||||
|
||||
Located the ground truth: the DOS test station writes a fully-rendered `.TXT` datasheet to `C:\STAGE\` before ingestion; it is mirrored to AD2 at `C:\Shares\test\STAGE\<TS>\<encoded-SN>.TXT`. Verified three examples against their originals — 8B35-04 (SN 179553-13, `H9553-13.TXT`), DSCA38-05 (SN 180224-7, `I0224-7.TXT`, a bridge module — NOT RTD), and DSCA34-05C (SN 180007-8, `I0007-8.TXT`, the actual DSCA RTD analog). Clarified for the thread that DSCA38 is a bridge/strain-gauge module, so its issue is Defect B; the DSCA RTD module Peter likely means is DSCA34, which has both A and B. Wrote the full diagnosis to `projects/dataforth-dos/DATASHEET-RTD-BUG-DIAGNOSIS-2026-06-17.md` (process doc + diagnosis + proposed fix).
|
||||
|
||||
The second half of the session was unplanned infrastructure work needed to `/sync` the results: AD2 had no Python (so `sync.sh` could not parse `identity.json`), no `identity.json` at all, and an unset git author identity. Installed Python 3.12.8, created a proper `identity.json` via the documented hand-create + `migrate-identity.sh` flow, and set git authorship. Then synced — which surfaced that `ad2` was 3 months behind main and that `sync.sh` cannot overwrite itself mid-run on Windows. Recovered cleanly, rebased `ad2` onto main (modernizing the fork to harness v1.4.3), relocated the Dataforth context out of the shared `.claude/CLAUDE.md` into `clients/dataforth/CLAUDE.dataforth.md` so future syncs stay conflict-free, and force-pushed.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Diagnose only, verify against the original file, not assumptions.** Pulled the staged DOS-generated `.TXT` as ground truth rather than trusting the code reading. This confirmed the RTD stimulus values are temperatures and the label is the defect.
|
||||
- **Classified the bug as renderer-only.** Ingestion (`multiline.js`), DB `raw_data`, and spec `SENTYPE` are all correct; the fix belongs in `datasheet-exact.js`. Defect A is surgical (fold RTD/sensorNum 7 into the temperature path); Defect B (DSCA) needs a per-subtype rebuild driven by `DSCFIN.DAT` and is larger.
|
||||
- **Installed Python from the official installer, not Chocolatey.** Choco 2.6.0 requires .NET 4.8 (Server 2019 ships 4.7.2); the python.org silent installer is self-contained and avoids a .NET install + reboot.
|
||||
- **Created identity.json via hand-create + migrate-identity.sh** (the documented flow) rather than inventing a generator. Used Mike's identity (strongly signaled by the gitea remote, branch, and user profile memory).
|
||||
- **Did not blind-commit the 205 MB WizTree zip until authorized.** `sync.sh` does `git add -A`; flagged the binary first. User confirmed AD2 is a Dataforth-ops fork and to sync everything.
|
||||
- **Modernized the ad2 fork by rebasing onto main and relocating the Dataforth doc.** Chose "take main's lean CLAUDE.md + move Dataforth context to a fork-specific file" so the fork stays additive and future rebases are conflict-free. The redundant CLAUDE.md-edit commit became empty and was auto-dropped.
|
||||
|
||||
## Problems Encountered
|
||||
|
||||
- **`sync.sh` aborted: "No Python interpreter found."** AD2 had no Python at all. Installed Python 3.12.8 (official amd64 installer, all-users, PATH + `py` launcher); `sync.sh` now detects `py`.
|
||||
- **`identity.json` missing** → commits would attribute as `unknown` and vault/coord fields were absent. Hand-created the core file and ran `migrate-identity.sh` to fill python/ollama/platform/arch/grok/coord_api. Validated all 13 required identity fields present; confirmed it is gitignored.
|
||||
- **Rebase failed mid-run: "unable to create file .claude/scripts/sync.sh: Permission denied."** `sync.sh` was the executing script while git's checkout-to-main tried to overwrite it (Windows file lock). The partial checkout left the working tree half-converted to origin/main (modified CLAUDE.md, deleted sync.sh, 240 untracked origin/main files). Recovered with `git reset --hard HEAD` + `git clean -fd`, then ran `git rebase origin/main` directly (outside sync.sh) — succeeded with only the expected `.claude/CLAUDE.md` conflict.
|
||||
- **CLAUDE.md rebase conflict** (Dataforth fork doc vs main's lean refactor). Resolved by taking main's version (`git checkout origin/main -- .claude/CLAUDE.md`); Dataforth content was preserved beforehand into `clients/dataforth/CLAUDE.dataforth.md`.
|
||||
- **NAS SSH host-key changed / SMB denied.** Could not reach `\\192.168.0.9\test\STAGE` over SSH (host key changed) or SMB (permission denied). Worked around via the rsync daemon (module `test`, user `rsync`) for listing, and found the staged originals already mirrored locally at `C:\Shares\test\STAGE\`.
|
||||
|
||||
## Configuration Changes
|
||||
|
||||
Created:
|
||||
- `C:\ClaudeTools\.claude\identity.json` (gitignored, per-machine) — full identity for AD2
|
||||
- `C:\ClaudeTools\projects\dataforth-dos\DATASHEET-RTD-BUG-DIAGNOSIS-2026-06-17.md` — diagnosis deliverable
|
||||
- `C:\ClaudeTools\clients\dataforth\CLAUDE.dataforth.md` — relocated Dataforth ops context
|
||||
- `C:\ClaudeTools\.claude\memory\project_ad2_dataforth_fork.md` (+ MEMORY.md index line)
|
||||
- Session log (this file)
|
||||
|
||||
Modified:
|
||||
- `.claude/CLAUDE.md` — now the lean fleet doc (was the Dataforth fork doc); rebased from main
|
||||
- Git author identity on AD2: `user.name="Mike Swanson"`, `user.email="mike@azcomputerguru.com"`
|
||||
- Whole harness advanced to main (v1.4.3): new `CLAUDE_EXTENDED.md`, `harness/`, `bootstrap/`, skills/commands
|
||||
|
||||
Installed:
|
||||
- Python 3.12.8 (64-bit, all-users) → `C:\Program Files\Python312\`, `py` launcher at `C:\Windows\py`
|
||||
|
||||
NOT changed (by design — diagnosis only): `templates/datasheet-exact.js`, the PostgreSQL DB, any datasheet content.
|
||||
|
||||
## Credentials & Secrets
|
||||
|
||||
No new credentials created or discovered this session. Used existing documented access:
|
||||
- NAS rsync daemon: `rsync://rsync@192.168.0.9/test` (module `test` = `/data/test`), password `IQ203s32119` (already in the Dataforth context doc).
|
||||
- PostgreSQL (local, AD2): default app creds from `database/db.js` — `testdatadb_app` / `DfTestDB2026!` on `localhost:5432/testdatadb`.
|
||||
|
||||
(AD2 has no SOPS vault cloned; these remain documented in `clients/dataforth/CLAUDE.dataforth.md`.)
|
||||
|
||||
## Infrastructure & Servers
|
||||
|
||||
- **AD2** 192.168.0.6 — testdatadb (Node/Express :3000) + PostgreSQL 18; this host. Hostname `AD2`.
|
||||
- **AD1** 192.168.0.27 — `\\AD1\Engineering`.
|
||||
- **D2TESTNAS** 192.168.0.9 — SMB1 bridge; rsync daemon port 873 module `test`; SSH host key changed this session (publickey/password denied).
|
||||
- testdatadb data: 464,671 records on website; log_types — 5BLOG 196,502 / 7BLOG 121,304 / DSCLOG 79,868 / 8BLOG 63,808 / others.
|
||||
- Impact of Defect A (RTD label): 8B35 5,476 + DSCA34 3,573 + SCM5B34/35 14,887 ≈ 24K certs. Defect B (DSCA template): up to 78,343 DSCLOG certs.
|
||||
|
||||
## Commands & Outputs
|
||||
|
||||
```bash
|
||||
# Render current generator output vs staged original (read-only diagnosis)
|
||||
cd C:\Shares\testdatadb
|
||||
node -e "const db=require('./database/db');const {renderContent}=require('./database/render-datasheet');(async()=>{const r=await db.queryOne('SELECT * FROM test_records WHERE serial_number=$1',['179553-13']);if(r.test_date&&r.test_date.toISOString)r.test_date=r.test_date.toISOString().slice(0,10);console.log(renderContent(r));await db.close();})()"
|
||||
type C:\Shares\test\STAGE\TS-4L\H9553-13.TXT # ground truth: header = " Temp. (C)"
|
||||
|
||||
# Python install (PowerShell)
|
||||
Start-Process python-3.12.8-amd64.exe -ArgumentList '/quiet','InstallAllUsers=1','PrependPath=1','Include_launcher=1' -Wait # exit 0
|
||||
|
||||
# Onboarding identity
|
||||
# hand-create .claude/identity.json (core fields) then:
|
||||
bash .claude/scripts/migrate-identity.sh # filled python/ollama/platform/arch/grok/coord_api
|
||||
|
||||
# Rebase recovery (sync.sh self-lock)
|
||||
git reset --hard HEAD && git clean -fd
|
||||
git rebase origin/main
|
||||
git checkout origin/main -- .claude/CLAUDE.md && git add .claude/CLAUDE.md
|
||||
GIT_EDITOR=true git rebase --continue
|
||||
git push --force-with-lease origin ad2
|
||||
```
|
||||
|
||||
Key error + resolution: `error: unable to create file .claude/scripts/sync.sh: Permission denied` → run the rebase directly (not via the executing sync.sh).
|
||||
|
||||
## Pending / Incomplete Tasks
|
||||
|
||||
- **Fix Defect A (RTD label/values)** in `templates/datasheet-exact.js`: in the input-header logic and `formatAccuracyLine`, route `sensorNum === 7` (RTD) through the temperature path (`' Temp. (C)'` + `formatSigned(stim, 2, 8)`). Verify leading-space alignment against a thermocouple original. Safe — `7` is reached only by RTD sentypes; no module currently needs `Rin (ohms)`. Awaiting review before changing the generator.
|
||||
- **Fix Defect B (DSCA template)**: rebuild the DSCA Final-Test parameter list + ACCURACY column titles/units per module subtype, driven by `specdata\DSCFIN.DAT` / the legacy QB DSC writer. Larger effort. Until done, treat all DSCA (DSCLOG) website datasheets as unreliable.
|
||||
- After fixes: re-push affected models by clearing `api_uploaded_at` (RE-PUSH is idempotent).
|
||||
- **AD2 tooling gaps**: `jq`, `sops`, `age` not installed; no vault cloned (`D:/vault` absent) → vault sync N/A. coord_api (172.16.3.30) unreachable from Dataforth LAN.
|
||||
- **`sync.sh` self-lock**: upstream fix candidate (re-exec from a temp copy, or rebase before the script can be overwritten) — offered to file as a CT thought.
|
||||
|
||||
## Reference Information
|
||||
|
||||
- Diagnosis doc: `projects/dataforth-dos/DATASHEET-RTD-BUG-DIAGNOSIS-2026-06-17.md`
|
||||
- Generator: `C:\Shares\testdatadb\templates\datasheet-exact.js` (repo: `projects/dataforth-dos/datasheet-pipeline/implementation/templates/datasheet-exact.js`)
|
||||
- Ground-truth originals: `C:\Shares\test\STAGE\<TS>\<encoded-SN>.TXT` (8.3 hex-prefix SN encoding: first 2 digits → letter, 55+n)
|
||||
- Examples: 8B35-04 SN 179553-13 (`H9553-13.TXT`, P1RTD4W, MAXIN 600); DSCA38-05 SN 180224-7 (`I0224-7.TXT`, FBRIDGE); DSCA34-05C SN 180007-8 (`I0007-8.TXT`, P1RTD3W)
|
||||
- Task prompt: `Prompt617.txt`
|
||||
- Commits this session (ad2): `49c9eb50` (work sync), `bbb19db2` (relocate Dataforth doc), `c4de16f6` (memory)
|
||||
- Contacts: John Lehman jlehman@dataforth.com, Peter Iliya pIliya@dataforth.com
|
||||
@@ -0,0 +1,42 @@
|
||||
# 8B/5B/SCM Render Fix — Handoff to the AD2 Session (2026-06-18)
|
||||
|
||||
## User
|
||||
- **User:** Mike Swanson (mike)
|
||||
- **Machine:** GURU-5070
|
||||
- **Role:** admin
|
||||
|
||||
## Note for mike (AD2 session — datasheet-exact.js owner)
|
||||
|
||||
I drove the 8B/5B/SCM unpublished-render investigation from GURU-5070. Root cause is the
|
||||
**same class as DSCA**, and finishing it needs **your existing DSCA machinery** — handing it over
|
||||
rather than reimplementing in the file you're actively editing.
|
||||
|
||||
**What's done + committed (you'll have it on pull):**
|
||||
- `projects/dataforth-dos/8b5bscm-templates.json` — **136 models** mined from Hoffman originals
|
||||
(accOut, accHeader, rows[name,spec], _srcSerial). Also on the box at
|
||||
`C:\Shares\testdatadb\8b5bscm-templates.json`.
|
||||
- Validated **general parseRawData fix**: a PASS/FAIL line is wrongly consumed as the step line
|
||||
for non-DSCA families that omit the `"0","0",v` line (8B45/8B49/5B39/SCM5B33...). Change the
|
||||
guard to `if (!((family === 'DSCA' || skipStepIfStatus) && looksLikeStatus))` and pass
|
||||
`skipStepIfStatus` only for templated models (non-templated byte-unchanged).
|
||||
- Proven the **Final-Test template branch generalizes** to 8B/5B/SCM (the 8 minimal patches are in
|
||||
`8B5BSCM-RENDER-VERIFY-2026-06-18.md`). Stage+verify vs Hoffman (content-normalized): **8B avg
|
||||
0.97, SCM5B 0.93**; 15 models content-perfect.
|
||||
|
||||
**What's left = your DSCA tools, applied to 8b5bscm-templates.json:**
|
||||
1. **slotmaps** (`_derive_slotmaps.js`) — 70 models + most unpublished units render null on a
|
||||
measurement-count mismatch; they need per-model slotMaps. THIS is the unlock for the ~5,148 units.
|
||||
2. **Math.fround QB rounding** — closes ~17 precision-distance models.
|
||||
3. **frequency/AAC accuracy renderer** — 8B45/8B49/5B45 (frequency) + SCM5B33 (AAC) need the
|
||||
accuracy input/output labels + stimulus formatting the renderer can't do yet. **This is the SAME
|
||||
unsolved code blocking your DSCA45/DSCA33** — solve once, covers both.
|
||||
|
||||
Recommend wiring 8b5bscm-templates into your template/slotmap/rounding path and finishing the
|
||||
freq/AAC accuracy block once for DSCA + 8B/5B/SCM together. Full detail + the verify harness:
|
||||
`projects/dataforth-dos/8B5BSCM-RENDER-VERIFY-2026-06-18.md`. I published the one already-renderable
|
||||
unit (8B32-01 165669-15); everything else awaits the slotmaps above.
|
||||
|
||||
## Status
|
||||
- 8B/5B/SCM unpublished render: diagnosed + 136 templates mined/committed; remaining work handed to
|
||||
AD2 (slotmaps / QB rounding / freq-AAC accuracy). Live UI redesign deployed; /api/search sort + UI
|
||||
presets/publish wiring done earlier today.
|
||||
@@ -0,0 +1,295 @@
|
||||
# DSCA Datasheet Fix 2 — STAGE 2 wire-in, STAGE 3 validator, publish of 68 clean models
|
||||
|
||||
## Update: 08:00 PT — diagnosed DSCA33/DSCA45 missing-specs gap (left blocked, documented)
|
||||
|
||||
Dug into why DSCA33-*/DSCA45-* render null. Root cause is a DATA GAP, not a code bug: their MAIN
|
||||
spec records are missing from every recovered `specdata/*.DAT`. DSCA33 appears in NO DAT file at all;
|
||||
DSCA45 appears only in `DSCFIN.DAT` (Final-Test layout, lacks SENTYPE/MAXIN/input-type). Loaded DSCA
|
||||
prefixes jump 32->34 and 43->47. Almost certainly lost in the cryptolocker wipe. Blocks ~8,763 PASS
|
||||
certs (DSCA33 3,350/35 models, DSCA45 5,413/23 models).
|
||||
|
||||
The Final-Test data is NOT the blocker — a DSCA45 cert's Final-Test block renders correctly via its
|
||||
slotMap with a stub spec (2 trivial diffs). Blockers are: (1) `render-datasheet.js` bails on missing
|
||||
specs before rendering; (2) special accuracy headers the sensor logic can't make — DSCA45
|
||||
`Frequency (Hz)`/`Output (V)`, DSCA33 `Vin (mVAC)`/`Output (VDC)`; (3) non-status rows (DSCA45
|
||||
`Zero-Crossing Input`/`TTL Input`) that have no PASS in golden but the renderer appends one.
|
||||
|
||||
Decision (Mike): leave blocked, documented. Recorded as memory `project_dsca33_45_spec_gap`. The
|
||||
clean fix is obtaining the authoritative DSCA33/DSCA45 main spec files from Dataforth (added to the
|
||||
#32441 handoff below); the self-contained alternative is template-driving the accuracy block. Final-
|
||||
Test slotMaps for the matchable models are already derived and ready for when specs arrive.
|
||||
|
||||
## Update: 07:52 PT — per-model slot maps resolve ambiguous DSCA layouts + re-publish of 92
|
||||
|
||||
Took on the ambiguous-layout families (models the count-guard skipped because raw_data carries more
|
||||
or fewer value-bearing STATUS entries than the template's spec-bearing rows — the test program
|
||||
measures slots the printed sheet omits, e.g. DSCA49's 5mA load pair).
|
||||
|
||||
New tool `derive-dsca-slotmaps.js`: derives a per-model `slotMap` (absolute statusEntries index per
|
||||
spec-bearing row) by greedily matching a staged original's printed values against the DB raw_data
|
||||
STATUS entries (same fround formatting), then picking the candidate map that validates against the
|
||||
most units. Models are grouped by identical row-name signature and one map is derived per group from
|
||||
ALL sibling units — this disambiguates duplicate values (DSCA49-04 alone has only 2 staged units,
|
||||
both with 5mA==50mA linearity, so greedy picked the wrong slot; its siblings' 25 units force the
|
||||
correct [0,1,2,3,6,7,8,9,11,13,15] map). Stored as `slotMap` in dsca-templates.json. Renderer
|
||||
consults slotMap ONLY when the sequential zip fails, so the 88 clean models keep their path (no
|
||||
regression).
|
||||
|
||||
STAGE 3 re-validation: FINAL-TEST CLEAN **88 -> 92**; 134 more certs render (null 450 -> 316);
|
||||
matches 2278 -> 2412; same 6 retest-vintage dirty, no new mismatches. Re-pushed all 92 clean models
|
||||
(41,362 serials): **updated=4024 unchanged=36109 created=0 errors=0 skipped=1229** (skips down from
|
||||
2165 — ~936 previously-guarded certs now render via slotMap).
|
||||
|
||||
DSCA49 family + DSCA40-03 group now clean and live. Still blocked, SEPARATE gap (not layout
|
||||
ambiguity): DSCA45-* and most DSCA33-* render null because they have NO spec-reader entries
|
||||
(`render-datasheet.js` bails before rendering) — their slotMaps are derived and ready, they just
|
||||
need spec coverage. One DSCA33 group (DSCA33-02/03/03A/04/05/1948) didn't reach the slotMap
|
||||
validation threshold (best 19/35 units). Commit `e9262f57`.
|
||||
|
||||
Net live DSCA Final-Test: 92/126 templated models content-clean. Remaining: 6 retest-vintage,
|
||||
DSCA45-*/DSCA33-* missing specs, 1 DSCA33 group below threshold, ~32 models with no DB-matched
|
||||
staged unit, and 231 untemplated models (STAGE 1 extension).
|
||||
|
||||
## Update: 07:33 PT — data-driven DSCA load note (DSCA39 footer-artifact fix) + re-publish of 88
|
||||
|
||||
Fixed the DSCA39 footer-note artifact properly at the STAGE 1 extractor level. The "Standard
|
||||
output load for test is 250 ohms." line is a footer note, not a parameter; the extractor had
|
||||
captured it as a column-truncated row ("Standard output load for te"), and the renderer's
|
||||
`OUTSIGTYPE==='CURRENT'` emission was wrong both ways (printed a spurious note after the underline
|
||||
for many `-C` current models that never had it, and never placed it for the models that do).
|
||||
|
||||
Data-driven fix: `derive-dsca-templates.js` now captures the note as a per-model `loadNote`
|
||||
property and excludes it from rows; `datasheet-exact.js` emits `loadNote` (blank + note) before the
|
||||
footer underline only for models that have it, and the OUTSIGTYPE emission was removed. Regenerated
|
||||
`dsca-templates.json` — surgically clean (only the 5 DSCA39 models changed; 121 others byte-identical).
|
||||
|
||||
STAGE 3 re-validation: FINAL-TEST CLEAN **85 -> 88**, mismatches 9 -> 6, matches 2206 -> 2278.
|
||||
DSCA39-01/02/07 now fully clean (DSCA39-01 byte-content-verified). No regression — `-C` models
|
||||
stayed clean and no longer carry the spurious note. Re-pushed all 88 clean models (38,274 serials):
|
||||
**updated=7092 unchanged=29017 created=0 errors=0 skipped=2165**.
|
||||
|
||||
The 6 still-dirty models (DSCA38-05/-1793/-19C/-19E, DSCA39-05, DSCA39-1950) are ALL retest
|
||||
data-vintage (staged .TXT is an older run than the DB record; Supply Current / Linearity differ by
|
||||
more than rounding) — not render bugs, cannot reconcile against an older sheet. Commit `61f54dc4`.
|
||||
|
||||
Net DSCA Final-Test render state: 88/126 templated models content-clean and live; 6 vintage-only;
|
||||
~32 ambiguous-layout families still skipped (need per-subtype slot mapping); 231 untemplated models
|
||||
still need a STAGE 1 extension.
|
||||
|
||||
## Update: 07:23 PT — rounding-mode fix (QB single-precision) + re-publish of 85 clean models
|
||||
|
||||
Fixed the rounding-mode issue behind the 26 last-digit-diff models. Root cause: the DOS
|
||||
QuickBASIC computed/stored values as single-precision floats, so half-boundary rounding follows
|
||||
single, not double, precision. `formatMeasuredExact` now applies `Math.fround(value)` before
|
||||
`toFixed`. Verified each boundary case against the staged golden (9.9995 -> "10.000", 46.85 ->
|
||||
"46.8", .45 -> "0.4", 3.3325 -> "3.332").
|
||||
|
||||
STAGE 3 re-validation: FINAL-TEST CLEAN models **68 -> 85** (+17), mismatches 26 -> 9, cert matches
|
||||
2123 -> 2206. Zero regression — every remaining dirty model was already dirty; no clean model flipped.
|
||||
|
||||
Re-pushed all 85 clean models (37,168 PASS serials): **updated=6054 unchanged=28949 created=0
|
||||
errors=0 skipped=2165**. Live site is now fround-correct across 85 DSCA models (the 6,054 updates =
|
||||
boundary corrections in the original 68 + the 17 newly-clean models).
|
||||
|
||||
The 9 still-dirty models are NOT rounding: 4 (DSCA38-05/-1793/-19C/-19E) are Supply Current retest
|
||||
data-vintage (staged .TXT predates the DB record — not a render bug); 5 (DSCA39-01/02/05/07/1950)
|
||||
are the STAGE 1 footer-note artifact ("Standard output load..." mis-captured as a truncated row).
|
||||
A renderer-side fix for the footer note was attempted but regressed 24 clean current-output `-C`
|
||||
models (the note placement differs per golden), so it was reverted — this needs a targeted STAGE 1
|
||||
extractor fix, not a renderer change. Commit `14ee61dc`.
|
||||
|
||||
## User
|
||||
- **User:** Mike Swanson (mike)
|
||||
- **Machine:** AD2
|
||||
- **Role:** admin
|
||||
|
||||
## Session Summary
|
||||
|
||||
Picked up the AD2-local handoff `projects/dataforth-dos/DATASHEET-FIX2-5-HANDOFF-2026-06-18.md`
|
||||
(ref Syncro #32441) and executed Fix 2 (DSCA Final-Test rebuild) STAGE 2 and STAGE 3, then
|
||||
published the validated subset to the live Hoffman site. Work was on the DEPLOYED pipeline at
|
||||
`C:\Shares\testdatadb` (Node + PostgreSQL 18); repo copies under
|
||||
`projects/dataforth-dos/datasheet-pipeline/implementation/` are stale and were reconciled after.
|
||||
|
||||
STAGE 2 wired the STAGE-1 output `dsca-templates.json` (126 per-model layouts) into the deployed
|
||||
`templates/datasheet-exact.js`. For `family === 'DSCA'` the Final-Test block now renders parameter
|
||||
names + specs from the staged template rows (not the single hardcoded `DATA_LINES['DSCA']` +
|
||||
`buildTSpecs` DSCA branch); value-bearing `raw_data` STATUS groups map positionally onto the
|
||||
spec-bearing rows; empty-spec rows (240VAC Withstand / Hi-Pot) render blank+PASS (the duplicate
|
||||
hardcoded footer for DSCA was removed). The ACCURACY block now uses the template `accOut`
|
||||
(`Output (V)`/`Output (mA)`) with `-` rule separators instead of `Vout (V)` + `=`. Two real
|
||||
defects were found and fixed in the process (both are the handoff's "lines drop / wrong values"
|
||||
defect): negative signs were dropped because value parsing started at index 5 instead of 4, and
|
||||
`parseRawData` always consumed the line after the 5 accuracy points as a step-response placeholder
|
||||
— but many DSCA models omit that bare `0` line, so the first STATUS group (and its rows) was being
|
||||
discarded. Both fixes were validated against the DSCA38-05 golden staged original: the rebuilt
|
||||
Final-Test block is byte-for-byte identical (the only residual diffs are deferred cosmetic ACCURACY
|
||||
column spacing).
|
||||
|
||||
STAGE 3 built a read-only validator (`tools/validate-dsca-stage3.js`) that, for every staged DSCA
|
||||
original we have ground truth for (2,806 across 126 models), looks up the DB record, renders it
|
||||
through the live path, and content-compares. The gate is the FINAL TEST RESULTS section with rule
|
||||
lines canonicalized and whitespace collapsed (so the deferred column-spacing cosmetic does not
|
||||
register); accuracy-section diffs are reported separately. Verdict: 68 models FINAL-TEST
|
||||
CONTENT-CLEAN (2,123/2,316 compared certs match exactly, 91.7%), 26 models with measured-value
|
||||
last-digit diffs only, and ~32 models rendering null via the count-guard. A safety guard was added:
|
||||
when a model's value count != its spec-row count the positional zip is ambiguous (the subtype
|
||||
measures load points the template omits, e.g. DSCA49's 5mA pair), so the cert is skipped/flagged
|
||||
rather than emitting misaligned data.
|
||||
|
||||
Per Mike's direction, published the 68 STAGE-3-clean models. Restarted the `testdatadb` service so
|
||||
the new template is live, canaried a single cert (DSCA30-01 / 148059-3 -> updated, 0 errors), then
|
||||
re-pushed all 30,423 PASS certs for the 68 clean models via `uploadBySerialNumbers` from a fresh
|
||||
node process. Result: updated=26,022, unchanged=2,738, created=0, errors=0, skipped=1,663. The
|
||||
26,022 updates replaced the old defective DSCA renders on the live site with the rebuilt, validated
|
||||
ones; the 1,663 skips are the count-guard correctly refusing individual ambiguous certs.
|
||||
|
||||
All work committed to the `ad2` branch (4 commits) and pushed. No vault access on AD2 (no
|
||||
sops/age, vault repo absent), so the Syncro #32441 ticket update could not be posted from here —
|
||||
left as a handoff to GURU-5070 (see Pending / Handoff).
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Skip, don't guess, on ambiguous layouts.** When a DSCA model's value-bearing STATUS count !=
|
||||
its spec-bearing template-row count, the simple positional zip would misplace values. Chose to
|
||||
return null (skip + flag for STAGE 3 per-subtype mapping) rather than publish misaligned data,
|
||||
per the handoff's "do not guess" discipline. This is what produces the 1,663 push skips.
|
||||
- **Scoped the parser/format fixes to DSCA only.** The step-response detection fix and the
|
||||
index-4 value-start fix live in `formatMeasuredExact` (new) and a DSCA-gated branch in
|
||||
`parseRawData`, leaving the already-validated 5B/8B/7B/DSCT/SCMVAS paths byte-unchanged.
|
||||
- **DSCA decimal code N -> toFixed(N) exactly.** Unlike 5B/8B (where format code 2 means 1
|
||||
decimal), DSCA uses the trailing status digit as a literal decimal-place count; verified against
|
||||
the golden (e.g. Output Reg code 2 -> "0.00").
|
||||
- **Gate STAGE 3 on the Final-Test section, not the whole sheet.** ACCURACY-block column spacing
|
||||
is the handoff's explicitly-deferred cosmetic gap; canonicalizing rule lines + collapsing
|
||||
whitespace isolates real content diffs (names/values/specs/statuses) from spacing noise.
|
||||
- **Published only the 68 clean models.** Highest-confidence set; idempotent push replaces
|
||||
defective live certs. Left the 26 last-digit-diff models, ~32 ambiguous layouts, and 231
|
||||
untemplated models for follow-up.
|
||||
- **Did not restart-then-publish through the running service.** Re-push ran from a fresh node
|
||||
process (picks up new template via `require`); the service restart only refreshes the internal
|
||||
tool's on-demand renders.
|
||||
|
||||
## Problems Encountered
|
||||
|
||||
- **Sync rebase conflict in `errorlog.md`** — two machines appended entries concurrently. Resolved
|
||||
by keeping both entries (append-only log) and `git rebase --continue`.
|
||||
- **`sync.sh` push failed (`src refspec main does not match any`)** — known AD2 fork gotcha
|
||||
(sync.sh is not fork-aware on push). Pushed manually with `git push --force-with-lease origin ad2`
|
||||
(rebase rewrote history). See memory `project_ad2_dataforth_fork`.
|
||||
- **Dropped rows on DSCA models without a step-response line** — `parseRawData` ate the first
|
||||
STATUS group. Fixed by skipping the step-response consume for DSCA when the next line starts with
|
||||
PASS/FAIL.
|
||||
- **Negative measured values rendered without sign** — value substring started at index 5 (assumed
|
||||
a space at index 4), but negatives put `-` at index 4. Fixed to start at index 4.
|
||||
- **First STAGE 3 run showed 0 matches / 2,316 mismatches** — the validator's normalization wasn't
|
||||
canonicalizing rule lines, so the deferred ACCURACY separator dash-count cosmetic masked all
|
||||
downstream content. Reworked to canonicalize rule lines and gate on the Final-Test section.
|
||||
- **No vault access on AD2** — cannot fetch Syncro creds to post #32441. Handoff to 5070 (below).
|
||||
|
||||
## Configuration Changes
|
||||
|
||||
Deployed (live, outside repo — `C:\Shares\testdatadb`):
|
||||
- `templates/datasheet-exact.js` — DSCA wire-in (edited). Save-state:
|
||||
`templates/datasheet-exact.js.bak-2026-06-18-1334`.
|
||||
- `_validate_dsca_stage3.js`, `_push_clean68.js`, `_clean-models.json`, `_dsca-stage3-report.txt`,
|
||||
`_stage3-run.log`, `_push-clean68.log` — operational scripts/outputs (created).
|
||||
|
||||
Repo (`ad2` branch):
|
||||
- `projects/dataforth-dos/datasheet-pipeline/implementation/templates/datasheet-exact.js` — reconciled (modified).
|
||||
- `projects/dataforth-dos/datasheet-pipeline/implementation/dsca-templates.json` — added.
|
||||
- `projects/dataforth-dos/tools/validate-dsca-stage3.js` — added.
|
||||
- `projects/dataforth-dos/tools/push-clean68.js` — added.
|
||||
- `projects/dataforth-dos/DSCA-STAGE3-REPORT-2026-06-18.txt` — added.
|
||||
- `projects/dataforth-dos/dsca-clean68-models.json` — added (the 68 published models).
|
||||
- `errorlog.md` — conflict resolution during sync.
|
||||
|
||||
## Credentials & Secrets
|
||||
|
||||
No new credentials discovered or created this session. Referenced (already vaulted per the handoff
|
||||
at `clients/dataforth/testdatadb-postgres`):
|
||||
- PostgreSQL superuser `postgres` / `Paper123!@#`; app `testdatadb_app` / `DfTestDB2026!`
|
||||
(`db.js` defaults: host localhost:5432, database `testdatadb`).
|
||||
- Hoffman/CloudFilter uploader creds: `C:\ProgramData\dataforth-uploader\credentials.json`
|
||||
(fields CF_TOKEN_URL, CF_API_BASE, CF_CLIENT_ID, CF_CLIENT_SECRET, CF_SCOPE). Not vaulted; lives
|
||||
on the box only.
|
||||
|
||||
## Infrastructure & Servers
|
||||
|
||||
- **AD2** (192.168.0.6) — Dataforth domain controller; runs the deployed pipeline at
|
||||
`C:\Shares\testdatadb`. Windows service `testdatadb` (Automatic, Running) — restarted this session.
|
||||
- **PostgreSQL 18** on AD2 (localhost:5432, db `testdatadb`); table `test_records`.
|
||||
- **Staged originals**: `C:/Shares/test/STAGE/**/*.TXT` (11,956 total; 2,806 DSCA) — STAGE-1 source
|
||||
and STAGE-3 ground truth.
|
||||
- **Hoffman API** (`CF_API_BASE/api/v1/TestReportDataFiles/bulk` POST, idempotent; GET is paginated
|
||||
list only) — the live customer-facing datasheet site.
|
||||
|
||||
## Commands & Outputs
|
||||
|
||||
- Sync conflict recovery: `git add errorlog.md && GIT_EDITOR=true git rebase --continue` then
|
||||
`git push --force-with-lease origin ad2`.
|
||||
- Module load-check after each edit: `node -e 'require("./templates/datasheet-exact.js")'`.
|
||||
- STAGE 3 run: `node _validate_dsca_stage3.js` -> `_dsca-stage3-report.txt`. Summary: 68 clean,
|
||||
26 Final-Test mismatch (193 certs), ~32 null; 2,123/2,316 match.
|
||||
- Canary: `uploadBySerialNumbers(["148059-3"])` -> `updated=1 errors=0`.
|
||||
- Full push: `node _push_clean68.js` -> `created=0 updated=26022 unchanged=2738 errors=0 skipped=1663`
|
||||
over 30,423 PASS certs.
|
||||
|
||||
## Pending / Incomplete Tasks
|
||||
|
||||
Not yet published / remaining DSCA work:
|
||||
1. **26 models, last-digit measured diffs** — two causes: (a) QB single-precision half-up rounding
|
||||
vs JS double `toFixed` (e.g. raw 9.9995 code3 -> "9.999" here, "10.000" in golden) — fixable but
|
||||
float-precision-sensitive, re-run validator to confirm no regression; (b) retest data-vintage,
|
||||
where the staged `.TXT` is an older test run than the DB latest-wins record (Fix 3) — not a
|
||||
render bug.
|
||||
2. **~32 ambiguous layouts** (DSCA33-*, DSCA45-*, DSCA49-* families) — need the canonical
|
||||
per-subtype slot mapping (status entry index -> canonical DSCA slot -> template row by name);
|
||||
currently skipped (null) by the count-guard.
|
||||
3. **231 untemplated models / 23,866 certs** — need a STAGE 1 extension (more staged originals;
|
||||
only 126/357 DSCA models in the DB currently have a template — 70.1% of certs).
|
||||
4. **Fix 5** (backfill 379 cryptolocker-era units from staged `.TXT`) and Fix-3 cleanup items — not
|
||||
started this session.
|
||||
|
||||
## Note for mike (handoff to GURU-5070)
|
||||
|
||||
AD2 has no vault / Discord / Syncro access from here (no sops/age, `D:/vault` absent), so these
|
||||
outbound actions are left for the GURU-5070 session, which has them. Mike will run these on 5070.
|
||||
|
||||
**TODO 1 — post an internal/hidden note to Syncro #32441** (customer-facing thread is John Lehman),
|
||||
text ready to paste:
|
||||
|
||||
> Fix 2 (DSCA Final-Test rebuild) STAGE 2 + STAGE 3 complete and published. Wired the per-model staged
|
||||
> templates into the live renderer and fixed a series of render defects: dropped negative signs;
|
||||
> dropped Final-Test rows on models lacking the step-response line; QB single-precision rounding (via
|
||||
> Math.fround); the DSCA39 "Standard output load" footer-note artifact (data-driven loadNote); and the
|
||||
> ambiguous multi-load layouts (per-model slot maps, e.g. DSCA49's 5mA load pair). Built a STAGE 3
|
||||
> validator over 2,806 staged DSCA originals. **92 of 126 templated DSCA models are Final-Test
|
||||
> content-clean and published to Hoffman, 0 errors**, replacing the old defective DSCA renders live.
|
||||
> Remaining: 6 models with retest data-vintage diffs (staged sheet older than the DB latest-wins
|
||||
> record — not a render bug); one DSCA33 group below the slot-map validation threshold; ~58
|
||||
> DSCA33-*/DSCA45-* models awaiting a spec file (see TODO 2); and 231 untemplated models needing a
|
||||
> STAGE 1 extension. Byte-for-byte DOS cosmetic fidelity (leading rule line + ~1-space input-column
|
||||
> spacing) still deferred. Details: `ad2` branch commits + `DSCA-STAGE3-REPORT-2026-06-18.txt`.
|
||||
|
||||
**TODO 2 — request from Dataforth/engineering (John):** the authoritative MAIN spec records for the
|
||||
**DSCA33-*** and **DSCA45-*** families (DSCMAIN/DSCOUT-style entries with SENTYPE/MAXIN/input-type).
|
||||
Missing from every recovered `specdata/*.DAT` (DSCA33 nowhere; DSCA45 only in DSCFIN.DAT) — lost in
|
||||
the wipe — blocking ~8,763 cert renders (DSCA33 3,350 / DSCA45 5,413). Drop into `specdata/`, reload,
|
||||
and those models render via the slot maps already derived this session. Detail: memory
|
||||
`project_dsca33_45_spec_gap`. (Also note the deferred byte-fidelity disclosure to John, per the
|
||||
original FIX2-5 handoff, once the whole effort is done.)
|
||||
|
||||
Nothing else on AD2 is pending — all code is committed to `ad2` and the live Hoffman site is current
|
||||
(92 clean models published, 0 errors).
|
||||
|
||||
## Reference Information
|
||||
|
||||
- Handoff: `projects/dataforth-dos/DATASHEET-FIX2-5-HANDOFF-2026-06-18.md`
|
||||
- STAGE 3 report: `projects/dataforth-dos/DSCA-STAGE3-REPORT-2026-06-18.txt`
|
||||
- Published set: `projects/dataforth-dos/dsca-clean-models.json` (92 models, final)
|
||||
- Ticket: Syncro **#32441** (customer-facing thread: John Lehman)
|
||||
- `ad2` commits this session: `e7fa7cc6` (STAGE 2), `c03bdc9a` (STAGE 3 validator+report),
|
||||
`f798e8ea` (publish artifacts); `fc9fff81` (sync auto-commit, errorlog conflict resolution).
|
||||
- Deployed render path: `database/render-datasheet.js` -> `templates/datasheet-exact.js`;
|
||||
push path: `database/upload-to-api.js` (`uploadBySerialNumbers`, BATCH=100).
|
||||
@@ -12,7 +12,7 @@
|
||||
--hover:#f1f5f9; --sel:#e0e7ff; --desk:#e7ecf1;
|
||||
--mono:ui-monospace,"SFMono-Regular",Consolas,"Liberation Mono",monospace;
|
||||
--sans:Inter,system-ui,-apple-system,"Segoe UI",Roboto,sans-serif;
|
||||
--r:6px; --insp:480px;
|
||||
--r:6px; --insp:500px;
|
||||
}
|
||||
*{box-sizing:border-box}
|
||||
html,body{height:100%;margin:0}
|
||||
@@ -112,6 +112,10 @@
|
||||
.pill{display:inline-block;font-size:10.5px;font-weight:700;padding:1px 8px;border-radius:4px;font-family:var(--mono)}
|
||||
.pill.PASS{background:var(--pass-bg);color:var(--pass-ink)}
|
||||
.pill.FAIL{background:var(--fail-bg);color:var(--fail-ink)}
|
||||
.tag{display:inline-flex;align-items:center;gap:5px;font-size:11px;font-weight:600;padding:2px 8px;border-radius:20px;font-family:var(--sans)}
|
||||
.tag::before{content:"";width:7px;height:7px;border-radius:50%}
|
||||
.tag.pub{background:var(--pass-bg);color:var(--pass-ink)} .tag.pub::before{background:var(--pass-ink)}
|
||||
.tag.unpub{background:#fef3c7;color:#92400e} .tag.unpub::before{background:#d97706;background:none;box-shadow:inset 0 0 0 1.5px #d97706}
|
||||
.web{font-size:13px}
|
||||
.pager{display:flex;align-items:center;gap:10px;padding:7px 12px;border-top:1px solid var(--border);font-size:12px;color:var(--ink-2)}
|
||||
.pager button{font-size:12px;height:28px;padding:0 11px;border:1px solid var(--border-strong);border-radius:var(--r);background:#fff;cursor:pointer;color:var(--ink)}
|
||||
@@ -200,7 +204,7 @@
|
||||
<div class="selbar" id="selbar">
|
||||
<span id="selcount">0 selected</span>
|
||||
<button id="copySel">Copy serials</button>
|
||||
<button id="repushSel" disabled title="Re-publish to the website — needs an API endpoint">Re-push ▴</button>
|
||||
<button id="repushSel" title="Publish / re-publish the selected serials to the public website">Re-push ▴</button>
|
||||
<span class="sp" style="flex:1"></span>
|
||||
<button id="selclear" style="border:0;background:none">clear</button>
|
||||
</div>
|
||||
@@ -240,7 +244,7 @@
|
||||
const API='';
|
||||
const $=id=>document.getElementById(id);
|
||||
const state={q:'',serial:'',model:'',result:'',station:'',logtype:'',from:'',to:'',size:50,page:0,total:0,
|
||||
selected:null,rows:[],sort:'',dir:'desc',checks:new Set(),force:''};
|
||||
selected:null,rows:[],sort:'',dir:'desc',checks:new Set(),force:'',webStatus:''};
|
||||
let timer=null, certTimer=null;
|
||||
const esc=s=>String(s==null?'':s).replace(/[&<>"]/g,c=>({'&':'&','<':'<','>':'>','"':'"'}[c]));
|
||||
const fmtDate=d=>d?String(d).slice(0,10):'';
|
||||
@@ -265,7 +269,8 @@ $('mode').onclick=()=>{ const seq=['auto','serial','model','text']; state.force=
|
||||
function params(forExport){
|
||||
const p=new URLSearchParams();
|
||||
for(const k of ['q','serial','model','result','station','logtype','from','to']) if(state[k]) p.set(k,state[k]);
|
||||
if(state.sort){ p.set('sort',state.sort); p.set('dir',state.dir); } // needs API; harmless if ignored
|
||||
if(state.sort){ p.set('sort',state.sort); p.set('dir',state.dir); } // honored by /api/search (whitelisted)
|
||||
if(state.webStatus) p.set('web_status',state.webStatus); // on=published, off=not yet published
|
||||
if(!forExport){ p.set('limit',state.size); p.set('offset',state.page*state.size); }
|
||||
return p;
|
||||
}
|
||||
@@ -321,14 +326,14 @@ function select(id,auto){
|
||||
<dt>Result</dt><dd><span class="pill ${r.overall_result}">${esc(r.overall_result)}</span></dd>
|
||||
<dt>Log</dt><dd>${esc(r.log_type)}</dd>
|
||||
${r.work_order?`<dt>WO</dt><dd>${esc(r.work_order)}</dd>`:''}
|
||||
<dt>Web</dt><dd>${r.api_uploaded_at?'published '+fmtDate(r.api_uploaded_at):'not published'}</dd></dl>`;
|
||||
<dt>Web</dt><dd>${r.api_uploaded_at?`<span class="tag pub">Published</span> <span style="color:var(--ink-3)">${fmtDate(r.api_uploaded_at)}</span>`:'<span class="tag unpub">Not published</span>'}</dd></dl>`;
|
||||
const ds=API+'/api/datasheet/'+id;
|
||||
$('acts').style.display='flex';
|
||||
$('acts').innerHTML=`<a class="pri" href="${ds}?format=html" target="_blank">Open ↗</a>
|
||||
<button onclick="printCert()">Print</button>
|
||||
<a href="${ds}?format=txt" download="${esc(r.serial_number)}.txt">TXT</a>
|
||||
<a href="${ds}?format=html" download="${esc(r.serial_number)}.html">HTML</a>
|
||||
<button disabled title="Re-publish this cert — needs a POST /api/publish endpoint">Push to Web ▴</button>`;
|
||||
<button onclick="pushWeb('${id}',this)" title="Publish / re-publish this cert to the public website">Push to Web ▴</button>`;
|
||||
$('insp').classList.add('open');
|
||||
// lazy-load the certificate so fast arrow/typing stays snappy
|
||||
$('viewer').innerHTML='<div class="state"><div class="skel" style="width:60%"></div></div>';
|
||||
@@ -338,18 +343,23 @@ function select(id,auto){
|
||||
}
|
||||
function loadCert(ds){
|
||||
$('viewer').innerHTML='<iframe id="dsframe" title="datasheet"></iframe>';
|
||||
const f=$('dsframe'); f.onload=()=>{styleCert();fitCert();}; f.src=ds+'?format=html';
|
||||
const f=$('dsframe'); f.onload=()=>{styleCert();fitCert();setTimeout(fitCert,120);setTimeout(fitCert,350);}; f.src=ds+'?format=html';
|
||||
}
|
||||
function styleCert(){ const f=$('dsframe'); try{ const doc=f.contentDocument; if(!doc)return;
|
||||
if(!doc.getElementById('_inj')){ const s=doc.createElement('style'); s.id='_inj';
|
||||
s.textContent='html,body{background:#fff!important;margin:0!important}body{padding:22px 26px!important;color:#0f172a}pre{margin:0;font-family:'+getComputedStyle(document.body).getPropertyValue('--mono')+';font-size:12.5px;line-height:1.32}';
|
||||
s.textContent='html,body{background:#fff!important;margin:0!important}body{padding:16px 20px!important;color:#0f172a}pre{margin:0;font-family:'+getComputedStyle(document.body).getPropertyValue('--mono')+';font-size:12.5px;line-height:1.32}';
|
||||
(doc.head||doc.documentElement).appendChild(s); }
|
||||
}catch(e){} }
|
||||
function fitCert(){ const f=$('dsframe'); if(!f)return; try{ const doc=f.contentDocument; if(!doc)return;
|
||||
const root=doc.documentElement; root.style.zoom='';
|
||||
const nat=Math.max(doc.body?doc.body.scrollWidth:0,root.scrollWidth), av=f.clientWidth-2;
|
||||
root.style.zoom=(nat>av)?Math.max(.45,av/nat):1;
|
||||
f.style.height=Math.ceil((doc.body?doc.body.scrollHeight:600)*(nat>av?av/nat:1)+4)+'px';
|
||||
function fitCert(){ const f=$('dsframe'); if(!f)return; try{ const doc=f.contentDocument; if(!doc||!doc.body)return;
|
||||
const b=doc.body, root=doc.documentElement;
|
||||
// measure natural content width (transform doesn't reflow, so text never rewraps)
|
||||
b.style.transform='none'; b.style.width='max-content'; b.style.transformOrigin='0 0';
|
||||
root.style.overflow='hidden';
|
||||
const nat=Math.max(b.scrollWidth,root.scrollWidth), av=f.clientWidth-12; // widest line + right margin
|
||||
if(!nat){ return; } // not laid out yet — the retry will catch it
|
||||
const k=nat>av ? Math.max(.4, av/nat) : 1;
|
||||
b.style.transform='scale('+k+')';
|
||||
f.style.height=Math.ceil(b.scrollHeight*k+2)+'px'; // size the frame to the scaled cert, no v-scroll-in-frame
|
||||
}catch(e){} }
|
||||
function printCert(){ const f=$('dsframe'); if(f&&f.contentWindow){f.contentWindow.focus();f.contentWindow.print();} }
|
||||
window.addEventListener('resize',()=>{fitCert();});
|
||||
@@ -363,6 +373,31 @@ $('copySel').onclick=()=>{ const sns=state.rows.filter(r=>state.checks.has(Strin
|
||||
navigator.clipboard&&navigator.clipboard.writeText(sns); $('copySel').textContent='Copied ✓'; setTimeout(()=>$('copySel').textContent='Copy serials',1200); };
|
||||
$('selclear').onclick=()=>{ state.checks.clear(); [...$('rows').querySelectorAll('[data-ck]')].forEach(c=>c.checked=false); $('ckAll').checked=false; updateSel(); };
|
||||
|
||||
/* ---------- publish to public website (POST /api/upload, idempotent) ---------- */
|
||||
async function doUpload(payload){
|
||||
const r=await fetch(API+'/api/upload',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify(payload)});
|
||||
const d=await r.json().catch(()=>({})); if(!r.ok) throw new Error(d.error||('HTTP '+r.status)); return d;
|
||||
}
|
||||
async function pushWeb(id,btn){
|
||||
const r=state.rows.find(x=>x.id==id); const sn=r?r.serial_number:id;
|
||||
if(!confirm('Publish '+sn+' to the public Dataforth website now?')) return;
|
||||
const t=btn.textContent; btn.disabled=true; btn.textContent='Publishing…';
|
||||
try{ const d=await doUpload({ids:[+id]});
|
||||
btn.textContent=d.errors?('✕ '+d.errors+' err'):(d.skipped&&!(d.created+d.updated+d.unchanged)?'skipped':'Published ✓');
|
||||
setTimeout(search,500); // refresh WEB status from the DB
|
||||
}catch(e){ btn.textContent='✕ '+e.message.slice(0,16); btn.disabled=false; }
|
||||
}
|
||||
async function pushSelected(){
|
||||
const ids=[...state.checks].map(Number); if(!ids.length) return;
|
||||
if(!confirm('Publish '+ids.length+' selected serial(s) to the public Dataforth website now?')) return;
|
||||
const b=$('repushSel'),t=b.textContent; b.disabled=true; b.textContent='Publishing…';
|
||||
try{ const d=await doUpload({ids});
|
||||
b.textContent='✓ '+((d.created||0)+(d.updated||0))+' pushed'+(d.skipped?(' · '+d.skipped+' skip'):'');
|
||||
setTimeout(()=>{b.textContent=t;b.disabled=false;search();},1400);
|
||||
}catch(e){ b.textContent='✕ failed'; b.disabled=false; alert('Push failed: '+e.message); }
|
||||
}
|
||||
$('repushSel').onclick=pushSelected;
|
||||
|
||||
/* ---------- sort ---------- */
|
||||
function updateSortHeads(){ [...document.querySelectorAll('thead th.s')].forEach(th=>{
|
||||
const on=th.dataset.s===state.sort; th.querySelector('.arr')?.remove();
|
||||
@@ -385,12 +420,12 @@ $('pageSize').onchange=e=>{state.size=+e.target.value;state.page=0;search();};
|
||||
$('prev').onclick=()=>{if(state.page>0){state.page--;search();$('twrap').scrollTop=0;}};
|
||||
$('next').onclick=()=>{state.page++;search();$('twrap').scrollTop=0;};
|
||||
$('reset').onclick=()=>{ ['serial','model','q','result','station','logtype','from','to'].forEach(k=>state[k]='');
|
||||
state.sort='';state.checks.clear();updateSel(); $('omni').value='';$('mode').textContent=state.force||'auto';
|
||||
state.sort='';state.webStatus='';state.checks.clear();updateSel(); $('omni').value='';$('mode').textContent=state.force||'auto';
|
||||
$('fFrom').value=$('fTo').value=$('fStation').value=$('fLog').value=$('fModel').value=''; state.page=0; search(); };
|
||||
$('menu').onclick=()=>document.body.classList.toggle('rail-open');
|
||||
|
||||
/* ---------- presets ---------- */
|
||||
function clearAll(){ ['serial','model','q','result','station','logtype','from','to'].forEach(k=>state[k]='');
|
||||
function clearAll(){ ['serial','model','q','result','station','logtype','from','to'].forEach(k=>state[k]=''); state.webStatus='';state.sort='';
|
||||
$('omni').value='';$('mode').textContent=state.force||'auto';$('fFrom').value=$('fTo').value=$('fStation').value=$('fLog').value=$('fModel').value=''; }
|
||||
function applyPreset(fn){ clearAll(); fn(); state.page=0; state.checks.clear(); updateSel(); document.body.classList.remove('rail-open'); search(); }
|
||||
const PRESETS=[
|
||||
@@ -398,11 +433,11 @@ const PRESETS=[
|
||||
{ic:'✕',label:'Failures',fn:()=>{state.result='FAIL';}},
|
||||
{ic:'•',label:'Today',fn:()=>{const t=isoD(new Date());state.from=t;state.to=t;}},
|
||||
{ic:'7',label:'Last 7 days',fn:()=>{const d=new Date();d.setDate(d.getDate()-7);state.from=isoD(d);}},
|
||||
{ic:'▴',label:'Latest uploads',fn:()=>{state.webStatus='on';state.sort='api_uploaded_at';state.dir='desc';}},
|
||||
{ic:'○',label:'Not yet published',fn:()=>{state.webStatus='off';}},
|
||||
];
|
||||
const SOON=[
|
||||
{label:'Latest upload batch',why:'needs sort=uploaded in /api/search'},
|
||||
{label:'Retested units',why:'needs a retest flag in the pipeline'},
|
||||
{label:'Not yet published',why:'needs a published filter in /api/search'},
|
||||
{label:'Retested units',why:'needs a retest flag in the pipeline (next)'},
|
||||
];
|
||||
function renderPresets(){
|
||||
$('presets').innerHTML='';
|
||||
|
||||
@@ -1,27 +1,17 @@
|
||||
/**
|
||||
* API Routes for Test Data Database
|
||||
*
|
||||
* Fixed version - uses a single persistent database connection instead of
|
||||
* opening and closing on every request. WAL journal mode enabled for
|
||||
* concurrent read support. Limit parameter capped at 1000.
|
||||
* PostgreSQL version - uses pg.Pool via database/db.js.
|
||||
* All route handlers are async. FTS uses tsvector/plainto_tsquery.
|
||||
*/
|
||||
|
||||
const express = require('express');
|
||||
const path = require('path');
|
||||
const Database = require('better-sqlite3');
|
||||
const db = require('../database/db');
|
||||
const { generateDatasheet } = require('../templates/datasheet');
|
||||
|
||||
const router = express.Router();
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Singleton database connection - opened once at module load
|
||||
// ---------------------------------------------------------------------------
|
||||
const DB_PATH = path.join(__dirname, '..', 'database', 'testdata.db');
|
||||
|
||||
const db = new Database(DB_PATH, { readonly: false });
|
||||
db.pragma('journal_mode = WAL');
|
||||
db.pragma('busy_timeout = 5000');
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -42,107 +32,88 @@ function clampOffset(value) {
|
||||
// ---------------------------------------------------------------------------
|
||||
// GET /api/search
|
||||
// Search test records
|
||||
// Query params: serial, model, from, to, result, q, station, logtype, limit, offset
|
||||
// Query params: serial, model, from, to, result, q, station, logtype, web_status, limit, offset
|
||||
// ---------------------------------------------------------------------------
|
||||
router.get('/search', (req, res) => {
|
||||
router.get('/search', async (req, res) => {
|
||||
try {
|
||||
const { serial, model, from, to, result, q, station, logtype } = req.query;
|
||||
const { serial, model, from, to, result, q, station, logtype, workorder, web_status } = req.query;
|
||||
const limit = clampLimit(req.query.limit || 100);
|
||||
const offset = clampOffset(req.query.offset || 0);
|
||||
|
||||
let sql = 'SELECT * FROM test_records WHERE 1=1';
|
||||
const conditions = [];
|
||||
const params = [];
|
||||
let paramIdx = 0;
|
||||
|
||||
const addParam = (val) => {
|
||||
paramIdx++;
|
||||
params.push(val);
|
||||
return '$' + paramIdx;
|
||||
};
|
||||
|
||||
if (q) {
|
||||
// Full-text search using tsvector
|
||||
conditions.push(`search_vector @@ plainto_tsquery('english', ${addParam(q)})`);
|
||||
}
|
||||
|
||||
if (serial) {
|
||||
sql += ' AND serial_number LIKE ?';
|
||||
params.push(serial.includes('%') ? serial : `%${serial}%`);
|
||||
const val = serial.includes('%') ? serial : `%${serial}%`;
|
||||
conditions.push(`serial_number LIKE ${addParam(val)}`);
|
||||
}
|
||||
|
||||
if (workorder) {
|
||||
conditions.push(`work_order = ${addParam(workorder)}`);
|
||||
}
|
||||
|
||||
if (model) {
|
||||
sql += ' AND model_number LIKE ?';
|
||||
params.push(model.includes('%') ? model : `%${model}%`);
|
||||
const val = model.includes('%') ? model : `%${model}%`;
|
||||
conditions.push(`model_number LIKE ${addParam(val)}`);
|
||||
}
|
||||
|
||||
if (from) {
|
||||
sql += ' AND test_date >= ?';
|
||||
params.push(from);
|
||||
conditions.push(`test_date >= ${addParam(from)}`);
|
||||
}
|
||||
|
||||
if (to) {
|
||||
sql += ' AND test_date <= ?';
|
||||
params.push(to);
|
||||
conditions.push(`test_date <= ${addParam(to)}`);
|
||||
}
|
||||
|
||||
if (result) {
|
||||
sql += ' AND overall_result = ?';
|
||||
params.push(result.toUpperCase());
|
||||
conditions.push(`overall_result = ${addParam(result.toUpperCase())}`);
|
||||
}
|
||||
|
||||
if (station) {
|
||||
sql += ' AND test_station = ?';
|
||||
params.push(station);
|
||||
conditions.push(`test_station = ${addParam(station)}`);
|
||||
}
|
||||
|
||||
if (logtype) {
|
||||
sql += ' AND log_type = ?';
|
||||
params.push(logtype);
|
||||
conditions.push(`log_type = ${addParam(logtype)}`);
|
||||
}
|
||||
|
||||
if (q) {
|
||||
// Full-text search - rebuild query with FTS
|
||||
sql = `SELECT test_records.* FROM test_records
|
||||
JOIN test_records_fts ON test_records.id = test_records_fts.rowid
|
||||
WHERE test_records_fts MATCH ?`;
|
||||
params.length = 0;
|
||||
params.push(q);
|
||||
|
||||
if (serial) {
|
||||
sql += ' AND serial_number LIKE ?';
|
||||
params.push(serial.includes('%') ? serial : `%${serial}%`);
|
||||
}
|
||||
if (model) {
|
||||
sql += ' AND model_number LIKE ?';
|
||||
params.push(model.includes('%') ? model : `%${model}%`);
|
||||
}
|
||||
if (station) {
|
||||
sql += ' AND test_station = ?';
|
||||
params.push(station);
|
||||
}
|
||||
if (logtype) {
|
||||
sql += ' AND log_type = ?';
|
||||
params.push(logtype);
|
||||
}
|
||||
if (result) {
|
||||
sql += ' AND overall_result = ?';
|
||||
params.push(result.toUpperCase());
|
||||
}
|
||||
if (from) {
|
||||
sql += ' AND test_date >= ?';
|
||||
params.push(from);
|
||||
}
|
||||
if (to) {
|
||||
sql += ' AND test_date <= ?';
|
||||
params.push(to);
|
||||
}
|
||||
if (req.query.web_status === 'off') {
|
||||
conditions.push('api_uploaded_at IS NULL');
|
||||
} else if (req.query.web_status === 'on') {
|
||||
conditions.push('api_uploaded_at IS NOT NULL');
|
||||
}
|
||||
|
||||
sql += ' ORDER BY test_date DESC, serial_number';
|
||||
sql += ' LIMIT ? OFFSET ?';
|
||||
params.push(limit, offset);
|
||||
const where = conditions.length > 0 ? 'WHERE ' + conditions.join(' AND ') : '';
|
||||
|
||||
const records = db.prepare(sql).all(...params);
|
||||
// whitelisted sort (prevents injection); NULLS LAST so e.g. unpublished rows don't lead an api_uploaded_at sort
|
||||
const SORTABLE = { serial_number:'serial_number', model_number:'model_number', test_date:'test_date', overall_result:'overall_result', test_station:'test_station', log_type:'log_type', api_uploaded_at:'api_uploaded_at' };
|
||||
const sortCol = SORTABLE[req.query.sort];
|
||||
const sortDir = String(req.query.dir || '').toLowerCase() === 'asc' ? 'ASC' : 'DESC';
|
||||
const orderBy = sortCol ? `ORDER BY ${sortCol} ${sortDir} NULLS LAST, serial_number` : 'ORDER BY test_date DESC, serial_number';
|
||||
const dataSql = `SELECT * FROM test_records ${where} ${orderBy} LIMIT ${addParam(limit)} OFFSET ${addParam(offset)}`;
|
||||
const countSql = `SELECT COUNT(*) as count FROM test_records ${where}`;
|
||||
const countParams = params.slice(0, paramIdx - 2); // exclude limit/offset
|
||||
|
||||
// Get total count
|
||||
let countSql = sql.replace(/SELECT .* FROM/, 'SELECT COUNT(*) as count FROM')
|
||||
.replace(/ORDER BY.*$/, '');
|
||||
countSql = countSql.replace(/LIMIT \? OFFSET \?/, '');
|
||||
|
||||
const countParams = params.slice(0, -2);
|
||||
const total = db.prepare(countSql).get(...countParams);
|
||||
const [records, countRow] = await Promise.all([
|
||||
db.query(dataSql, params),
|
||||
db.queryOne(countSql, countParams),
|
||||
]);
|
||||
|
||||
res.json({
|
||||
records,
|
||||
total: total?.count || records.length,
|
||||
total: countRow?.count ? parseInt(countRow.count, 10) : records.length,
|
||||
limit,
|
||||
offset
|
||||
});
|
||||
@@ -156,9 +127,9 @@ router.get('/search', (req, res) => {
|
||||
// GET /api/record/:id
|
||||
// Get single record by ID
|
||||
// ---------------------------------------------------------------------------
|
||||
router.get('/record/:id', (req, res) => {
|
||||
router.get('/record/:id', async (req, res) => {
|
||||
try {
|
||||
const record = db.prepare('SELECT * FROM test_records WHERE id = ?').get(req.params.id);
|
||||
const record = await db.queryOne('SELECT * FROM test_records WHERE id = $1', [req.params.id]);
|
||||
|
||||
if (!record) {
|
||||
return res.status(404).json({ error: 'Record not found' });
|
||||
@@ -176,21 +147,102 @@ router.get('/record/:id', (req, res) => {
|
||||
// Generate datasheet for a record
|
||||
// Query params: format (html, txt)
|
||||
// ---------------------------------------------------------------------------
|
||||
router.get('/datasheet/:id', (req, res) => {
|
||||
router.get('/datasheet/:id', async (req, res) => {
|
||||
try {
|
||||
const record = db.prepare('SELECT * FROM test_records WHERE id = ?').get(req.params.id);
|
||||
const record = await db.queryOne('SELECT * FROM test_records WHERE id = $1', [req.params.id]);
|
||||
|
||||
if (!record) {
|
||||
return res.status(404).json({ error: 'Record not found' });
|
||||
}
|
||||
|
||||
const format = req.query.format || 'html';
|
||||
const datasheet = generateDatasheet(record, format);
|
||||
|
||||
if (format === 'html') {
|
||||
res.type('html').send(datasheet);
|
||||
// Try exact-match formatter first
|
||||
const { loadAllSpecs, getSpecs } = require('../parsers/spec-reader');
|
||||
const { generateExactDatasheet } = require('../templates/datasheet-exact');
|
||||
const specMap = loadAllSpecs();
|
||||
const specs = getSpecs(specMap, record.model_number);
|
||||
const exactTxt = generateExactDatasheet(record, specs);
|
||||
|
||||
if (exactTxt && format === 'html') {
|
||||
// Render exact-match TXT as styled HTML page
|
||||
const escaped = exactTxt
|
||||
.replace(/&/g, '&')
|
||||
.replace(/</g, '<')
|
||||
.replace(/>/g, '>');
|
||||
const html = `<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>Test Data Sheet - ${record.serial_number}</title>
|
||||
<style>
|
||||
body {
|
||||
margin: 0;
|
||||
padding: 20px;
|
||||
background: #f0f0f0;
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
}
|
||||
.page {
|
||||
background: white;
|
||||
padding: 40px 30px;
|
||||
max-width: 720px;
|
||||
width: 100%;
|
||||
box-shadow: 0 2px 8px rgba(0,0,0,0.15);
|
||||
border: 1px solid #ccc;
|
||||
}
|
||||
pre {
|
||||
font-family: 'Courier New', Courier, monospace;
|
||||
font-size: 11px;
|
||||
line-height: 1.4;
|
||||
margin: 0;
|
||||
white-space: pre;
|
||||
overflow-x: auto;
|
||||
}
|
||||
.toolbar {
|
||||
position: fixed;
|
||||
top: 10px;
|
||||
right: 10px;
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
}
|
||||
.toolbar button {
|
||||
padding: 8px 16px;
|
||||
border: 1px solid #999;
|
||||
background: white;
|
||||
cursor: pointer;
|
||||
font-size: 13px;
|
||||
border-radius: 4px;
|
||||
}
|
||||
.toolbar button:hover { background: #e0e0e0; }
|
||||
@media print {
|
||||
body { background: white; padding: 0; }
|
||||
.page { box-shadow: none; border: none; padding: 0; }
|
||||
.toolbar { display: none; }
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="toolbar">
|
||||
<button onclick="window.print()">Print</button>
|
||||
<button onclick="window.open('/api/datasheet/${record.id}/pdf')">Download PDF</button>
|
||||
<button onclick="window.close()">Close</button>
|
||||
</div>
|
||||
<div class="page">
|
||||
<pre>${escaped}</pre>
|
||||
</div>
|
||||
</body>
|
||||
</html>`;
|
||||
res.type('html').send(html);
|
||||
} else if (exactTxt && format === 'txt') {
|
||||
res.type('text/plain').send(exactTxt);
|
||||
} else {
|
||||
res.type('text/plain').send(datasheet);
|
||||
// Fall back to generic template
|
||||
const datasheet = generateDatasheet(record, format);
|
||||
if (format === 'html') {
|
||||
res.type('html').send(datasheet);
|
||||
} else {
|
||||
res.type('text/plain').send(datasheet);
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
console.error(`[${new Date().toISOString()}] [DATASHEET ERROR] ${err.message}`);
|
||||
@@ -198,45 +250,83 @@ router.get('/datasheet/:id', (req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// GET /api/datasheet/:id/pdf
|
||||
// Generate PDF datasheet for a record (on-demand download)
|
||||
// ---------------------------------------------------------------------------
|
||||
router.get('/datasheet/:id/pdf', async (req, res) => {
|
||||
try {
|
||||
const record = await db.queryOne('SELECT * FROM test_records WHERE id = $1', [req.params.id]);
|
||||
|
||||
if (!record) {
|
||||
return res.status(404).json({ error: 'Record not found' });
|
||||
}
|
||||
|
||||
const { loadAllSpecs, getSpecs } = require('../parsers/spec-reader');
|
||||
const { generateExactDatasheet } = require('../templates/datasheet-exact');
|
||||
const PDFDocument = require('pdfkit');
|
||||
|
||||
const specMap = loadAllSpecs();
|
||||
const specs = getSpecs(specMap, record.model_number);
|
||||
let txt = generateExactDatasheet(record, specs);
|
||||
|
||||
// Fall back to generic datasheet if exact-match formatter doesn't support this family
|
||||
if (!txt) {
|
||||
txt = generateDatasheet(record, 'txt');
|
||||
}
|
||||
|
||||
if (!txt) {
|
||||
return res.status(422).json({ error: 'Could not generate datasheet (missing specs or data)' });
|
||||
}
|
||||
|
||||
const doc = new PDFDocument({
|
||||
size: 'LETTER',
|
||||
margins: { top: 36, bottom: 36, left: 36, right: 36 }
|
||||
});
|
||||
|
||||
res.setHeader('Content-Type', 'application/pdf');
|
||||
res.setHeader('Content-Disposition', `attachment; filename="${record.serial_number}.pdf"`);
|
||||
doc.pipe(res);
|
||||
|
||||
doc.font('Courier').fontSize(9.5);
|
||||
const lines = txt.split(/\r?\n/);
|
||||
for (const line of lines) {
|
||||
doc.text(line, { lineGap: 1 });
|
||||
}
|
||||
|
||||
doc.end();
|
||||
} catch (err) {
|
||||
console.error(`[${new Date().toISOString()}] [PDF ERROR] ${err.message}`);
|
||||
res.status(500).json({ error: err.message });
|
||||
}
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// GET /api/stats
|
||||
// Get database statistics
|
||||
// ---------------------------------------------------------------------------
|
||||
router.get('/stats', (req, res) => {
|
||||
router.get('/stats', async (req, res) => {
|
||||
try {
|
||||
const stats = {
|
||||
total_records: db.prepare('SELECT COUNT(*) as count FROM test_records').get().count,
|
||||
by_log_type: db.prepare(`
|
||||
SELECT log_type, COUNT(*) as count
|
||||
FROM test_records
|
||||
GROUP BY log_type
|
||||
ORDER BY count DESC
|
||||
`).all(),
|
||||
by_result: db.prepare(`
|
||||
SELECT overall_result, COUNT(*) as count
|
||||
FROM test_records
|
||||
GROUP BY overall_result
|
||||
`).all(),
|
||||
by_station: db.prepare(`
|
||||
SELECT test_station, COUNT(*) as count
|
||||
FROM test_records
|
||||
WHERE test_station IS NOT NULL AND test_station != ''
|
||||
GROUP BY test_station
|
||||
ORDER BY test_station
|
||||
`).all(),
|
||||
date_range: db.prepare(`
|
||||
SELECT MIN(test_date) as oldest, MAX(test_date) as newest
|
||||
FROM test_records
|
||||
`).get(),
|
||||
recent_serials: db.prepare(`
|
||||
SELECT DISTINCT serial_number, model_number, test_date
|
||||
FROM test_records
|
||||
ORDER BY test_date DESC
|
||||
LIMIT 10
|
||||
`).all()
|
||||
};
|
||||
const [totalRow, byLogType, byResult, byStation, dateRange, recentSerials] = await Promise.all([
|
||||
db.queryOne('SELECT COUNT(*) as count FROM test_records'),
|
||||
db.query('SELECT log_type, COUNT(*) as count FROM test_records GROUP BY log_type ORDER BY count DESC'),
|
||||
db.query('SELECT overall_result, COUNT(*) as count FROM test_records GROUP BY overall_result'),
|
||||
db.query(`SELECT test_station, COUNT(*) as count FROM test_records
|
||||
WHERE test_station IS NOT NULL AND test_station != ''
|
||||
GROUP BY test_station ORDER BY test_station`),
|
||||
db.queryOne('SELECT MIN(test_date) as oldest, MAX(test_date) as newest FROM test_records'),
|
||||
db.query(`SELECT DISTINCT serial_number, model_number, test_date
|
||||
FROM test_records ORDER BY test_date DESC LIMIT 10`),
|
||||
]);
|
||||
|
||||
res.json(stats);
|
||||
res.json({
|
||||
total_records: parseInt(totalRow.count, 10),
|
||||
by_log_type: byLogType.map(r => ({ ...r, count: parseInt(r.count, 10) })),
|
||||
by_result: byResult.map(r => ({ ...r, count: parseInt(r.count, 10) })),
|
||||
by_station: byStation.map(r => ({ ...r, count: parseInt(r.count, 10) })),
|
||||
date_range: dateRange,
|
||||
recent_serials: recentSerials,
|
||||
});
|
||||
} catch (err) {
|
||||
console.error(`[${new Date().toISOString()}] [STATS ERROR] ${err.message}`);
|
||||
res.status(500).json({ error: err.message });
|
||||
@@ -247,30 +337,22 @@ router.get('/stats', (req, res) => {
|
||||
// GET /api/filters
|
||||
// Get available filter options (test stations, log types, models)
|
||||
// ---------------------------------------------------------------------------
|
||||
router.get('/filters', (req, res) => {
|
||||
router.get('/filters', async (req, res) => {
|
||||
try {
|
||||
const filters = {
|
||||
stations: db.prepare(`
|
||||
SELECT DISTINCT test_station
|
||||
FROM test_records
|
||||
WHERE test_station IS NOT NULL AND test_station != ''
|
||||
ORDER BY test_station
|
||||
`).all().map(r => r.test_station),
|
||||
log_types: db.prepare(`
|
||||
SELECT DISTINCT log_type
|
||||
FROM test_records
|
||||
ORDER BY log_type
|
||||
`).all().map(r => r.log_type),
|
||||
models: db.prepare(`
|
||||
SELECT DISTINCT model_number, COUNT(*) as count
|
||||
FROM test_records
|
||||
GROUP BY model_number
|
||||
ORDER BY count DESC
|
||||
LIMIT 500
|
||||
`).all()
|
||||
};
|
||||
const [stations, logTypes, models] = await Promise.all([
|
||||
db.query(`SELECT DISTINCT test_station FROM test_records
|
||||
WHERE test_station IS NOT NULL AND test_station != ''
|
||||
ORDER BY test_station`),
|
||||
db.query('SELECT DISTINCT log_type FROM test_records ORDER BY log_type'),
|
||||
db.query(`SELECT DISTINCT model_number, COUNT(*) as count FROM test_records
|
||||
GROUP BY model_number ORDER BY count DESC LIMIT 500`),
|
||||
]);
|
||||
|
||||
res.json(filters);
|
||||
res.json({
|
||||
stations: stations.map(r => r.test_station),
|
||||
log_types: logTypes.map(r => r.log_type),
|
||||
models: models.map(r => ({ ...r, count: parseInt(r.count, 10) })),
|
||||
});
|
||||
} catch (err) {
|
||||
console.error(`[${new Date().toISOString()}] [FILTERS ERROR] ${err.message}`);
|
||||
res.status(500).json({ error: err.message });
|
||||
@@ -281,51 +363,60 @@ router.get('/filters', (req, res) => {
|
||||
// GET /api/export
|
||||
// Export search results as CSV
|
||||
// ---------------------------------------------------------------------------
|
||||
router.get('/export', (req, res) => {
|
||||
router.get('/export', async (req, res) => {
|
||||
try {
|
||||
const { serial, model, from, to, result, station, logtype } = req.query;
|
||||
|
||||
let sql = 'SELECT * FROM test_records WHERE 1=1';
|
||||
const conditions = [];
|
||||
const params = [];
|
||||
let paramIdx = 0;
|
||||
|
||||
const addParam = (val) => {
|
||||
paramIdx++;
|
||||
params.push(val);
|
||||
return '$' + paramIdx;
|
||||
};
|
||||
|
||||
if (serial) {
|
||||
sql += ' AND serial_number LIKE ?';
|
||||
params.push(serial.includes('%') ? serial : `%${serial}%`);
|
||||
const val = serial.includes('%') ? serial : `%${serial}%`;
|
||||
conditions.push(`serial_number LIKE ${addParam(val)}`);
|
||||
}
|
||||
|
||||
if (model) {
|
||||
sql += ' AND model_number LIKE ?';
|
||||
params.push(model.includes('%') ? model : `%${model}%`);
|
||||
const val = model.includes('%') ? model : `%${model}%`;
|
||||
conditions.push(`model_number LIKE ${addParam(val)}`);
|
||||
}
|
||||
|
||||
if (from) {
|
||||
sql += ' AND test_date >= ?';
|
||||
params.push(from);
|
||||
conditions.push(`test_date >= ${addParam(from)}`);
|
||||
}
|
||||
|
||||
if (to) {
|
||||
sql += ' AND test_date <= ?';
|
||||
params.push(to);
|
||||
conditions.push(`test_date <= ${addParam(to)}`);
|
||||
}
|
||||
|
||||
if (result) {
|
||||
sql += ' AND overall_result = ?';
|
||||
params.push(result.toUpperCase());
|
||||
conditions.push(`overall_result = ${addParam(result.toUpperCase())}`);
|
||||
}
|
||||
|
||||
if (station) {
|
||||
sql += ' AND test_station = ?';
|
||||
params.push(station);
|
||||
conditions.push(`test_station = ${addParam(station)}`);
|
||||
}
|
||||
|
||||
if (logtype) {
|
||||
sql += ' AND log_type = ?';
|
||||
params.push(logtype);
|
||||
conditions.push(`log_type = ${addParam(logtype)}`);
|
||||
}
|
||||
|
||||
sql += ' ORDER BY test_date DESC, serial_number LIMIT 10000';
|
||||
if (req.query.web_status === 'off') {
|
||||
conditions.push('api_uploaded_at IS NULL');
|
||||
} else if (req.query.web_status === 'on') {
|
||||
conditions.push('api_uploaded_at IS NOT NULL');
|
||||
}
|
||||
|
||||
const records = db.prepare(sql).all(...params);
|
||||
const where = conditions.length > 0 ? 'WHERE ' + conditions.join(' AND ') : '';
|
||||
const sql = `SELECT * FROM test_records ${where} ORDER BY test_date DESC, serial_number LIMIT 10000`;
|
||||
|
||||
const records = await db.query(sql, params);
|
||||
|
||||
// Generate CSV
|
||||
const headers = ['id', 'log_type', 'model_number', 'serial_number', 'test_date', 'test_station', 'overall_result', 'source_file'];
|
||||
@@ -348,16 +439,119 @@ router.get('/export', (req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// GET /api/workorder/:wo
|
||||
// Get work order details and all associated test lines
|
||||
// ---------------------------------------------------------------------------
|
||||
router.get('/workorder/:wo', async (req, res) => {
|
||||
try {
|
||||
const wo = req.params.wo;
|
||||
|
||||
const [header, lines, testRecords] = await Promise.all([
|
||||
db.queryOne('SELECT * FROM work_orders WHERE wo_number = $1', [wo]),
|
||||
db.query('SELECT * FROM work_order_lines WHERE wo_number = $1 ORDER BY test_date, test_time', [wo]),
|
||||
db.query(
|
||||
'SELECT id, log_type, model_number, serial_number, test_date, test_station, overall_result, work_order FROM test_records WHERE work_order = $1 ORDER BY serial_number',
|
||||
[wo]
|
||||
),
|
||||
]);
|
||||
|
||||
res.json({
|
||||
work_order: header || { wo_number: wo },
|
||||
lines,
|
||||
test_records: testRecords,
|
||||
});
|
||||
} catch (err) {
|
||||
console.error(`[${new Date().toISOString()}] [WO ERROR] ${err.message}`);
|
||||
res.status(500).json({ error: err.message });
|
||||
}
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// GET /api/workorder-search?q=<query>
|
||||
// Search work orders by number (prefix match)
|
||||
// ---------------------------------------------------------------------------
|
||||
router.get('/workorder-search', async (req, res) => {
|
||||
try {
|
||||
const q = req.query.q || '';
|
||||
if (q.length < 2) {
|
||||
return res.json({ results: [] });
|
||||
}
|
||||
|
||||
const results = await db.query(
|
||||
'SELECT wo_number, wo_date, program, test_station FROM work_orders WHERE wo_number LIKE $1 ORDER BY wo_date DESC LIMIT 50',
|
||||
[q + '%']
|
||||
);
|
||||
|
||||
res.json({ results });
|
||||
} catch (err) {
|
||||
res.status(500).json({ error: err.message });
|
||||
}
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Cleanup function for graceful shutdown
|
||||
// ---------------------------------------------------------------------------
|
||||
function cleanup() {
|
||||
async function cleanup() {
|
||||
try {
|
||||
db.close();
|
||||
await db.close();
|
||||
} catch (err) {
|
||||
console.error(`[${new Date().toISOString()}] [CLEANUP ERROR] ${err.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* POST /api/upload
|
||||
*
|
||||
* Body: { ids?: number[], serialNumbers?: string[], all_unuploaded?: boolean }
|
||||
*
|
||||
* Pushes selected records to the Dataforth website API. Accepts either a set
|
||||
* of record IDs (resolved to serial_number + checked for exported status), a
|
||||
* direct list of serial numbers, or all_unuploaded:true to push every PASS
|
||||
* record where api_uploaded_at IS NULL.
|
||||
*
|
||||
* Response: { created, updated, unchanged, errors, skipped, processed, sns }
|
||||
*/
|
||||
router.post('/upload', async (req, res) => {
|
||||
try {
|
||||
const { ids, serialNumbers, all_unuploaded } = req.body || {};
|
||||
const { uploadBySerialNumbers } = require('../database/upload-to-api');
|
||||
|
||||
let sns = [];
|
||||
if (all_unuploaded) {
|
||||
const rows = await db.query(
|
||||
`SELECT DISTINCT serial_number FROM test_records
|
||||
WHERE overall_result = 'PASS'
|
||||
AND api_uploaded_at IS NULL
|
||||
ORDER BY serial_number`
|
||||
);
|
||||
sns = rows.map(r => r.serial_number);
|
||||
} else if (Array.isArray(ids) && ids.length > 0) {
|
||||
const placeholders = ids.map((_, i) => `$${i + 1}`).join(',');
|
||||
const rows = await db.query(
|
||||
`SELECT DISTINCT serial_number FROM test_records
|
||||
WHERE id IN (${placeholders})
|
||||
AND overall_result = 'PASS'`,
|
||||
ids,
|
||||
);
|
||||
sns = rows.map(r => r.serial_number);
|
||||
} else if (Array.isArray(serialNumbers) && serialNumbers.length > 0) {
|
||||
sns = [...new Set(serialNumbers)];
|
||||
} else {
|
||||
return res.status(400).json({ error: 'provide ids[], serialNumbers[], or all_unuploaded=true' });
|
||||
}
|
||||
|
||||
if (sns.length === 0) {
|
||||
return res.json({ created:0, updated:0, unchanged:0, errors:0, skipped:0, processed:0, sns:[] });
|
||||
}
|
||||
|
||||
const result = await uploadBySerialNumbers(sns);
|
||||
res.json({ ...result, processed: sns.length, sns });
|
||||
} catch (err) {
|
||||
console.error(`[UPLOAD] ${err.message}`);
|
||||
res.status(500).json({ error: err.message });
|
||||
}
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
module.exports.cleanup = cleanup;
|
||||
|
||||
171
projects/dataforth-dos/tools/derive-dsca-slotmaps.js
Normal file
171
projects/dataforth-dos/tools/derive-dsca-slotmaps.js
Normal file
@@ -0,0 +1,171 @@
|
||||
// Fix 2 — derive per-model DSCA slot maps for the ambiguous layouts.
|
||||
//
|
||||
// Problem: for some DSCA subtypes the raw_data STATUS groups carry MORE (or fewer)
|
||||
// value-bearing entries than the template's spec-bearing rows — the test program
|
||||
// measures slots (e.g. an extra 5mA load pair) that the printed sheet omits. A
|
||||
// simple in-order zip then misaligns values onto rows, so those models are skipped
|
||||
// by the renderer's count-guard.
|
||||
//
|
||||
// Fix: for each such model, pair a staged original with its DB raw_data and greedily
|
||||
// match each printed measured value to a STATUS entry (same fround formatting the
|
||||
// renderer uses), recording the ABSOLUTE statusEntries index per spec-bearing row.
|
||||
// That ordered subsequence is the slotMap; the renderer reads statusEntries[slotMap[s]]
|
||||
// for the s-th spec-bearing row. Stored in dsca-templates.json as `slotMap`.
|
||||
//
|
||||
// Data-vintage safe: a staged unit that is a retest (printed values != DB record)
|
||||
// won't match cleanly; we try multiple staged units per model and accept the first
|
||||
// that matches ALL spec rows. Read-only except the templates JSON it rewrites.
|
||||
const fs = require('fs'), path = require('path');
|
||||
// This tool is specific to the deployed pipeline (it reads the staged originals and
|
||||
// rewrites the deployed templates JSON), so it requires the deployed modules by
|
||||
// absolute path and is runnable from anywhere.
|
||||
const DEPLOY = 'C:/Shares/testdatadb';
|
||||
const db = require(DEPLOY + '/database/db');
|
||||
const dse = require(DEPLOY + '/templates/datasheet-exact');
|
||||
|
||||
const STAGE = 'C:/Shares/test/STAGE';
|
||||
const OUT = DEPLOY + '/dsca-templates.json';
|
||||
|
||||
function walk(d, out) { let it = []; try { it = fs.readdirSync(d, { withFileTypes: true }); } catch { return out; } for (const e of it) { const p = path.join(d, e.name); if (e.isDirectory()) walk(p, out); else if (/\.txt$/i.test(e.name)) out.push(p); } return out; }
|
||||
function colSpans(sep) { const cols = []; let m; const re = /=+/g; while ((m = re.exec(sep))) cols.push([m.index, m.index + m[0].length]); return cols; }
|
||||
|
||||
// fround + toFixed(decimal-code), value start at index 4 — must match formatMeasuredExact.
|
||||
function fmt(statusStr) {
|
||||
if (!statusStr || statusStr.length <= 4) return null;
|
||||
const decimalDigit = statusStr[statusStr.length - 1];
|
||||
const valueStr = statusStr.substring(4, statusStr.length - 1).trim();
|
||||
const parsed = parseFloat(valueStr);
|
||||
if (isNaN(parsed)) return valueStr;
|
||||
const v = Math.fround(parsed);
|
||||
const d = parseInt(decimalDigit, 10);
|
||||
return isNaN(d) ? v.toFixed(1) : v.toFixed(d);
|
||||
}
|
||||
|
||||
// Parse the staged Final-Test section -> ordered list of spec-bearing printed values.
|
||||
function stagedPrintedValues(t) {
|
||||
const L = t.replace(/\r\n/g, '\n').split('\n');
|
||||
const fi = L.findIndex(l => /FINAL TEST RESULTS/.test(l)); if (fi < 0) return null;
|
||||
let hi = -1; for (let i = fi + 1; i < L.length; i++) { if (/Parameter\s+Measured/.test(L[i])) { hi = i; break; } } if (hi < 0) return null;
|
||||
const sep = L[hi + 1] || ''; const cols = colSpans(sep); if (cols.length < 4) return null;
|
||||
const [pc, mc, sc, stc] = cols;
|
||||
const vals = [];
|
||||
for (let i = hi + 2; i < L.length; i++) {
|
||||
const l = L[i];
|
||||
if (/Check List|^\s*_{5,}/.test(l)) break;
|
||||
if (!l.trim()) continue;
|
||||
if (/^\s*Standard output load/i.test(l)) continue;
|
||||
const measured = (l.slice(mc[0], sc[0]) || '').trim();
|
||||
// spec column ONLY (cols 48..69) — not the trailing Status column (PASS),
|
||||
// so empty-spec rows (240VAC Withstand / Hi-Pot) are correctly skipped.
|
||||
const spec = (l.slice(sc[0], stc[0]) || '').trim();
|
||||
if (!spec) continue;
|
||||
const v = measured.split(/\s+/)[0]; // strip trailing unit
|
||||
if (v === '') return null; // a spec row with no printed value -> can't use this unit
|
||||
vals.push(v);
|
||||
}
|
||||
return vals;
|
||||
}
|
||||
|
||||
// Greedy in-order match printed values -> absolute statusEntries indices.
|
||||
function greedyMap(printed, statusEntries) {
|
||||
const map = []; let j = 0;
|
||||
for (const pv of printed) {
|
||||
let found = -1;
|
||||
for (let k = j; k < statusEntries.length; k++) {
|
||||
if (fmt(statusEntries[k]) === pv) { found = k; break; }
|
||||
}
|
||||
if (found < 0) return null;
|
||||
map.push(found); j = found + 1;
|
||||
}
|
||||
return map;
|
||||
}
|
||||
|
||||
// Does this slotMap reproduce a unit's printed values exactly?
|
||||
function mapMatches(map, printed, statusEntries) {
|
||||
if (map.length !== printed.length) return false;
|
||||
for (let s = 0; s < map.length; s++) {
|
||||
if (fmt(statusEntries[map[s]]) !== printed[s]) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
(async () => {
|
||||
const onlyModels = process.argv.slice(2).filter(a => !a.startsWith('--'));
|
||||
const tpl = JSON.parse(fs.readFileSync(OUT, 'utf8'));
|
||||
// index staged DSCA files by model
|
||||
const files = walk(STAGE, []);
|
||||
const byModel = {};
|
||||
for (const f of files) {
|
||||
let t; try { t = fs.readFileSync(f, 'utf8'); } catch { continue; }
|
||||
const model = (t.match(/^\s*Model:\s*(\S+)/m) || [])[1] || '';
|
||||
if (!/^DSCA/i.test(model)) continue;
|
||||
const sn = (t.match(/^\s*SN:\s*(\S+)/m) || [])[1] || '';
|
||||
if (!sn) continue;
|
||||
(byModel[model.trim()] = byModel[model.trim()] || []).push({ f, sn: sn.trim(), text: t });
|
||||
}
|
||||
|
||||
let derived = 0, failed = [];
|
||||
const UNITS_PER_MODEL = 8, MAX_SAMPLES = 48; // bounds DB lookups per signature group
|
||||
|
||||
// Seeds = the ambiguous models to solve (args), or all models if none given.
|
||||
const seeds = new Set(onlyModels.length ? onlyModels.filter(m => tpl[m]) : Object.keys(tpl));
|
||||
|
||||
// Group ALL models by identical row-name signature. Same printed layout => same
|
||||
// canonical-slot mapping, so one slotMap serves the whole group; pooling units
|
||||
// across the group lets siblings disambiguate duplicate values (e.g. a unit where
|
||||
// 5mA != 50mA linearity forces the correct slot). Only process groups that contain
|
||||
// a seed, so a targeted run touches only the relevant families.
|
||||
const sigOf = (m) => tpl[m].rows.map(r => r.name).join('|');
|
||||
const groups = {};
|
||||
for (const model of Object.keys(tpl)) { (groups[sigOf(model)] = groups[sigOf(model)] || []).push(model); }
|
||||
|
||||
for (const models of Object.values(groups)) {
|
||||
if (![...models].some(m => seeds.has(m))) continue;
|
||||
const specRowCount = tpl[models[0]].rows.filter(r => (r.spec || '').trim()).length;
|
||||
const samples = [];
|
||||
for (const model of models) {
|
||||
if (samples.length >= MAX_SAMPLES) break;
|
||||
const units = (byModel[model] || []).slice(0, UNITS_PER_MODEL);
|
||||
for (const u of units) {
|
||||
if (samples.length >= MAX_SAMPLES) break;
|
||||
let row = await db.queryOne('SELECT raw_data FROM test_records WHERE serial_number=$1 AND model_number=$2 LIMIT 1', [u.sn, model]);
|
||||
if (!row) row = await db.queryOne('SELECT raw_data FROM test_records WHERE raw_serial_number=$1 AND model_number=$2 LIMIT 1', [u.sn, model]);
|
||||
if (!row || !row.raw_data) continue;
|
||||
const printed = stagedPrintedValues(u.text);
|
||||
if (!printed || printed.length !== specRowCount) continue;
|
||||
const p = dse.parseRawData(row.raw_data, 'DSCA');
|
||||
if (!p) continue;
|
||||
samples.push({ printed, status: p.statusEntries });
|
||||
}
|
||||
}
|
||||
if (!samples.length) continue;
|
||||
|
||||
// Candidate slotMaps = greedy map from each sample; the TRUE map reproduces the
|
||||
// most units (a duplicate-confused map fails where the duplicated slots differ;
|
||||
// retest-vintage units fail every map and are ignored).
|
||||
const cands = new Map();
|
||||
for (const s of samples) { const m = greedyMap(s.printed, s.status); if (m) cands.set(m.join(','), m); }
|
||||
let best = null, bestScore = -1;
|
||||
for (const m of cands.values()) {
|
||||
let score = 0;
|
||||
for (const s of samples) if (mapMatches(m, s.printed, s.status)) score++;
|
||||
if (score > bestScore) { bestScore = score; best = m; }
|
||||
}
|
||||
const ratio = bestScore / samples.length;
|
||||
const accept = best && (samples.length === 1 ? bestScore === 1 : (ratio >= 0.6 && bestScore >= 2));
|
||||
if (accept) {
|
||||
// Apply to every model in the group. The renderer only consults slotMap when
|
||||
// the sequential value-zip fails (value count != spec-row count), so clean
|
||||
// models keep their current path and only ambiguous ones use the map.
|
||||
for (const model of models) tpl[model].slotMap = best;
|
||||
derived += models.length;
|
||||
console.log(' [' + models.length + '] ' + models[0].padEnd(13) + ' slotMap=[' + best.join(',') + '] matched ' + bestScore + '/' + samples.length + ' units' + (models.length > 1 ? ' (+' + (models.length - 1) + ' siblings)' : ''));
|
||||
} else if (best) {
|
||||
failed.push(models.join('/') + '(best ' + bestScore + '/' + samples.length + ')');
|
||||
}
|
||||
}
|
||||
fs.writeFileSync(OUT, JSON.stringify(tpl));
|
||||
console.log('\nderived slotMaps: ' + derived);
|
||||
if (failed.length) console.log('no clean match (left as-is): ' + failed.join(', '));
|
||||
await db.close();
|
||||
})().catch(e => { console.error('ERR', e.message, e.stack); process.exit(1); });
|
||||
@@ -15,16 +15,23 @@ function extract(t) {
|
||||
const cols = colSpans(sep); if (cols.length < 4) return null;
|
||||
const [pc, mc, sc, stc] = cols;
|
||||
const rows = [];
|
||||
let loadNote = null;
|
||||
for (let i = hi + 2; i < lines.length; i++) {
|
||||
const l = lines[i];
|
||||
if (/Check List|^\s*_{5,}/.test(l)) break;
|
||||
if (!l.trim()) continue;
|
||||
// The "Standard output load for test is ... ohms." line is a footer note, not
|
||||
// a parameter row — it spans past the name column so column-slicing truncates
|
||||
// it ("Standard output load for te"). Capture the full line as loadNote and
|
||||
// keep it out of rows; the renderer emits it (before the footer underline)
|
||||
// only for models whose staged original actually printed it.
|
||||
if (/^Standard output load/i.test(l.trim())) { loadNote = l.trim(); continue; }
|
||||
const name = (l.slice(pc[0], mc[0]) || '').trim();
|
||||
const spec = (l.slice(sc[0], stc[0]) || '').trim();
|
||||
if (!name && !spec) continue;
|
||||
rows.push({ name, spec });
|
||||
}
|
||||
return { accOut, rows };
|
||||
return { accOut, rows, loadNote };
|
||||
}
|
||||
(async () => {
|
||||
const files = walk(STAGE, []);
|
||||
@@ -40,7 +47,7 @@ function extract(t) {
|
||||
}
|
||||
const models = Object.keys(byModel).sort();
|
||||
console.log('DSCA models templated: ' + models.length);
|
||||
const out = {}; for (const m of models) out[m] = { accOut: byModel[m].accOut, rows: byModel[m].rows };
|
||||
const out = {}; for (const m of models) { out[m] = { accOut: byModel[m].accOut, rows: byModel[m].rows }; if (byModel[m].loadNote) out[m].loadNote = byModel[m].loadNote; }
|
||||
fs.writeFileSync(OUT, JSON.stringify(out));
|
||||
console.log('wrote ' + OUT + ' (' + fs.statSync(OUT).size + ' bytes)');
|
||||
const rc = {}; for (const m of models) { const n = byModel[m].rows.length; rc[n] = (rc[n] || 0) + 1; }
|
||||
|
||||
123
projects/dataforth-dos/tools/mine-hoffman-dsca.py
Normal file
123
projects/dataforth-dos/tools/mine-hoffman-dsca.py
Normal file
@@ -0,0 +1,123 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Mine per-model DSCA33/DSCA45 Final-Test templates from the ORIGINAL certs stored
|
||||
on Dataforth's Hoffman API (the spec files lost in the cryptolocker event are
|
||||
recoverable here because the original software published these before the wipe).
|
||||
|
||||
Input : a JSON map [{"m": model, "s": serial}, ...] of UPLOADED serials.
|
||||
Output: dsca33-45-templates.json (schema-compatible with dsca-templates.json:
|
||||
{ model: { "accOut": "...", "rows": [ {"name","spec"}, ... ] } })
|
||||
+ a human report on stdout.
|
||||
|
||||
Same extraction as the STAGE-1 extractor: the '===' rule under the Final-Test
|
||||
"Parameter ... Measured" header gives exact column spans; name = Parameter col,
|
||||
spec = Specification col. Keeps the richest sheet (most rows) per model.
|
||||
"""
|
||||
import json, re, sys, time, urllib.request, urllib.parse, os
|
||||
|
||||
TOKEN_URL = "https://login.dataforth.com/connect/token"
|
||||
API_BASE = "https://www.dataforth.com"
|
||||
CID, CSEC, SCOPE = "dataforth.onprem.sync", "Trxvwee2234-Awer8723-2", "dataforth.web"
|
||||
|
||||
def get_token():
|
||||
body = urllib.parse.urlencode({
|
||||
"grant_type": "client_credentials", "client_id": CID,
|
||||
"client_secret": CSEC, "scope": SCOPE}).encode()
|
||||
req = urllib.request.Request(TOKEN_URL, body,
|
||||
{"Content-Type": "application/x-www-form-urlencoded"})
|
||||
return json.loads(urllib.request.urlopen(req, timeout=30).read())["access_token"]
|
||||
|
||||
def get_cert(serial, tok):
|
||||
url = f"{API_BASE}/api/v1/TestReportDataFiles/{urllib.parse.quote(serial)}"
|
||||
req = urllib.request.Request(url, headers={"Authorization": f"Bearer {tok}"})
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=30) as r:
|
||||
return json.loads(r.read())
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code == 404: return None
|
||||
raise
|
||||
|
||||
def col_spans(sep):
|
||||
return [(m.start(), m.end()) for m in re.finditer(r"=+", sep)]
|
||||
|
||||
def extract(t):
|
||||
lines = t.replace("\r\n", "\n").split("\n")
|
||||
ahi = next((i for i, l in enumerate(lines)
|
||||
if "Error (%)" in l and "Status" in l), -1)
|
||||
acc_hdr = lines[ahi] if ahi >= 0 else ""
|
||||
# capture the verbatim 2-line accuracy header (super-header + column line) so
|
||||
# AD2 can reproduce the model-specific input label + VDC/mADC/Hz headers exactly
|
||||
acc_header = [lines[ahi - 1].rstrip(), lines[ahi].rstrip()] if ahi > 0 else []
|
||||
m = re.search(r"Output \([^)]*\)|Vout \([^)]*\)", acc_hdr)
|
||||
acc_out = m.group(0) if m else "?"
|
||||
fi = next((i for i, l in enumerate(lines) if "FINAL TEST RESULTS" in l), -1)
|
||||
if fi < 0: return None
|
||||
hi = next((i for i in range(fi + 1, len(lines))
|
||||
if re.search(r"Parameter\s+Measured", lines[i])), -1)
|
||||
if hi < 0: return None
|
||||
sep = lines[hi + 1] if hi + 1 < len(lines) else ""
|
||||
if "=" not in sep: return None
|
||||
cols = col_spans(sep)
|
||||
if len(cols) < 4: return None
|
||||
pc, mc, sc, stc = cols[0], cols[1], cols[2], cols[3]
|
||||
rows = []
|
||||
for i in range(hi + 2, len(lines)):
|
||||
l = lines[i]
|
||||
if re.search(r"Check List|^\s*_{5,}", l): break
|
||||
if not l.strip(): continue
|
||||
name = l[pc[0]:mc[0]].strip()
|
||||
spec = l[sc[0]:stc[0]].strip()
|
||||
if not name and not spec: continue
|
||||
rows.append({"name": name, "spec": spec})
|
||||
return {"accOut": acc_out, "rows": rows, "accHdr": acc_hdr.strip(),
|
||||
"accHeader": acc_header}
|
||||
|
||||
def main():
|
||||
mp = json.load(open(sys.argv[1]))
|
||||
outpath = sys.argv[2]
|
||||
tok = get_token()
|
||||
by_model = {} # model -> best {accOut, rows, accHdr, serial}
|
||||
meta = {} # model -> diagnostics
|
||||
missing = []
|
||||
for row in mp:
|
||||
model, serial = row["m"], row["s"]
|
||||
cert = get_cert(serial, tok)
|
||||
if not cert or not cert.get("Content"):
|
||||
missing.append((model, serial)); continue
|
||||
tpl = extract(cert["Content"])
|
||||
if not tpl:
|
||||
meta.setdefault(model, {}).setdefault("noextract", []).append(serial); continue
|
||||
cur = by_model.get(model)
|
||||
if not cur or len(tpl["rows"]) > len(cur["rows"]):
|
||||
tpl["serial"] = serial
|
||||
by_model[model] = tpl
|
||||
# build schema-compatible output
|
||||
out = {}
|
||||
for model in sorted(by_model):
|
||||
t = by_model[model]
|
||||
out[model] = {"accOut": t["accOut"], "accHeader": t["accHeader"],
|
||||
"rows": t["rows"], "_srcSerial": t["serial"]}
|
||||
with open(outpath, "w") as f:
|
||||
json.dump(out, f, indent=0)
|
||||
# report
|
||||
fams = {}
|
||||
print(f"=== Mined {len(out)} models from Hoffman -> {outpath} ===\n")
|
||||
print(f"{'MODEL':<14} {'rows':>4} {'accOut':<16} src-serial accuracy-header")
|
||||
for model in sorted(out):
|
||||
t = by_model[model]
|
||||
fam = model.split("-")[0]
|
||||
fams[fam] = fams.get(fam, 0) + 1
|
||||
flag = " <-- LOW" if len(t["rows"]) < 3 else ""
|
||||
print(f"{model:<14} {len(t['rows']):>4} {t['accOut']:<16} {t['serial']:<11} {t['accHdr'][:60]}{flag}")
|
||||
print("\nper-family models mined:", dict(fams))
|
||||
distinct_accout = sorted(set(o["accOut"] for o in out.values()))
|
||||
print("distinct accOut tokens:", distinct_accout)
|
||||
if missing:
|
||||
print(f"\n[WARN] {len(missing)} serials returned 404 (not on Hoffman):",
|
||||
missing[:10], "..." if len(missing) > 10 else "")
|
||||
no_tpl = [m for m in {r['m'] for r in mp} if m not in out]
|
||||
if no_tpl:
|
||||
print(f"\n[WARN] models with NO usable template ({len(no_tpl)}):", no_tpl)
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
71
projects/dataforth-dos/tools/preview-proxy.py
Normal file
71
projects/dataforth-dos/tools/preview-proxy.py
Normal file
@@ -0,0 +1,71 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Same-origin preview proxy for the testdatadb front-end.
|
||||
|
||||
Serves the static prototype from ROOT and reverse-proxies /api/* to the live
|
||||
AD2 testdatadb server, so the app AND the cert iframe share one origin
|
||||
(http://127.0.0.1:PORT). That lets the same-origin-only cert styling/fit logic
|
||||
(styleCert/fitCert read iframe.contentDocument) actually run during preview —
|
||||
which it can't when the iframe is loaded cross-origin straight from AD2.
|
||||
|
||||
Usage: python preview-proxy.py <port> <root-dir> [target]
|
||||
"""
|
||||
import http.server, socketserver, urllib.request, urllib.error, os, sys
|
||||
|
||||
PORT = int(sys.argv[1])
|
||||
ROOT = os.path.abspath(sys.argv[2]) if len(sys.argv) > 2 else "."
|
||||
TARGET = sys.argv[3] if len(sys.argv) > 3 else "http://192.168.0.6:3000"
|
||||
|
||||
class H(http.server.BaseHTTPRequestHandler):
|
||||
def do_GET(self):
|
||||
if self.path.startswith("/api/"):
|
||||
try:
|
||||
with urllib.request.urlopen(TARGET + self.path, timeout=30) as r:
|
||||
body = r.read(); ct = r.headers.get("Content-Type", "application/octet-stream")
|
||||
self._send(200, ct, body)
|
||||
except urllib.error.HTTPError as e:
|
||||
self._send(e.code, "application/json", e.read())
|
||||
except Exception as e:
|
||||
self._send(502, "text/plain", str(e).encode())
|
||||
else:
|
||||
p = self.path.split("?")[0]
|
||||
p = "/index.html" if p == "/" else p
|
||||
fp = os.path.join(ROOT, p.lstrip("/"))
|
||||
if os.path.isfile(fp):
|
||||
ct = "text/html; charset=utf-8" if fp.endswith(".html") else "application/octet-stream"
|
||||
self._send(200, ct, open(fp, "rb").read())
|
||||
else:
|
||||
self._send(404, "text/plain", b"not found")
|
||||
|
||||
def do_POST(self):
|
||||
if self.path.startswith("/api/"):
|
||||
n = int(self.headers.get("Content-Length", 0))
|
||||
body = self.rfile.read(n) if n else b""
|
||||
req = urllib.request.Request(
|
||||
TARGET + self.path, data=body, method="POST",
|
||||
headers={"Content-Type": self.headers.get("Content-Type", "application/json")})
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=120) as r:
|
||||
self._send(200, r.headers.get("Content-Type", "application/json"), r.read())
|
||||
except urllib.error.HTTPError as e:
|
||||
self._send(e.code, "application/json", e.read())
|
||||
except Exception as e:
|
||||
self._send(502, "text/plain", str(e).encode())
|
||||
else:
|
||||
self._send(404, "text/plain", b"not found")
|
||||
|
||||
def _send(self, code, ct, body):
|
||||
self.send_response(code)
|
||||
self.send_header("Content-Type", ct)
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.send_header("Cache-Control", "no-store")
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def log_message(self, *a): pass
|
||||
|
||||
class S(socketserver.ThreadingTCPServer):
|
||||
allow_reuse_address = True
|
||||
|
||||
print(f"preview proxy: http://127.0.0.1:{PORT} root={ROOT} -> {TARGET}")
|
||||
S(("127.0.0.1", PORT), H).serve_forever()
|
||||
27
projects/dataforth-dos/tools/push-clean68.js
Normal file
27
projects/dataforth-dos/tools/push-clean68.js
Normal file
@@ -0,0 +1,27 @@
|
||||
// Fix 2 publish — re-push the 68 STAGE-3 Final-Test-clean DSCA models to Hoffman.
|
||||
// Idempotent (Updated/Unchanged/Created); renderContent uses the new template;
|
||||
// null renders (count-guard) + FAIL certs + unregistered models auto-skip.
|
||||
const db = require('./database/db');
|
||||
const { uploadBySerialNumbers } = require('./database/upload-to-api');
|
||||
const clean = require('./_clean-models.json');
|
||||
|
||||
(async () => {
|
||||
const ph = clean.map((_, i) => '$' + (i + 1)).join(',');
|
||||
const rows = await db.query(
|
||||
`SELECT serial_number FROM test_records WHERE overall_result='PASS' AND model_number IN (${ph}) ORDER BY serial_number`,
|
||||
clean,
|
||||
);
|
||||
const sns = rows.map(r => r.serial_number);
|
||||
console.log(`[PUSH] ${sns.length} PASS serials across ${clean.length} clean models`);
|
||||
const CHUNK = 1000;
|
||||
const tot = { created: 0, updated: 0, unchanged: 0, errors: 0, skipped: 0 };
|
||||
for (let i = 0; i < sns.length; i += CHUNK) {
|
||||
const chunk = sns.slice(i, i + CHUNK);
|
||||
const r = await uploadBySerialNumbers(chunk);
|
||||
for (const k of Object.keys(tot)) tot[k] += r[k] || 0;
|
||||
console.log(`[PUSH] ${Math.min(i + CHUNK, sns.length)}/${sns.length} cumulative ` +
|
||||
`created=${tot.created} updated=${tot.updated} unchanged=${tot.unchanged} errors=${tot.errors} skipped=${tot.skipped}`);
|
||||
}
|
||||
console.log('[PUSH] COMPLETE ' + JSON.stringify(tot));
|
||||
await db.close();
|
||||
})().catch(e => { console.error('ERR', e.message, e.stack); process.exit(1); });
|
||||
171
projects/dataforth-dos/tools/validate-dsca-stage3.js
Normal file
171
projects/dataforth-dos/tools/validate-dsca-stage3.js
Normal file
@@ -0,0 +1,171 @@
|
||||
// Fix 2 STAGE 3 — per-subtype byte-validation of DSCA renders vs staged originals.
|
||||
// For every staged DSCA .TXT we have ground truth for, look up the DB record,
|
||||
// render it through the live render path, and content-normalize-compare the two.
|
||||
// Grouped by model (layout). Whitespace is collapsed per line (column spacing is
|
||||
// the deferred cosmetic gap) so the compare tests CONTENT — names, values, specs,
|
||||
// statuses — not pixel alignment. Read-only; no DB writes, no Hoffman push.
|
||||
//
|
||||
// Usage: node _validate_dsca_stage3.js [--limit-per-model N] [--report path]
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const db = require('./database/db');
|
||||
const { renderContent } = require('./database/render-datasheet');
|
||||
|
||||
const STAGE = 'C:/Shares/test/STAGE';
|
||||
const args = process.argv.slice(2);
|
||||
const LIMIT = (() => { const i = args.indexOf('--limit-per-model'); return i >= 0 ? parseInt(args[i + 1], 10) : Infinity; })();
|
||||
const REPORT = (() => { const i = args.indexOf('--report'); return i >= 0 ? args[i + 1] : 'C:/Shares/testdatadb/_dsca-stage3-report.txt'; })();
|
||||
|
||||
function walk(d, out) {
|
||||
let it = [];
|
||||
try { it = fs.readdirSync(d, { withFileTypes: true }); } catch { return out; }
|
||||
for (const e of it) {
|
||||
const p = path.join(d, e.name);
|
||||
if (e.isDirectory()) walk(p, out);
|
||||
else if (/\.txt$/i.test(e.name)) out.push(p);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
// Content-normalize a single line: trim + collapse whitespace. Rule lines (runs
|
||||
// of = - ~ _ separated by spaces) canonicalize to <RULE> so the deferred cosmetic
|
||||
// dash/equal-count differences don't register as content diffs.
|
||||
function normLine(l) {
|
||||
const t = l.trim();
|
||||
if (t.length && /^[=~_\- ]+$/.test(t) && /[=~_\-]/.test(t)) return '<RULE>';
|
||||
return t.replace(/\s+/g, ' ');
|
||||
}
|
||||
function norm(s) {
|
||||
return s.replace(/\r/g, '').split('\n').map(normLine).filter(l => l.length > 0);
|
||||
}
|
||||
|
||||
// Extract the FINAL TEST RESULTS section: from its header to just before the
|
||||
// footer underline (___). This is the Fix 2 deliverable; compared content-strict.
|
||||
function finalTestLines(s) {
|
||||
const L = s.replace(/\r/g, '').split('\n');
|
||||
const fi = L.findIndex(l => /FINAL TEST RESULTS/.test(l));
|
||||
if (fi < 0) return [];
|
||||
const out = [];
|
||||
for (let i = fi; i < L.length; i++) {
|
||||
const t = L[i].trim();
|
||||
if (i > fi && /^_{5,}$/.test(t)) break; // footer underline
|
||||
if (/It is hereby certified/.test(t)) break;
|
||||
out.push(normLine(L[i]));
|
||||
}
|
||||
return out.filter(l => l.length > 0);
|
||||
}
|
||||
|
||||
// Accuracy section lines (informational — spacing + any calc rounding live here).
|
||||
function accuracyLines(s) {
|
||||
const L = s.replace(/\r/g, '').split('\n');
|
||||
const ai = L.findIndex(l => /ACCURACY TEST/.test(l));
|
||||
if (ai < 0) return [];
|
||||
const fi = L.findIndex(l => /FINAL TEST RESULTS/.test(l));
|
||||
const end = fi < 0 ? L.length : fi;
|
||||
return L.slice(ai, end).map(normLine).filter(l => l.length > 0);
|
||||
}
|
||||
|
||||
(async () => {
|
||||
const files = walk(STAGE, []);
|
||||
// index staged DSCA files: model + SN + text
|
||||
const staged = [];
|
||||
for (const f of files) {
|
||||
let t; try { t = fs.readFileSync(f, 'utf8'); } catch { continue; }
|
||||
const model = (t.match(/^\s*Model:\s*(\S+)/m) || [])[1] || '';
|
||||
if (!/^DSCA/i.test(model)) continue;
|
||||
const sn = (t.match(/^\s*SN:\s*(\S+)/m) || [])[1] || '';
|
||||
if (!sn) continue;
|
||||
staged.push({ f, model: model.trim(), sn: sn.trim(), text: t });
|
||||
}
|
||||
console.log(`staged DSCA originals: ${staged.length}`);
|
||||
|
||||
const byModel = {}; // model -> tallies
|
||||
function rec(model) {
|
||||
if (!byModel[model]) byModel[model] = { compared: 0, ftMatch: 0, ftMismatch: 0, accMismatch: 0, noRecord: 0, notRendered: 0, samples: [] };
|
||||
return byModel[model];
|
||||
}
|
||||
function firstDiff(a, b) {
|
||||
const max = Math.max(a.length, b.length);
|
||||
for (let i = 0; i < max; i++) if (a[i] !== b[i]) return i;
|
||||
return -1;
|
||||
}
|
||||
|
||||
const seenPerModel = {};
|
||||
for (const s of staged) {
|
||||
seenPerModel[s.model] = (seenPerModel[s.model] || 0) + 1;
|
||||
if (seenPerModel[s.model] > LIMIT) continue;
|
||||
const r = rec(s.model);
|
||||
let row = await db.queryOne('SELECT * FROM test_records WHERE serial_number=$1 AND model_number=$2 LIMIT 1', [s.sn, s.model]);
|
||||
if (!row) row = await db.queryOne('SELECT * FROM test_records WHERE raw_serial_number=$1 AND model_number=$2 LIMIT 1', [s.sn, s.model]);
|
||||
if (!row) { r.noRecord++; continue; }
|
||||
let rendered;
|
||||
try { rendered = renderContent(row); } catch (e) { rendered = null; }
|
||||
if (!rendered) { r.notRendered++; continue; }
|
||||
r.compared++;
|
||||
// GATE: Final-Test section content must match exactly (rules canonicalized).
|
||||
const ftA = finalTestLines(rendered), ftB = finalTestLines(s.text);
|
||||
const fd = firstDiff(ftA, ftB);
|
||||
if (fd === -1) r.ftMatch++;
|
||||
else {
|
||||
r.ftMismatch++;
|
||||
if (r.samples.length < 3) r.samples.push({ sn: s.sn, line: fd, render: ftA[fd], golden: ftB[fd] });
|
||||
}
|
||||
// INFO: accuracy section (deferred cosmetic spacing + any calc rounding).
|
||||
const acA = accuracyLines(rendered), acB = accuracyLines(s.text);
|
||||
if (firstDiff(acA, acB) !== -1) r.accMismatch++;
|
||||
}
|
||||
|
||||
// report
|
||||
const models = Object.keys(byModel).sort();
|
||||
let totC = 0, totFM = 0, totFMM = 0, totAcc = 0, totNR = 0, totNRn = 0, cleanModels = 0;
|
||||
const ftDirty = [];
|
||||
for (const m of models) {
|
||||
const x = byModel[m];
|
||||
totC += x.compared; totFM += x.ftMatch; totFMM += x.ftMismatch; totAcc += x.accMismatch;
|
||||
totNR += x.noRecord; totNRn += x.notRendered;
|
||||
if (x.compared > 0 && x.ftMismatch === 0) cleanModels++;
|
||||
if (x.ftMismatch > 0) ftDirty.push(m);
|
||||
}
|
||||
const out = [];
|
||||
out.push('Fix 2 STAGE 3 — DSCA Final-Test render vs staged-original content validation');
|
||||
out.push('GATE = FINAL TEST RESULTS section, content-strict (rule lines canonicalized,');
|
||||
out.push('whitespace collapsed). Accuracy-section diffs reported separately (deferred');
|
||||
out.push('cosmetic spacing + any pre-existing calc rounding — NOT a Fix 2 gate).');
|
||||
out.push('Corpus: ' + staged.length + ' staged DSCA originals across ' + models.length + ' models.');
|
||||
out.push('='.repeat(78));
|
||||
out.push('');
|
||||
out.push('SUMMARY');
|
||||
out.push(' models with staged originals: ' + models.length);
|
||||
out.push(' models FINAL-TEST CLEAN (>=1 compared, 0 mismatch): ' + cleanModels);
|
||||
out.push(' models with FINAL-TEST mismatches: ' + ftDirty.length);
|
||||
out.push(' certs compared: ' + totC);
|
||||
out.push(' Final-Test match: ' + totFM);
|
||||
out.push(' Final-Test mismatch: ' + totFMM);
|
||||
out.push(' (certs with accuracy-section diffs: ' + totAcc + ' — informational)');
|
||||
out.push(' staged serials not in DB: ' + totNR);
|
||||
out.push(' in DB but not rendered (skipped/null): ' + totNRn);
|
||||
out.push('');
|
||||
out.push('MODELS WITH FINAL-TEST CONTENT MISMATCHES (investigate before re-push):');
|
||||
if (!ftDirty.length) out.push(' (none — Final-Test renders are content-clean for all compared models)');
|
||||
for (const m of ftDirty) {
|
||||
const x = byModel[m];
|
||||
out.push(' ' + m + ' compared=' + x.compared + ' ftMatch=' + x.ftMatch + ' ftMismatch=' + x.ftMismatch);
|
||||
for (const s of x.samples) {
|
||||
out.push(' [finaltest L' + s.line + '] SN ' + s.sn);
|
||||
out.push(' render: ' + JSON.stringify(s.render));
|
||||
out.push(' golden: ' + JSON.stringify(s.golden));
|
||||
}
|
||||
}
|
||||
out.push('');
|
||||
out.push('FINAL-TEST CLEAN MODELS (' + cleanModels + '):');
|
||||
out.push(' ' + models.filter(m => byModel[m].compared > 0 && byModel[m].ftMismatch === 0).join(', '));
|
||||
out.push('');
|
||||
out.push('MODELS WITH NO COMPARABLE CERT (no staged serial in DB, or all skipped/null):');
|
||||
out.push(' ' + models.filter(m => byModel[m].compared === 0).map(m => m + '(' + (byModel[m].notRendered ? 'null' : 'noDBrec') + ')').join(', '));
|
||||
|
||||
const text = out.join('\n');
|
||||
fs.writeFileSync(REPORT, text);
|
||||
console.log(text);
|
||||
console.log('\n[report written] ' + REPORT);
|
||||
await db.close();
|
||||
})().catch(e => { console.error('ERR', e.message, e.stack); process.exit(1); });
|
||||
88
scripts/Configure-TranscriptLogging.ps1
Normal file
88
scripts/Configure-TranscriptLogging.ps1
Normal file
@@ -0,0 +1,88 @@
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Configures PowerShell transcript logging for remote sessions.
|
||||
|
||||
.DESCRIPTION
|
||||
Enables comprehensive transcript logging via registry settings,
|
||||
creates the logging directory with proper permissions, and sets up
|
||||
automatic log rotation.
|
||||
|
||||
.NOTES
|
||||
Author: ClaudeTools Automation
|
||||
Version: 1.0
|
||||
Run as Administrator
|
||||
#>
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
$transcriptPath = "C:\ClaudeTools\Logs\Transcripts"
|
||||
|
||||
Write-Host "Configuring PowerShell Transcript Logging..." -ForegroundColor Cyan
|
||||
|
||||
# Create transcript directory
|
||||
if (-not (Test-Path $transcriptPath)) {
|
||||
New-Item -ItemType Directory -Path $transcriptPath -Force | Out-Null
|
||||
Write-Host "Created transcript directory: $transcriptPath" -ForegroundColor Green
|
||||
}
|
||||
|
||||
# Set permissions on transcript directory
|
||||
# Administrators: Full Control, SYSTEM: Full Control, Remote Management Users: Read/Write
|
||||
$acl = Get-Acl $transcriptPath
|
||||
$acl.SetAccessRuleProtection($true, $false) # Disable inheritance
|
||||
|
||||
# Add Administrators - Full Control
|
||||
$adminRule = New-Object System.Security.AccessControl.FileSystemAccessRule(
|
||||
"Administrators", "FullControl", "ContainerInherit,ObjectInherit", "None", "Allow"
|
||||
)
|
||||
$acl.AddAccessRule($adminRule)
|
||||
|
||||
# Add SYSTEM - Full Control
|
||||
$systemRule = New-Object System.Security.AccessControl.FileSystemAccessRule(
|
||||
"SYSTEM", "FullControl", "ContainerInherit,ObjectInherit", "None", "Allow"
|
||||
)
|
||||
$acl.AddAccessRule($systemRule)
|
||||
|
||||
# Add Remote Management Users - Modify (so they can write transcripts)
|
||||
$rmRule = New-Object System.Security.AccessControl.FileSystemAccessRule(
|
||||
"Remote Management Users", "Modify", "ContainerInherit,ObjectInherit", "None", "Allow"
|
||||
)
|
||||
$acl.AddAccessRule($rmRule)
|
||||
|
||||
Set-Acl $transcriptPath $acl
|
||||
Write-Host "Set permissions on transcript directory" -ForegroundColor Green
|
||||
|
||||
# Configure PowerShell transcript logging via registry
|
||||
$psPath = "HKLM:\SOFTWARE\Policies\Microsoft\Windows\PowerShell\Transcription"
|
||||
|
||||
if (-not (Test-Path $psPath)) {
|
||||
New-Item -Path $psPath -Force | Out-Null
|
||||
}
|
||||
|
||||
# Enable transcription
|
||||
Set-ItemProperty -Path $psPath -Name "EnableTranscripting" -Value 1 -Type DWord
|
||||
Set-ItemProperty -Path $psPath -Name "EnableInvocationHeader" -Value 1 -Type DWord
|
||||
Set-ItemProperty -Path $psPath -Name "OutputDirectory" -Value $transcriptPath -Type String
|
||||
|
||||
Write-Host "Enabled PowerShell transcription via registry" -ForegroundColor Green
|
||||
|
||||
# Also enable module logging for additional audit trail
|
||||
$modulePath = "HKLM:\SOFTWARE\Policies\Microsoft\Windows\PowerShell\ModuleLogging"
|
||||
if (-not (Test-Path $modulePath)) {
|
||||
New-Item -Path $modulePath -Force | Out-Null
|
||||
}
|
||||
Set-ItemProperty -Path $modulePath -Name "EnableModuleLogging" -Value 1 -Type DWord
|
||||
|
||||
# Enable script block logging
|
||||
$scriptPath = "HKLM:\SOFTWARE\Policies\Microsoft\Windows\PowerShell\ScriptBlockLogging"
|
||||
if (-not (Test-Path $scriptPath)) {
|
||||
New-Item -Path $scriptPath -Force | Out-Null
|
||||
}
|
||||
Set-ItemProperty -Path $scriptPath -Name "EnableScriptBlockLogging" -Value 1 -Type DWord
|
||||
|
||||
Write-Host "Enabled module and script block logging" -ForegroundColor Green
|
||||
|
||||
Write-Host "`nTranscript logging configuration complete!" -ForegroundColor Green
|
||||
Write-Host "Transcripts will be saved to: $transcriptPath"
|
||||
|
||||
# Display current settings
|
||||
Write-Host "`n--- Current Settings ---" -ForegroundColor Yellow
|
||||
Get-ItemProperty -Path $psPath | Select-Object EnableTranscripting, EnableInvocationHeader, OutputDirectory
|
||||
87
scripts/Get-ADComputerReport.ps1
Normal file
87
scripts/Get-ADComputerReport.ps1
Normal file
@@ -0,0 +1,87 @@
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Generates a report of all Active Directory computers.
|
||||
|
||||
.DESCRIPTION
|
||||
This script queries Active Directory for all computer accounts and exports
|
||||
key properties including name, operating system, last logon, and OU location.
|
||||
|
||||
.PARAMETER OutputPath
|
||||
Optional. Path to export CSV report. If not specified, outputs to console.
|
||||
|
||||
.PARAMETER OperatingSystem
|
||||
Optional. Filter by operating system (e.g., "Windows Server*", "*Windows 10*").
|
||||
|
||||
.EXAMPLE
|
||||
.\Get-ADComputerReport.ps1
|
||||
Lists all computers to console.
|
||||
|
||||
.EXAMPLE
|
||||
.\Get-ADComputerReport.ps1 -OperatingSystem "Windows Server*" -OutputPath "C:\ClaudeTools\Logs\servers.csv"
|
||||
Exports all Windows Server computers to CSV.
|
||||
|
||||
.NOTES
|
||||
Author: ClaudeTools Automation
|
||||
Version: 1.0
|
||||
Requires: ActiveDirectory PowerShell module
|
||||
#>
|
||||
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[Parameter(Mandatory=$false)]
|
||||
[string]$OutputPath,
|
||||
|
||||
[Parameter(Mandatory=$false)]
|
||||
[string]$OperatingSystem = "*"
|
||||
)
|
||||
|
||||
# Import AD module
|
||||
Import-Module ActiveDirectory -ErrorAction Stop
|
||||
|
||||
Write-Host "Querying Active Directory computers..." -ForegroundColor Cyan
|
||||
|
||||
# Get computers with properties
|
||||
$computers = Get-ADComputer -Filter "OperatingSystem -like '$OperatingSystem'" -Properties `
|
||||
OperatingSystem,
|
||||
OperatingSystemVersion,
|
||||
LastLogonDate,
|
||||
Created,
|
||||
Enabled,
|
||||
IPv4Address,
|
||||
Description,
|
||||
DistinguishedName |
|
||||
Select-Object `
|
||||
@{N='Name';E={$_.Name}},
|
||||
@{N='OperatingSystem';E={$_.OperatingSystem}},
|
||||
@{N='OSVersion';E={$_.OperatingSystemVersion}},
|
||||
@{N='Enabled';E={$_.Enabled}},
|
||||
@{N='IPv4Address';E={$_.IPv4Address}},
|
||||
@{N='LastLogon';E={$_.LastLogonDate}},
|
||||
@{N='Created';E={$_.Created}},
|
||||
@{N='OU';E={($_.DistinguishedName -split ',',2)[1]}},
|
||||
@{N='Description';E={$_.Description}}
|
||||
|
||||
$computerCount = ($computers | Measure-Object).Count
|
||||
Write-Host "Found $computerCount computers." -ForegroundColor Green
|
||||
|
||||
if ($OutputPath) {
|
||||
$computers | Export-Csv -Path $OutputPath -NoTypeInformation
|
||||
Write-Host "Report exported to: $OutputPath" -ForegroundColor Green
|
||||
} else {
|
||||
$computers | Format-Table -AutoSize
|
||||
}
|
||||
|
||||
# Summary by OS
|
||||
Write-Host "`n--- Operating System Summary ---" -ForegroundColor Yellow
|
||||
$computers | Group-Object OperatingSystem | Sort-Object Count -Descending |
|
||||
Format-Table @{N='Operating System';E={$_.Name}}, Count -AutoSize
|
||||
|
||||
# Summary by status
|
||||
$enabledCount = ($computers | Where-Object { $_.Enabled -eq $true } | Measure-Object).Count
|
||||
$disabledCount = ($computers | Where-Object { $_.Enabled -eq $false } | Measure-Object).Count
|
||||
Write-Host "Enabled: $enabledCount | Disabled: $disabledCount"
|
||||
|
||||
# Stale computers (no logon in 90 days)
|
||||
$staleDate = (Get-Date).AddDays(-90)
|
||||
$staleCount = ($computers | Where-Object { $_.LastLogon -lt $staleDate -or $null -eq $_.LastLogon } | Measure-Object).Count
|
||||
Write-Host "Stale (no logon 90+ days): $staleCount" -ForegroundColor $(if ($staleCount -gt 0) { 'Yellow' } else { 'Green' })
|
||||
92
scripts/Get-ADUserReport.ps1
Normal file
92
scripts/Get-ADUserReport.ps1
Normal file
@@ -0,0 +1,92 @@
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Generates a report of all Active Directory users with key properties.
|
||||
|
||||
.DESCRIPTION
|
||||
This script queries Active Directory for all user accounts and exports
|
||||
key properties including name, email, last logon, account status, and group memberships.
|
||||
|
||||
.PARAMETER OutputPath
|
||||
Optional. Path to export CSV report. If not specified, outputs to console.
|
||||
|
||||
.PARAMETER IncludeDisabled
|
||||
Switch to include disabled accounts in the report.
|
||||
|
||||
.EXAMPLE
|
||||
.\Get-ADUserReport.ps1
|
||||
Lists all enabled users to console.
|
||||
|
||||
.EXAMPLE
|
||||
.\Get-ADUserReport.ps1 -OutputPath "C:\ClaudeTools\Logs\users.csv" -IncludeDisabled
|
||||
Exports all users (including disabled) to CSV file.
|
||||
|
||||
.NOTES
|
||||
Author: ClaudeTools Automation
|
||||
Version: 1.0
|
||||
Requires: ActiveDirectory PowerShell module
|
||||
#>
|
||||
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[Parameter(Mandatory=$false)]
|
||||
[string]$OutputPath,
|
||||
|
||||
[Parameter(Mandatory=$false)]
|
||||
[switch]$IncludeDisabled
|
||||
)
|
||||
|
||||
# Import AD module
|
||||
Import-Module ActiveDirectory -ErrorAction Stop
|
||||
|
||||
Write-Host "Querying Active Directory users..." -ForegroundColor Cyan
|
||||
|
||||
# Build filter
|
||||
$filter = if ($IncludeDisabled) { "*" } else { "Enabled -eq 'True'" }
|
||||
|
||||
# Get users with properties
|
||||
$users = Get-ADUser -Filter $filter -Properties `
|
||||
DisplayName,
|
||||
EmailAddress,
|
||||
Department,
|
||||
Title,
|
||||
Manager,
|
||||
LastLogonDate,
|
||||
PasswordLastSet,
|
||||
PasswordNeverExpires,
|
||||
Enabled,
|
||||
Created,
|
||||
MemberOf,
|
||||
Description |
|
||||
Select-Object `
|
||||
@{N='SamAccountName';E={$_.SamAccountName}},
|
||||
@{N='DisplayName';E={$_.DisplayName}},
|
||||
@{N='Email';E={$_.EmailAddress}},
|
||||
@{N='Department';E={$_.Department}},
|
||||
@{N='Title';E={$_.Title}},
|
||||
@{N='Enabled';E={$_.Enabled}},
|
||||
@{N='LastLogon';E={$_.LastLogonDate}},
|
||||
@{N='PasswordLastSet';E={$_.PasswordLastSet}},
|
||||
@{N='PasswordNeverExpires';E={$_.PasswordNeverExpires}},
|
||||
@{N='Created';E={$_.Created}},
|
||||
@{N='GroupCount';E={($_.MemberOf | Measure-Object).Count}},
|
||||
@{N='Description';E={$_.Description}}
|
||||
|
||||
$userCount = ($users | Measure-Object).Count
|
||||
Write-Host "Found $userCount users." -ForegroundColor Green
|
||||
|
||||
if ($OutputPath) {
|
||||
$users | Export-Csv -Path $OutputPath -NoTypeInformation
|
||||
Write-Host "Report exported to: $OutputPath" -ForegroundColor Green
|
||||
} else {
|
||||
$users | Format-Table -AutoSize
|
||||
}
|
||||
|
||||
# Summary statistics
|
||||
Write-Host "`n--- Summary ---" -ForegroundColor Yellow
|
||||
Write-Host "Total Users: $userCount"
|
||||
$enabledCount = ($users | Where-Object { $_.Enabled -eq $true } | Measure-Object).Count
|
||||
$disabledCount = ($users | Where-Object { $_.Enabled -eq $false } | Measure-Object).Count
|
||||
Write-Host "Enabled: $enabledCount"
|
||||
Write-Host "Disabled: $disabledCount"
|
||||
$neverExpire = ($users | Where-Object { $_.PasswordNeverExpires -eq $true } | Measure-Object).Count
|
||||
Write-Host "Password Never Expires: $neverExpire"
|
||||
111
scripts/Get-GPOStatus.ps1
Normal file
111
scripts/Get-GPOStatus.ps1
Normal file
@@ -0,0 +1,111 @@
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Reports on Group Policy Object status and replication.
|
||||
|
||||
.DESCRIPTION
|
||||
This script checks all GPOs in the domain and reports their status,
|
||||
including version information, links, and replication status between
|
||||
AD and SYSVOL.
|
||||
|
||||
.PARAMETER OutputPath
|
||||
Optional. Path to export CSV report. If not specified, outputs to console.
|
||||
|
||||
.PARAMETER CheckReplication
|
||||
Switch to perform detailed replication check between AD and SYSVOL.
|
||||
|
||||
.EXAMPLE
|
||||
.\Get-GPOStatus.ps1
|
||||
Lists all GPOs with basic status.
|
||||
|
||||
.EXAMPLE
|
||||
.\Get-GPOStatus.ps1 -CheckReplication -OutputPath "C:\ClaudeTools\Logs\gpo-status.csv"
|
||||
Full replication check with CSV export.
|
||||
|
||||
.NOTES
|
||||
Author: ClaudeTools Automation
|
||||
Version: 1.0
|
||||
Requires: GroupPolicy PowerShell module
|
||||
#>
|
||||
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[Parameter(Mandatory=$false)]
|
||||
[string]$OutputPath,
|
||||
|
||||
[Parameter(Mandatory=$false)]
|
||||
[switch]$CheckReplication
|
||||
)
|
||||
|
||||
# Import required modules
|
||||
Import-Module GroupPolicy -ErrorAction Stop
|
||||
Import-Module ActiveDirectory -ErrorAction Stop
|
||||
|
||||
Write-Host "Querying Group Policy Objects..." -ForegroundColor Cyan
|
||||
|
||||
# Get all GPOs
|
||||
$gpos = Get-GPO -All | Select-Object `
|
||||
@{N='Name';E={$_.DisplayName}},
|
||||
@{N='ID';E={$_.Id}},
|
||||
@{N='Status';E={$_.GpoStatus}},
|
||||
@{N='CreationTime';E={$_.CreationTime}},
|
||||
@{N='ModificationTime';E={$_.ModificationTime}},
|
||||
@{N='UserVersion';E={$_.User.DSVersion}},
|
||||
@{N='ComputerVersion';E={$_.Computer.DSVersion}},
|
||||
@{N='WMIFilter';E={$_.WmiFilter.Name}}
|
||||
|
||||
$gpoCount = ($gpos | Measure-Object).Count
|
||||
Write-Host "Found $gpoCount GPOs." -ForegroundColor Green
|
||||
|
||||
# Check GPO links
|
||||
Write-Host "`nChecking GPO links..." -ForegroundColor Cyan
|
||||
$gpoLinks = @()
|
||||
foreach ($gpo in (Get-GPO -All)) {
|
||||
$report = [xml](Get-GPOReport -Guid $gpo.Id -ReportType Xml)
|
||||
$links = $report.GPO.LinksTo.SOMPath
|
||||
|
||||
$gpoLinks += [PSCustomObject]@{
|
||||
Name = $gpo.DisplayName
|
||||
LinkCount = if ($links) { ($links | Measure-Object).Count } else { 0 }
|
||||
Links = if ($links) { $links -join "; " } else { "Not Linked" }
|
||||
}
|
||||
}
|
||||
|
||||
if ($CheckReplication) {
|
||||
Write-Host "`nChecking SYSVOL replication status..." -ForegroundColor Cyan
|
||||
|
||||
$domain = (Get-ADDomain).DNSRoot
|
||||
$dcs = Get-ADDomainController -Filter *
|
||||
|
||||
foreach ($dc in $dcs) {
|
||||
Write-Host " Checking $($dc.HostName)..." -ForegroundColor Gray
|
||||
$sysvolPath = "\\$($dc.HostName)\SYSVOL\$domain\Policies"
|
||||
|
||||
if (Test-Path $sysvolPath) {
|
||||
$sysvolGPOs = Get-ChildItem $sysvolPath -Directory | Where-Object { $_.Name -match '^{' }
|
||||
Write-Host " SYSVOL GPO count: $($sysvolGPOs.Count)" -ForegroundColor Green
|
||||
} else {
|
||||
Write-Host " Unable to access SYSVOL" -ForegroundColor Red
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# Output results
|
||||
if ($OutputPath) {
|
||||
$gpos | Export-Csv -Path $OutputPath -NoTypeInformation
|
||||
Write-Host "`nReport exported to: $OutputPath" -ForegroundColor Green
|
||||
} else {
|
||||
Write-Host "`n--- GPO List ---" -ForegroundColor Yellow
|
||||
$gpos | Format-Table Name, Status, ModificationTime, UserVersion, ComputerVersion -AutoSize
|
||||
|
||||
Write-Host "`n--- GPO Links ---" -ForegroundColor Yellow
|
||||
$gpoLinks | Format-Table Name, LinkCount, Links -AutoSize
|
||||
}
|
||||
|
||||
# Summary
|
||||
Write-Host "`n--- Summary ---" -ForegroundColor Yellow
|
||||
Write-Host "Total GPOs: $gpoCount"
|
||||
$unlinked = ($gpoLinks | Where-Object { $_.LinkCount -eq 0 } | Measure-Object).Count
|
||||
Write-Host "Unlinked GPOs: $unlinked" -ForegroundColor $(if ($unlinked -gt 0) { 'Yellow' } else { 'Green' })
|
||||
|
||||
$disabled = ($gpos | Where-Object { $_.Status -ne 'AllSettingsEnabled' } | Measure-Object).Count
|
||||
Write-Host "Disabled/Partial GPOs: $disabled" -ForegroundColor $(if ($disabled -gt 0) { 'Yellow' } else { 'Green' })
|
||||
173
scripts/Get-ReplicationHealth.ps1
Normal file
173
scripts/Get-ReplicationHealth.ps1
Normal file
@@ -0,0 +1,173 @@
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Checks Active Directory replication health across domain controllers.
|
||||
|
||||
.DESCRIPTION
|
||||
This script performs comprehensive AD replication health checks including
|
||||
replication status, partner connectivity, and identifies any replication failures.
|
||||
|
||||
.PARAMETER OutputPath
|
||||
Optional. Path to export results. If not specified, outputs to console.
|
||||
|
||||
.PARAMETER Detailed
|
||||
Switch to show detailed replication information per DC.
|
||||
|
||||
.EXAMPLE
|
||||
.\Get-ReplicationHealth.ps1
|
||||
Basic replication health check.
|
||||
|
||||
.EXAMPLE
|
||||
.\Get-ReplicationHealth.ps1 -Detailed -OutputPath "C:\ClaudeTools\Logs\repl-health.txt"
|
||||
Detailed check with output to file.
|
||||
|
||||
.NOTES
|
||||
Author: ClaudeTools Automation
|
||||
Version: 1.0
|
||||
Requires: ActiveDirectory PowerShell module, repadmin.exe
|
||||
#>
|
||||
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[Parameter(Mandatory=$false)]
|
||||
[string]$OutputPath,
|
||||
|
||||
[Parameter(Mandatory=$false)]
|
||||
[switch]$Detailed
|
||||
)
|
||||
|
||||
# Import AD module
|
||||
Import-Module ActiveDirectory -ErrorAction Stop
|
||||
|
||||
$output = @()
|
||||
$output += "=" * 60
|
||||
$output += "AD REPLICATION HEALTH REPORT"
|
||||
$output += "Generated: $(Get-Date -Format 'yyyy-MM-dd HH:mm:ss')"
|
||||
$output += "=" * 60
|
||||
|
||||
Write-Host "Checking AD Replication Health..." -ForegroundColor Cyan
|
||||
|
||||
# Get all DCs
|
||||
$dcs = Get-ADDomainController -Filter *
|
||||
$output += "`nDomain Controllers Found: $($dcs.Count)"
|
||||
|
||||
foreach ($dc in $dcs) {
|
||||
$output += "`n--- $($dc.HostName) ---"
|
||||
Write-Host "Checking $($dc.HostName)..." -ForegroundColor Gray
|
||||
}
|
||||
|
||||
# Check replication summary using repadmin
|
||||
$output += "`n" + "=" * 60
|
||||
$output += "REPLICATION SUMMARY (repadmin /replsummary)"
|
||||
$output += "=" * 60
|
||||
|
||||
try {
|
||||
$replSummary = repadmin /replsummary 2>&1
|
||||
$output += $replSummary
|
||||
Write-Host "Replication summary retrieved." -ForegroundColor Green
|
||||
} catch {
|
||||
$output += "ERROR: Unable to run repadmin /replsummary"
|
||||
Write-Host "Error running repadmin" -ForegroundColor Red
|
||||
}
|
||||
|
||||
# Check for replication failures
|
||||
$output += "`n" + "=" * 60
|
||||
$output += "REPLICATION FAILURES (repadmin /showrepl * /errorsonly)"
|
||||
$output += "=" * 60
|
||||
|
||||
try {
|
||||
$replErrors = repadmin /showrepl * /errorsonly 2>&1
|
||||
if ($replErrors -match "error" -or $replErrors -match "fail") {
|
||||
$output += $replErrors
|
||||
Write-Host "Replication ERRORS detected!" -ForegroundColor Red
|
||||
} else {
|
||||
$output += "No replication errors detected."
|
||||
Write-Host "No replication errors." -ForegroundColor Green
|
||||
}
|
||||
} catch {
|
||||
$output += "ERROR: Unable to check replication errors"
|
||||
}
|
||||
|
||||
# Queue length
|
||||
$output += "`n" + "=" * 60
|
||||
$output += "REPLICATION QUEUE (repadmin /queue)"
|
||||
$output += "=" * 60
|
||||
|
||||
try {
|
||||
$replQueue = repadmin /queue 2>&1
|
||||
$output += $replQueue
|
||||
} catch {
|
||||
$output += "ERROR: Unable to check replication queue"
|
||||
}
|
||||
|
||||
if ($Detailed) {
|
||||
$output += "`n" + "=" * 60
|
||||
$output += "DETAILED REPLICATION STATUS (repadmin /showrepl)"
|
||||
$output += "=" * 60
|
||||
|
||||
try {
|
||||
$replDetail = repadmin /showrepl 2>&1
|
||||
$output += $replDetail
|
||||
} catch {
|
||||
$output += "ERROR: Unable to get detailed replication status"
|
||||
}
|
||||
|
||||
# DFSR Health (if applicable)
|
||||
$output += "`n" + "=" * 60
|
||||
$output += "DFSR SYSVOL REPLICATION STATUS"
|
||||
$output += "=" * 60
|
||||
|
||||
try {
|
||||
$dfsrStatus = Get-DfsrMember -ErrorAction SilentlyContinue
|
||||
if ($dfsrStatus) {
|
||||
$output += "DFSR Members:"
|
||||
foreach ($member in $dfsrStatus) {
|
||||
$output += " - $($member.ComputerName): $($member.DomainName)"
|
||||
}
|
||||
} else {
|
||||
$output += "DFSR not configured or FRS in use."
|
||||
}
|
||||
} catch {
|
||||
$output += "Unable to query DFSR status (may be using FRS)"
|
||||
}
|
||||
}
|
||||
|
||||
# AD Database health
|
||||
$output += "`n" + "=" * 60
|
||||
$output += "AD DATABASE INTEGRITY"
|
||||
$output += "=" * 60
|
||||
|
||||
$adDb = Get-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Services\NTDS\Parameters" -ErrorAction SilentlyContinue
|
||||
if ($adDb) {
|
||||
$dbPath = $adDb.'DSA Database file'
|
||||
$logPath = $adDb.'Database log files path'
|
||||
$output += "Database Path: $dbPath"
|
||||
$output += "Log Path: $logPath"
|
||||
|
||||
if (Test-Path $dbPath) {
|
||||
$dbSize = (Get-Item $dbPath).Length / 1MB
|
||||
$output += "Database Size: $([math]::Round($dbSize, 2)) MB"
|
||||
}
|
||||
}
|
||||
|
||||
# Final summary
|
||||
$output += "`n" + "=" * 60
|
||||
$output += "HEALTH CHECK COMPLETE"
|
||||
$output += "=" * 60
|
||||
|
||||
# Output results
|
||||
if ($OutputPath) {
|
||||
$output | Out-File -FilePath $OutputPath -Encoding UTF8
|
||||
Write-Host "`nReport saved to: $OutputPath" -ForegroundColor Green
|
||||
} else {
|
||||
$output | ForEach-Object { Write-Host $_ }
|
||||
}
|
||||
|
||||
# Quick status summary
|
||||
Write-Host "`n--- Quick Status ---" -ForegroundColor Yellow
|
||||
Write-Host "Domain Controllers: $($dcs.Count)"
|
||||
$errorMatch = $replErrors -match "error|fail"
|
||||
if ($errorMatch) {
|
||||
Write-Host "Replication Status: ERRORS DETECTED" -ForegroundColor Red
|
||||
} else {
|
||||
Write-Host "Replication Status: HEALTHY" -ForegroundColor Green
|
||||
}
|
||||
107
scripts/Invoke-LogRotation.ps1
Normal file
107
scripts/Invoke-LogRotation.ps1
Normal file
@@ -0,0 +1,107 @@
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Rotates and cleans up old log files.
|
||||
|
||||
.DESCRIPTION
|
||||
Removes transcript and log files older than the specified retention period.
|
||||
Designed to run as a scheduled task daily.
|
||||
|
||||
.PARAMETER RetentionDays
|
||||
Number of days to retain log files. Default is 30.
|
||||
|
||||
.PARAMETER LogPath
|
||||
Path to the logs directory. Default is C:\ClaudeTools\Logs.
|
||||
|
||||
.PARAMETER WhatIf
|
||||
Shows what would be deleted without actually deleting.
|
||||
|
||||
.EXAMPLE
|
||||
.\Invoke-LogRotation.ps1
|
||||
Removes logs older than 30 days.
|
||||
|
||||
.EXAMPLE
|
||||
.\Invoke-LogRotation.ps1 -RetentionDays 14 -WhatIf
|
||||
Shows what would be deleted with 14-day retention.
|
||||
|
||||
.NOTES
|
||||
Author: ClaudeTools Automation
|
||||
Version: 1.0
|
||||
#>
|
||||
|
||||
[CmdletBinding(SupportsShouldProcess)]
|
||||
param(
|
||||
[Parameter(Mandatory=$false)]
|
||||
[int]$RetentionDays = 30,
|
||||
|
||||
[Parameter(Mandatory=$false)]
|
||||
[string]$LogPath = "C:\ClaudeTools\Logs"
|
||||
)
|
||||
|
||||
$rotationLog = Join-Path $LogPath "rotation.log"
|
||||
$timestamp = Get-Date -Format "yyyy-MM-dd HH:mm:ss"
|
||||
|
||||
function Write-Log {
|
||||
param([string]$Message)
|
||||
$logEntry = "[$timestamp] $Message"
|
||||
Add-Content -Path $rotationLog -Value $logEntry
|
||||
Write-Host $logEntry
|
||||
}
|
||||
|
||||
Write-Log "=== Log Rotation Started ==="
|
||||
Write-Log "Retention Period: $RetentionDays days"
|
||||
Write-Log "Log Path: $LogPath"
|
||||
|
||||
$cutoffDate = (Get-Date).AddDays(-$RetentionDays)
|
||||
$totalDeleted = 0
|
||||
$totalSizeFreed = 0
|
||||
|
||||
# Find and delete old files
|
||||
$oldFiles = Get-ChildItem -Path $LogPath -Recurse -File |
|
||||
Where-Object { $_.LastWriteTime -lt $cutoffDate -and $_.Name -ne "rotation.log" }
|
||||
|
||||
$fileCount = ($oldFiles | Measure-Object).Count
|
||||
Write-Log "Found $fileCount files older than $RetentionDays days"
|
||||
|
||||
foreach ($file in $oldFiles) {
|
||||
$fileSize = $file.Length
|
||||
$filePath = $file.FullName
|
||||
|
||||
if ($PSCmdlet.ShouldProcess($filePath, "Delete")) {
|
||||
try {
|
||||
Remove-Item $filePath -Force
|
||||
$totalDeleted++
|
||||
$totalSizeFreed += $fileSize
|
||||
Write-Log "Deleted: $filePath ($([math]::Round($fileSize/1KB, 2)) KB)"
|
||||
} catch {
|
||||
Write-Log "ERROR deleting $filePath : $_"
|
||||
}
|
||||
} else {
|
||||
Write-Log "WhatIf: Would delete $filePath ($([math]::Round($fileSize/1KB, 2)) KB)"
|
||||
}
|
||||
}
|
||||
|
||||
# Delete empty subdirectories
|
||||
$emptyDirs = Get-ChildItem -Path $LogPath -Directory -Recurse |
|
||||
Where-Object { (Get-ChildItem $_.FullName -Force).Count -eq 0 }
|
||||
|
||||
foreach ($dir in $emptyDirs) {
|
||||
if ($PSCmdlet.ShouldProcess($dir.FullName, "Remove empty directory")) {
|
||||
try {
|
||||
Remove-Item $dir.FullName -Force
|
||||
Write-Log "Removed empty directory: $($dir.FullName)"
|
||||
} catch {
|
||||
Write-Log "ERROR removing directory $($dir.FullName) : $_"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# Summary
|
||||
$sizeMB = [math]::Round($totalSizeFreed / 1MB, 2)
|
||||
Write-Log "=== Rotation Complete ==="
|
||||
Write-Log "Files Deleted: $totalDeleted"
|
||||
Write-Log "Space Freed: $sizeMB MB"
|
||||
|
||||
# Show current disk usage
|
||||
$currentSize = (Get-ChildItem -Path $LogPath -Recurse -File | Measure-Object -Property Length -Sum).Sum
|
||||
$currentSizeMB = [math]::Round($currentSize / 1MB, 2)
|
||||
Write-Log "Current Log Directory Size: $currentSizeMB MB"
|
||||
177
session-logs/2026-04-03-session-ad2.md
Normal file
177
session-logs/2026-04-03-session-ad2.md
Normal file
@@ -0,0 +1,177 @@
|
||||
# Session Log - AD2 - 2026-04-03
|
||||
|
||||
## Session Summary
|
||||
|
||||
Continued test datasheet pipeline work from the 03-27/28/29 session. Major focus on:
|
||||
1. Fixing formatting issues in generated TXT datasheets (column alignment matching QuickBASIC)
|
||||
2. Adding 7B product family support to the exact-match formatter
|
||||
3. Loading additional spec files (5B49_2.DAT for the last missing Quatronix datasheet)
|
||||
4. Work order report ingestion (33K WOs, 63K lines, 2.27M records linked)
|
||||
5. UI improvements (WO search, WO detail popup, View button styled HTML)
|
||||
6. Generating full Dataforth audit document
|
||||
7. Planning and beginning PostgreSQL migration (SQLite hitting 4.4GB, queries timing out)
|
||||
8. Uninstalling SQL Server Express, installing PostgreSQL 18
|
||||
|
||||
## Key Decisions
|
||||
- PostgreSQL over SQL Server Express (no 10GB limit, better FTS, no licensing)
|
||||
- Install PG locally on AD2 (not SAGE-SQL or new VM)
|
||||
- Clean cutover (no parallel run with SQLite)
|
||||
- MSSQL Express uninstalled to free resources
|
||||
|
||||
## Credentials
|
||||
|
||||
### PostgreSQL (newly installed)
|
||||
- Host: localhost (AD2, 192.168.0.6)
|
||||
- Port: 5432
|
||||
- Superuser: postgres / DfPgSQL2026!
|
||||
- App role: testdatadb_app / DfTestDB2026!
|
||||
- Database: testdatadb
|
||||
- Service: postgresql-18 (runs as INTRANET\sysadmin)
|
||||
- Install path: C:\Program Files\PostgreSQL\18\
|
||||
- Data dir: C:\Program Files\PostgreSQL\18\data\
|
||||
|
||||
### TestDataDB (existing)
|
||||
- Service: testdatadb (runs as INTRANET\svc_testdatadb / DfSvcTDB2026!)
|
||||
- URL: http://192.168.0.6:3000
|
||||
- SQLite DB: C:\Shares\testdatadb\database\testdata.db (4.4 GB)
|
||||
|
||||
### Existing (from CLAUDE.md)
|
||||
- AD Sysadmin: INTRANET\sysadmin / Paper123!@#
|
||||
- D2TESTNAS SSH: root@192.168.0.9 / Paper123!@#-nas
|
||||
- Rsync: port 873, user rsync / IQ203s32119
|
||||
- M365 Tenant: 7dfa3ce8-c496-4b51-ab8d-bd3dcd78b584
|
||||
- Gitea: mike@azcomputerguru.com / Gptf*77ttb123!@#-git
|
||||
- Gitea API Token: 9b1da4b79a38ef782268341d25a4b6880572063f
|
||||
|
||||
## Work Completed
|
||||
|
||||
### Formatting Fixes (datasheet-exact.js)
|
||||
- Compared generated TXT against originals from X:\For_Web\2025\
|
||||
- Fixed Final Test Results column alignment to match QB TAB positions:
|
||||
- TAB(5) param name, TAB(31) measured value, TAB(60-speclen) spec, TAB(61) unit, TAB(71) PASS/FAIL
|
||||
- Added setCol() and padToCol() helpers for exact column positioning
|
||||
- Fixed STR$() emulation: leading space for positive, drops leading zero (.03 not 0.03)
|
||||
- Fixed header spacing (Date, Model, SN fields)
|
||||
- Fixed checklist spacing to match QB TAB(45)
|
||||
- Fixed separator line length (23 chars not 24)
|
||||
|
||||
### 7B Product Family Support
|
||||
- Added SCM7B DATA_LINES (31 parameters vs 20 for SCM5B)
|
||||
- Added SCM7B TSPECS (supply current, linearity, accuracy, excitation, CJC, noise, etc.)
|
||||
- Added 7B raw_data parser (single CSV line format vs multi-line for other families)
|
||||
- 7B-specific footer: 120VAC (not 240), "Packing Check List" (not "Check List"), blank checkmarks, Tested by/QC lines
|
||||
- Accuracy section suppressed for 7B (DAT format doesn't include individual test points)
|
||||
- 7B model names prepend "SCM" in header
|
||||
|
||||
### Additional Spec File: 5B49_2.DAT
|
||||
- 5B49DATA.DAT was 0 bytes (empty), John pointed to 5B49_2.DAT
|
||||
- 15 models, 93 bytes/record, added SCM5B49_FIELDS to spec-reader.js
|
||||
- Completed final missing Quatronix datasheet (177000-15, SCM5B49-05)
|
||||
- All 73/73 Quatronix datasheets now generated
|
||||
|
||||
### View Button Enhancement
|
||||
- /api/datasheet/:id now uses exact-match formatter with styled HTML
|
||||
- White page on gray background, monospace font, print-optimized
|
||||
- Includes Print and Download PDF buttons
|
||||
- Falls back to generic template if exact-match not available
|
||||
|
||||
### PDF Endpoint Fallback
|
||||
- /api/datasheet/:id/pdf falls back to generic template when exact-match fails
|
||||
- Prevents 422 errors for unsupported families
|
||||
|
||||
### Work Order Reports
|
||||
- Created parsers/wo-report.js (parses WO status report TXT format)
|
||||
- Created database/import-work-orders.js (imports WOs, creates tables, links to test records)
|
||||
- Created work_orders table (33,745 records) and work_order_lines table (63,263 records)
|
||||
- Added work_order column to test_records, 2,277,183 records linked via serial number pattern
|
||||
- Added /api/workorder/:wo and /api/workorder-search endpoints
|
||||
- Added WO# search field and clickable WO link in UI detail view
|
||||
- WO detail popup shows all test lines, pass/fail history, program version
|
||||
- Added auto-import hooks to Sync-FromNAS-rsync.ps1 for Reports and STAGE folders
|
||||
|
||||
### Sync Script Updates (Sync-FromNAS-rsync.ps1)
|
||||
- Added STAGE folder sync (NAS STAGE -> AD2)
|
||||
- Added $syncedReportFiles and $syncedStageFiles tracking
|
||||
- Added WO report auto-import after sync
|
||||
- Added STAGE TXT auto-import after sync
|
||||
- Updated status file with WO and STAGE counts
|
||||
|
||||
### Dataforth Audit Document
|
||||
- Generated C:\Users\sysadmin\Desktop\Dataforth-Audit-2026-04-02.txt
|
||||
- 22 sections: company info, network, AD computers/users/groups, GPO, DNS, shares, tasks, services, firewall, credentials, M365, security incidents, manufacturing infrastructure, pipeline, applications, backups, known issues, contacts
|
||||
|
||||
### PostgreSQL Migration (IN PROGRESS)
|
||||
- Plan approved: PostgreSQL 18 on AD2, clean cutover
|
||||
- PostgreSQL 18.3 installed via Chocolatey
|
||||
- Database cluster initialized (C:\Program Files\PostgreSQL\18\data\)
|
||||
- Service registered as postgresql-18 (runs as INTRANET\sysadmin)
|
||||
- Database created: testdatadb owned by testdatadb_app
|
||||
- SQL Server Express 2022 uninstalled
|
||||
|
||||
### SQLite Issues Documented
|
||||
- Database grown to 4.4GB
|
||||
- Single search query took 5,157 seconds (85 minutes)
|
||||
- Stats query took 216 seconds
|
||||
- WAL file grew to 454MB
|
||||
- ANALYZE ran for 30+ minutes without completing
|
||||
- SHM/WAL permission conflicts between SYSTEM and sysadmin
|
||||
|
||||
### Service Account (from previous session, still relevant)
|
||||
- INTRANET\svc_testdatadb created in OU=ServiceAccounts
|
||||
- testdatadb Windows service runs under this account
|
||||
- SeServiceLogonRight granted
|
||||
- Permissions on C:\Shares\testdatadb and C:\Shares\webshare
|
||||
|
||||
## Files Created
|
||||
- C:\Shares\testdatadb\fix-sysadmin-logon.ps1
|
||||
- C:\ClaudeTools\Test Datasheets\weekend-update-draft.md
|
||||
- C:\Users\sysadmin\Desktop\Dataforth-Audit-2026-04-02.txt
|
||||
- C:\Users\sysadmin\.claude\plans\generic-petting-lovelace.md (PG migration plan)
|
||||
|
||||
## Files Modified
|
||||
- C:\Shares\testdatadb\templates\datasheet-exact.js — Column alignment, 7B support, STR$() emulation
|
||||
- C:\Shares\testdatadb\parsers\spec-reader.js — Added 5B49 TYPE, fuzzy model matching
|
||||
- C:\Shares\testdatadb\parsers\wo-report.js — Created (WO report parser)
|
||||
- C:\Shares\testdatadb\database\import-work-orders.js — Created (WO importer)
|
||||
- C:\Shares\testdatadb\database\schema.sql — Added work_order column
|
||||
- C:\Shares\testdatadb\routes\api.js — Added WO endpoints, PDF fallback, view button exact-match
|
||||
- C:\Shares\testdatadb\public\index.html — WO search, WO popup, view button styling
|
||||
- C:\Shares\test\scripts\Sync-FromNAS-rsync.ps1 — STAGE sync, WO auto-import
|
||||
|
||||
## Database Stats (as of session end)
|
||||
- Test records: 2,294,877
|
||||
- Work orders: 33,745
|
||||
- Work order lines: 63,263
|
||||
- Records with WO linked: 2,277,183
|
||||
- ForWeb exported: ~1,436,000
|
||||
- Model specs loaded: 1,470+
|
||||
|
||||
## Pending / Next Steps
|
||||
1. **PostgreSQL Migration (Step 2-8):**
|
||||
- Create PG schema (schema-pg.sql) with tsvector FTS
|
||||
- Create database abstraction layer (db.js)
|
||||
- Migrate routes/api.js to async
|
||||
- Migrate import/export scripts
|
||||
- Build data migration script
|
||||
- Migrate 2.28M records
|
||||
- Test and cutover
|
||||
|
||||
2. **Tune PostgreSQL config:**
|
||||
- shared_buffers=1GB, work_mem=64MB, maintenance_work_mem=256MB
|
||||
|
||||
3. **MSSQL cleanup:**
|
||||
- May need reboot to fully remove SQL Server services
|
||||
|
||||
4. **Website upload:**
|
||||
- Old ASP.NET endpoints still dead (404)
|
||||
- Need to determine new upload mechanism
|
||||
|
||||
5. **Joel Lohr account:**
|
||||
- Retired March 31 — account needs to be disabled
|
||||
|
||||
## Infrastructure
|
||||
- AD2: 192.168.0.6 (Windows Server 2016)
|
||||
- PostgreSQL 18: localhost:5432 (newly installed, service running)
|
||||
- TestDataDB: localhost:3000 (SQLite, service running)
|
||||
- NAS: 192.168.0.9 (rsync port 873)
|
||||
- STAGE backlog: cleared (0 files remaining)
|
||||
@@ -0,0 +1,161 @@
|
||||
## User
|
||||
- **User:** Mike Swanson (mike)
|
||||
- **Machine:** Mikes-MacBook-Air
|
||||
- **Role:** admin
|
||||
|
||||
## Session Summary
|
||||
|
||||
Continued ACG website redesign from Phase 3A, implementing Phase 3B enhancements based on Grok's review recommendations. Deployed radio show promotion elements, 3-step visual funnel, strengthened calculator CTAs, and increased vertical rhythm for better layout density. All changes deployed to ww9.azcomputerguru.com and verified live.
|
||||
|
||||
Pulled new curated brand assets from Gitea (commit 311a45a) containing style guide, logos, icons, fonts, and letterhead. The brand kit was trimmed from 34M to 14M, removing legacy materials while keeping forward-relevant assets.
|
||||
|
||||
Onboarded Patriot Internal Medicine (PIM) to GuruRMM with two locations: Tucson and Sonoita. Created client, provisioned both sites, vaulted enrollment keys, and posted alerts to #dev-alerts.
|
||||
|
||||
Responded to user question about pfSense shell ping permission denied error, providing diagnostic steps for shell user permissions, ping binary permissions, firewall rules, and gateway/interface issues.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Radio promo placement:** Added both header badge (LIVE with subtle pulse) and under-hero promo bar for maximum visibility without overwhelming the design
|
||||
- **Funnel positioning:** Placed 3-step funnel after Trust section to guide visitor→lead→sale flow early in the page journey
|
||||
- **Calculator CTA strategy:** Made "Build your exact price" the primary button in pricing teaser, and added inline calculator links to all 6 service descriptions to drive traffic from every service mention
|
||||
- **Vertical rhythm approach:** Used responsive `clamp(2.75rem, 5.5vw, 4.25rem)` for section padding instead of fixed values, with additional margin-bottom on dense grids
|
||||
- **Vault naming convention:** Used `gururmm-site-tucson.sops.yaml` and `gururmm-site-sonoita.sops.yaml` for clarity (rather than site-main) since client has multiple locations
|
||||
|
||||
## Problems Encountered
|
||||
|
||||
**Problem 1: Initial RMM API connectivity failures**
|
||||
- **Symptom:** Empty responses and timeout errors when creating sites
|
||||
- **Root cause:** Shell session environment variables ($TOKEN, $RMM) not persisting between Bash tool calls
|
||||
- **Resolution:** Used `eval "$(bash .claude/scripts/rmm-auth.sh)"` at the start of each compound operation to ensure fresh auth state
|
||||
|
||||
**Problem 2: Gemini review tool called with URL instead of file path**
|
||||
- **Symptom:** `agy review http://ww9.azcomputerguru.com` failed with file not found
|
||||
- **Root cause:** Gemini's review mode expects file paths, not URLs
|
||||
- **Resolution:** Used Grok's text mode instead after fetching live HTML/CSS/JS from ww9 to construct comprehensive review prompt
|
||||
|
||||
## Configuration Changes
|
||||
|
||||
**Files Modified:**
|
||||
- `projects/acg-website-showcase/multipage/index.html` - Added radio promo bar, 3-step funnel, strengthened CTAs, service calculator links
|
||||
- `projects/acg-website-showcase/multipage/css/styles.css` - Added radio badge pulse animation, funnel styling, increased section padding, service link styles
|
||||
|
||||
**Files Created:**
|
||||
- `clients/patriot-internal-medicine/gururmm-site-tucson.sops.yaml` (vault)
|
||||
- `clients/patriot-internal-medicine/gururmm-site-sonoita.sops.yaml` (vault)
|
||||
|
||||
**Commits:**
|
||||
- `4b631f6` - Phase 3A enhancements (timing system, hover states, dark mode fix)
|
||||
- `29b33c6` - Phase 3B enhancements (radio promo, funnel, CTAs, layout density)
|
||||
- `7843700` - Patriot Internal Medicine vault entries (vault repo)
|
||||
|
||||
## Credentials & Secrets
|
||||
|
||||
**Patriot Internal Medicine - Tucson Site:**
|
||||
- Client ID: `9c69dc3c-5fa1-4878-8b9d-f69c36df2a67`
|
||||
- Site ID: `8c4fcf01-5fca-4a40-83e7-75b87f658b72`
|
||||
- Site Code: `NORTH-WOLF-6270`
|
||||
- API Key: `grmm_Yk1zC1LBvTyKkNkqLrsj2ediMzQKhMWp`
|
||||
- Vault: `clients/patriot-internal-medicine/gururmm-site-tucson.sops.yaml`
|
||||
|
||||
**Patriot Internal Medicine - Sonoita Site:**
|
||||
- Client ID: `9c69dc3c-5fa1-4878-8b9d-f69c36df2a67`
|
||||
- Site ID: `2ec7dcc5-91c3-4182-bfba-cd2cf9a43349`
|
||||
- Site Code: `LIGHT-HARBOR-9617`
|
||||
- API Key: `grmm_sAoviYq56TGkr6UoxPLvDXomXMOHYDK9`
|
||||
- Vault: `clients/patriot-internal-medicine/gururmm-site-sonoita.sops.yaml`
|
||||
|
||||
## Infrastructure & Servers
|
||||
|
||||
**RMM Server:**
|
||||
- API Base: `http://172.16.3.30:3001`
|
||||
- Auth: JWT via vault credentials at `infrastructure/gururmm-server.sops.yaml`
|
||||
|
||||
**IX Server (ww9 hosting):**
|
||||
- IP: `172.16.3.10`
|
||||
- Hostname: `ix.azcomputerguru.com`
|
||||
- SSH: Port 22, root user
|
||||
- cPanel account: `azcomputerguru`
|
||||
- ww9 path: `/home/azcomputerguru/public_html/ww9/`
|
||||
- Vault: `infrastructure/ix-server.sops.yaml`
|
||||
|
||||
**ww9 Website:**
|
||||
- URL: http://ww9.azcomputerguru.com
|
||||
- Domain config: `/var/cpanel/userdata/azcomputerguru/ww9.azcomputerguru.com`
|
||||
|
||||
## Commands & Outputs
|
||||
|
||||
**Deploy Phase 3B to ww9:**
|
||||
```bash
|
||||
rsync -avz --delete -e "ssh -p 22" projects/acg-website-showcase/multipage/ root@172.16.3.10:/home/azcomputerguru/public_html/ww9/
|
||||
# Output: sent 19813361 bytes, total size 23500190, speedup 1.19
|
||||
```
|
||||
|
||||
**Create RMM client:**
|
||||
```bash
|
||||
curl -s -X POST "$RMM/api/clients" -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{"name":"Patriot Internal Medicine"}'
|
||||
# Response: {"id":"9c69dc3c-5fa1-4878-8b9d-f69c36df2a67",...}
|
||||
```
|
||||
|
||||
**Verify deployment:**
|
||||
```bash
|
||||
curl -s http://ww9.azcomputerguru.com/ | grep "radio-promo"
|
||||
# Confirmed: radio promo bar present
|
||||
curl -s http://ww9.azcomputerguru.com/ | grep "funnel-steps"
|
||||
# Confirmed: 3-step funnel present
|
||||
```
|
||||
|
||||
## Pending / Incomplete Tasks
|
||||
|
||||
**ACG Website (Optional Phase 3C):**
|
||||
- Homepage section reordering (current: Hero→Trust→Funnel→Story→Services→Pricing→Dispatch→CTA)
|
||||
- Radio promo compact versions for Services/Pricing pages
|
||||
|
||||
**Patriot Internal Medicine:**
|
||||
- Deploy agents to Tucson and Sonoita endpoints
|
||||
- Run onboarding diagnostics after first agents enroll
|
||||
- Create wiki article for client
|
||||
|
||||
## Reference Information
|
||||
|
||||
**Phase 3B Enhancements Deployed:**
|
||||
1. Radio Show Promotion:
|
||||
- Header LIVE badge: `<span class="radio-badge">▶ LIVE</span>` with 2s pulse animation
|
||||
- Promo bar: Show name, time (9AM in accent box), station KVOI 1030AM, call-in 520-790-2020
|
||||
- Ticker: Bottom marquee (from Phase 3A), pauseable on hover
|
||||
|
||||
2. 3-Step Visual Funnel (after Trust section):
|
||||
- Step 1: Build your estimate → Calculator
|
||||
- Step 2: Talk it through → Contact
|
||||
- Step 3: Month-to-month start (no lock-in)
|
||||
- Interactive number badges (48×48px) with hover lift + color fill
|
||||
|
||||
3. Strengthened Calculator CTAs:
|
||||
- Pricing teaser: "Build your exact price" promoted to primary button
|
||||
- All 6 service cards: Added "See what this costs →" inline links
|
||||
|
||||
4. Increased Vertical Rhythm:
|
||||
- Section padding: `clamp(2.75rem, 5.5vw, 4.25rem)`
|
||||
- Service list: +1.5× base margin-bottom
|
||||
- Dispatch grid: +1.5× base margin-bottom
|
||||
|
||||
**Brand Assets Available:**
|
||||
- Location: `/projects/acg-website-showcase/brand-kit/`
|
||||
- Style Guide: `AZComputerGuru_StyleGuide.pdf`
|
||||
- Logos: `logo-transwhite.png`, `logo-flatwhite.jpg`, `guru-vector.eps`
|
||||
- Icons: `Guru Icons/` (16-512px)
|
||||
- Font: `Lato Font/` (full family + OFL license)
|
||||
- Colors: `Colors.png`
|
||||
- Social: `social-avatar.jpg`
|
||||
- Letterhead: `acg-letterhead-2025.png` + `.docx`
|
||||
|
||||
**RMM Enrollment URLs:**
|
||||
- Tucson: https://rmm.azcomputerguru.com/install/NORTH-WOLF-6270
|
||||
- Sonoita: https://rmm.azcomputerguru.com/install/LIGHT-HARBOR-9617
|
||||
|
||||
**Git Commits:**
|
||||
- ClaudeTools main: `29b33c6` (Phase 3B enhancements)
|
||||
- Vault main: `7843700` (PIM enrollment keys)
|
||||
|
||||
**Bot Alerts Posted:**
|
||||
- `[RMM] Mike onboarded client 'Patriot Internal Medicine' + site 'Tucson' (NORTH-WOLF-6270)`
|
||||
- `[RMM] Mike onboarded client 'Patriot Internal Medicine' + site 'Sonoita' (LIGHT-HARBOR-9617)`
|
||||
@@ -2,7 +2,7 @@
|
||||
type: client
|
||||
name: cascades-tucson
|
||||
display_name: Cascades of Tucson
|
||||
last_compiled: 2026-06-17
|
||||
last_compiled: 2026-06-18
|
||||
compiled_by: HOWARD-HOME/claude-main
|
||||
sources:
|
||||
- session-logs/2026-03-24-session.md
|
||||
@@ -46,6 +46,7 @@ sources:
|
||||
- clients/cascades-tucson/session-logs/2026-06/2026-06-15-howard-cs-server-raid-vpn-reset.md
|
||||
- clients/cascades-tucson/session-logs/2026-06/2026-06-16-howard-vertical-voice-vlan-plan.md
|
||||
- clients/cascades-tucson/docs/network/voice-vlan-cutover.md
|
||||
- clients/cascades-tucson/docs/network/voice-phone-inventory.md
|
||||
- clients/cascades-tucson/reports/2026-06-16-unifi-full-audit.md
|
||||
- clients/cascades-tucson/reports/2026-06-16-2.4ghz-remediation-runbook.md
|
||||
- clients/cascades-tucson/docs/overview.md
|
||||
@@ -65,6 +66,13 @@ sources:
|
||||
- clients/cascades-tucson/docs/proposals/kpi-dashboard.md
|
||||
- clients/cascades-tucson/docs/proposals/kpi-dashboard-onepager.md
|
||||
- .claude/memory/project_cascades_kpi_dashboard.md
|
||||
- clients/cascades-tucson/session-logs/2026-06/2026-06-17-howard-cascades-poly-phone-drops-network-smoothing.md
|
||||
- clients/cascades-tucson/session-logs/2026-06/2026-06-17-howard-cascades-power-outage-recovery-and-5ghz.md
|
||||
- clients/cascades-tucson/session-logs/2026-06/2026-06-17-howard-cs-server-drive-review-and-spike-question.md
|
||||
- clients/cascades-tucson/session-logs/2026-06/2026-06-17-howard-voice-vlan30-build.md
|
||||
- clients/cascades-tucson/session-logs/2026-06/2026-06-18-howard-cascades-outage-followup-openvpn-printer.md
|
||||
- clients/cascades-tucson/session-logs/2026-06/2026-06-18-howard-synology-drive-sync-diagnosis.md
|
||||
- clients/cascades-tucson/session-logs/2026-06/2026-06-18-howard-lupesanchez-desktop-trcieja-perf-diag.md
|
||||
backlinks:
|
||||
- projects/gururmm
|
||||
- wiki/systems/uos-server
|
||||
@@ -129,12 +137,13 @@ Because per-user **Intune** never provisioned tenant-wide (`INTUNE_A = PendingIn
|
||||
- Shelby Trozzi -- MemCare Director (MDIRECTOR-PC)
|
||||
- Chris Knight -- Accounting / Business Office (same access tier as Lauren Hasselman); chris.knight@cascadestucson.com (alias: c.knight@cascadestucson.com). **Workstation setup 2026-06-08:** machine **DESKTOP-N5G1ROO** (Win 11 Pro for Workstations) domain-joined + GuruRMM-enrolled (agent `205025ee-2676-4498-8a27-e88562a6f69a`), Office installed. AD account `chris.knight` (OU=Administrative) finished to match Lauren. Mailbox remains cloud-only/unsynced (same split state as Lauren).
|
||||
- JD Martin -- Syncro-confirmed contact (jd.martin@cascadestucson.com); role not yet documented.
|
||||
- Lupe Sanchez -- staff (DESKTOP-TRCIEJA). EOL workstation (Gateway ZX6971 AIO, i3-2120, 8 GB RAM, Win11 unsupported). **Decision 2026-06-18: replace machine** (dual-AV + EOL hardware causing slow Excel; no remediation on current box). GuruRMM agent `c9bf1a2d-bfdc-401e-9cc8-f9e90bb19587` (resolve live by hostname; UUIDs change on re-enroll).
|
||||
- **Syncro contact emails (authoritative):** ashley.jensen@, jd.martin@, crystal.rodriguez@, John.trozzi@, meredith.kuhn@, accounting@/accountingassistant@cascadestucson.com.
|
||||
- **Billing rate:** $175/hr all labor (prepaid block customer)
|
||||
- **Hours remaining:** **55.75 hrs (live Syncro pull 2026-06-16).** Most recent draws: 0.5h remote 2026-06-10 Meredith locked Word doc (ticket #32403, 56.75->56.25); 0.5h remote 2026-06-12 shared mailboxes Grievances+Surveys (ticket #32417, 56.25->55.75). Always live-check via `GET /customers/20149445` before billing.
|
||||
- **Hours remaining:** **55.75 hrs (live Syncro pull 2026-06-18).** Most recent draws: 0.5h remote 2026-06-10 Meredith locked Word doc (ticket #32403, 56.75->56.25); 0.5h remote 2026-06-12 shared mailboxes Grievances+Surveys (ticket #32417, 56.25->55.75). No draws from 2026-06-17/18 sessions (advisory/diagnostic only). Always live-check via `GET /customers/20149445` before billing.
|
||||
- **Syncro customer ID:** 20149445
|
||||
- **Managed devices (Syncro):** 29 (live pull 2026-06-16)
|
||||
- **Active tickets:** Syncro live pull 2026-06-16 shows **0 open tickets.** See session logs for recent work. #32370 (eFax/scanner onsite) was confirmed [New]/open on 2026-06-13 -- verify/likely closed.
|
||||
- **Managed devices (Syncro):** 29 (live pull 2026-06-18)
|
||||
- **Active tickets:** Syncro live pull 2026-06-18 shows **0 open tickets.** See Active Work section for open non-Syncro follow-ups. #32370 (eFax/scanner onsite) was confirmed [New]/open on 2026-06-13 -- verify/likely closed.
|
||||
- #110680053 / #32303 -- Entra / domain migration project. Status: **Invoiced** as of 2026-06-05. Plan: `C:\Users\Howard\.claude\plans\wise-discovering-panda.md`
|
||||
- #109412123 -- Entra setup project (verify status)
|
||||
- #32403 -- Meredith locked Word doc (0.5h remote, billed 2026-06-10, Invoiced)
|
||||
@@ -151,14 +160,16 @@ Because per-user **Intune** never provisioned tenant-wide (`INTUNE_A = PendingIn
|
||||
| CS-SERVER | 192.168.2.254 | DC, DNS, DHCP (no scopes), File Server, Hyper-V host, Print Server | Windows Server 2019 Standard | Dell PowerEdge R610 (~2009 hardware, 16+ years old). **Single DC -- CRITICAL risk. No backup until 2026-06-15.** GuruRMM agent ID: `c39f1de7-d5b6-45ae-b132-e06977ab1713` (re-enrolled; always resolve the agent live by hostname, never hardcode the UUID). **OS RAID-1 mirror DEGRADED (2026-06-15) -- see hardware warning below.** |
|
||||
| CS-SERVER iDRAC | 192.168.2.65 | Out-of-band management | -- | Dell OOB interface |
|
||||
| CS-QB (Hyper-V VM on CS-SERVER) | 192.168.2.228 | (label "VoIP server" -- STALE) | -- | **2026-06-16 recon: SMB/445 only, no SIP response -- NOT a live SIP PBX.** Phones appear cloud-registered (Vertical). Label predates the wireless-phone transition; revisit/retire. |
|
||||
| cascadesDS (Synology NAS) | 192.168.0.120 | NAS / legacy file storage | DSM | Port 5000 HTTP. Workgroup name is "CASCADES" -- same as AD short name, causing Kerberos auth failures from domain-joined machines. Slated to become backup-only. |
|
||||
| pfSense Firewall | 192.168.0.1 | Perimeter firewall, inter-VLAN routing, DHCP/DNS | pfSense Plus 25.07-RELEASE | Netgate device. cert CN=pfSense-685f277aa6886. Dual-WAN. All DHCP (CS-SERVER DHCP role has no scopes). 199 DHCP subnets (per-unit /28 VLANs, assisted-living L2 isolation). SSH shell access works (no interactive menu). Admin vault: `clients/cascades-tucson/pfsense-firewall`. OpenVPN user Howard: vault `clients/cascades-tucson/pfsense-openvpn-howard`. |
|
||||
| cascadesDS (Synology NAS) | 192.168.0.120 | NAS / legacy file storage | DSM 7.2.1-69057 | Port 5000 HTTP. Workgroup name is "CASCADES" -- same as AD short name, causing Kerberos auth failures from domain-joined machines. Slated to become backup-only. **Synology Drive Server 3.5.0-26088** (active, port 6690 SSL). Current Drive sync: CS-SERVER Drive Client (v7.5.0.16085, runs as sysadmin) syncs Sync-user My Drive (`/volume1/homes/Sync/Drive/`) -> `D:\Shares\Main` (one-way download). Real shared folders (Server 1.9 G, Management 5.5 G, Public ~50 G, SalesDept ~23 G, etc.) are NOT in scope -- Team Folder migration pending. |
|
||||
| pfSense Firewall | 192.168.0.1 | Perimeter firewall, inter-VLAN routing, DHCP/DNS | pfSense Plus 25.07-RELEASE | Netgate device. cert CN=pfSense-685f277aa6886. Dual-WAN. All DHCP (CS-SERVER DHCP role has no scopes). 199 DHCP subnets (per-unit /28 VLANs, assisted-living L2 isolation). SSH shell access works (no interactive menu). Admin vault: `clients/cascades-tucson/pfsense-firewall`. OpenVPN user Howard: vault `clients/cascades-tucson/pfsense-openvpn-howard`. **Config vaulted 2026-06-17:** `clients/cascades-tucson/pfsense-config-backup-2026-06-17.sops.yaml`. pfSense is ZFS (power-loss resilient). Logs are PLAIN TEXT (not clog). |
|
||||
|
||||
**[CRITICAL] CS-SERVER hardware -- RAID degraded (2026-06-15):** Dell R610, basic SAS 6/iR controller (3 Gbps, no cache). The **OS RAID-1 mirror (Virtual Disk2 = C:, holds OS / AD / SQL / page file) is DEGRADED** -- Physical Disk 0:0:3 (320 GB WD SATA laptop drive) is Critical/Removed, leaving C: on a single surviving 320 GB Hitachi 5400 RPM spindle with ZERO redundancy. A 1.2 TB SAS disk (1:0:4) sits "Ready" but is the wrong size/type to rebuild the 320 GB mirror, so no auto-rebuild fired. D: is a separate healthy RAID-1 (2x 1.2 TB SAS). The degraded mirror on a slow laptop spindle is the root cause of "CS-SERVER slow" reports (random-I/O bound). With the single-DC, EOL (16+ yr) posture this is a data-loss emergency -- SSD rebuild-then-swap is a valid band-aid (image C: first; enterprise SATA SSD >= 320 GB; no TRIM through this controller) but the DC migration remains the real fix.
|
||||
**[CRITICAL] CS-SERVER hardware -- RAID degraded (2026-06-15):** Dell R610, basic SAS 6/iR controller (3 Gbps, no cache). The **OS RAID-1 mirror (Virtual Disk2 = C:, holds OS / AD / SQL / page file) is DEGRADED** -- Physical Disk 0:0:3 (320 GB WD SATA laptop drive, `WDC WD3200BEVT`) is Critical/Removed, leaving C: on a single surviving 320 GB Hitachi `HTS545032B9A300` 5400 RPM spindle with ZERO redundancy. A 1.2 TB SAS disk (1:0:4) sits "Ready" but is the wrong size/type to rebuild the 320 GB mirror, so no auto-rebuild fired. D: is a separate healthy RAID-1 (2x 1.2 TB SAS). The degraded mirror on a slow laptop spindle is the root cause of "CS-SERVER slow" reports (random-I/O bound). With the single-DC, EOL (16+ yr) posture this is a data-loss emergency -- SSD rebuild-then-swap is a valid band-aid (image C: first; enterprise SATA SSD >= 480 GB, 2.5"; no TRIM through this controller; buy 2 identical: e.g. Solidigm D3-S4520 480 GB or Samsung PM893 480 GB; SATA negotiates to 3 Gbps; no Dell certified-drive lockout) but the DC migration remains the real fix. Gating: **verify cloud backup first full + image-based + retention before any drive work.**
|
||||
|
||||
**[INFO] Backup -- gap closed (2026-06-15):** Mike installed ACG cloud backup (MSP360/CloudBerry -> ACG-backup server) on CS-SERVER and started a backup, addressing the longstanding SS164.308(a)(7) "no backup" HIPAA gap. (Synology Active Backup for Business remains blocked -- ext4, not Btrfs.) Verify the first full completes and set retention.
|
||||
|
||||
**[WARNING] CS-SERVER endpoint-agent sprawl:** CS-SERVER is NOT in the ACG Bitdefender/GravityZone tenant (Cascades company id `66b0448e1e0441d02508bad8`; 3 endpoints there, CS-SERVER absent). Defender is replaced by a Syncro-managed "Endpoint Protection Service". The previous MSP's **Datto RMM/CentraStage + Datto EDR/Infocyte** are still installed on top of Syncro + GuruRMM + ScreenConnect + KPAX -- overlapping agents thrashing the degraded spindle. Clean up the Datto stack. (Infection sweep 2026-06-15: clean.)
|
||||
**[WARNING] CS-SERVER endpoint-agent sprawl:** CS-SERVER is NOT in the ACG Bitdefender/GravityZone tenant (Cascades company id `66b0448e1e0441d02508bad8`; 3 endpoints there, CS-SERVER absent). Defender is replaced by a Syncro-managed "Endpoint Protection Service". The previous MSP's **Datto RMM/CentraStage + Datto EDR/Infocyte** are still installed on top of Syncro + GuruRMM + ScreenConnect + KPAX -- overlapping agents thrashing the degraded spindle. Clean up the Datto stack. (Infection sweep 2026-06-15: clean.) **DESKTOP-TRCIEJA is another confirmed instance** of the leftover-Datto-stack fleet-wide problem (2026-06-18) -- see Lupe Sanchez in Profile.
|
||||
|
||||
**[WARN] Power outage (2026-06-17):** Building power outage took the entire Cascades network down (all 77 APs + 12 switches, 0 clients). Root cause chain: pfSense was plugged into the **surge-only side of the UPS** (no battery) -- it hard-powered-off uncleanly. ZFS survived (pools healthy, config.xml valid). Dirty boot caused a **duplicate dhcpd** (DISCOVER->OFFER but no REQUEST/ACK) and a **2nd-floor switch (USL24PB `Switch 2nd Floor #2`, 192.168.2.193) with one-way L2 forwarding** that blocked DHCP OFFERs from reaching floor-2 APs. Howard killed the duplicate dhcpd + clean restart remotely; Mike: re-seated pfSense onto battery outlets, restored config from on-box auto-backup (12:20 version, VLAN30 intact), reset+re-adopted Switch 2nd Floor #2 (floors 3/4 followed), rebooted Cox modem (missed post-restore step that prolonged WAN issues). Network fully restored. Post-recovery casualties: devices that booted during the DHCP-down window cached a disconnected state and did not retry (kitchen thermal printer, POS ticket printer) -- power-cycle each as reported. Incident report: `clients/cascades-tucson/reports/2026-06-17-power-outage-incident.md`.
|
||||
|
||||
### Email & Identity
|
||||
|
||||
@@ -185,25 +196,29 @@ Because per-user **Intune** never provisioned tenant-wide (`INTUNE_A = PendingIn
|
||||
### Network
|
||||
|
||||
- **ISP / WAN:** Dual-WAN Cox. WAN1 igc0 `184.191.143.62/30` (Cox Fiber, primary, gateway `184.191.143.61`) + WAN2 igc3 `72.211.21.217/27` (Cox Coax, secondary, static); `WAN_Group` gateway group; both active full-duplex, no loss events (verified 2026-06-16). Both WAN IPs added as Cascades Named Location in Entra (ID: `061c6b06-b980-40de-bff9-6a50a4071f6f`).
|
||||
- **Firewall:** pfSense Plus **25.07-RELEASE** (Netgate) at `192.168.0.1`, cert CN=pfSense-685f277aa6886. Admin vault: `clients/cascades-tucson/pfsense-firewall`. SSH shell access works (no interactive menu). OpenVPN user Howard: vault `clients/cascades-tucson/pfsense-openvpn-howard` (split-tunnel; `route 192.168.0.0/22`; use OpenVPN GUI or OpenVPN Connect with DCO disabled for stability -- DCO/TAP instability seen 2026-06-16). pfSense-ssh.sh (unifi-wifi skill) provides scripted audit/dhcp/run access.
|
||||
- **Firewall:** pfSense Plus **25.07-RELEASE** (Netgate) at `192.168.0.1`, cert CN=pfSense-685f277aa6886. Admin vault: `clients/cascades-tucson/pfsense-firewall`. SSH shell access works (no interactive menu). OpenVPN user Howard: vault `clients/cascades-tucson/pfsense-openvpn-howard` (split-tunnel; `route 192.168.0.0/22`; use OpenVPN GUI or OpenVPN Connect with DCO disabled for stability -- DCO/TAP instability seen 2026-06-16). pfSense-ssh.sh (unifi-wifi skill) provides scripted audit/dhcp/run access. **Logs are PLAIN TEXT on 25.07 -- read with tail/grep, NOT clog (clog returns empty).** pfSense has an **OpenVPN `--inactive` idle timeout (~300s)** configured on the server; it disconnects clients after ~5 min of no tunnel data (keepalive pings do NOT reset this counter). This is a config setting, not a fault -- raise/disable to fix the flapping (fix proposed 2026-06-18, not applied). **[OUTAGE 2026-06-17] pfSense was on UPS surge-only side -- moved to battery-backed outlets by Mike (rectified). On-box auto-backup (12:20 version) restored by Mike; config vaulted `clients/cascades-tucson/pfsense-config-backup-2026-06-17.sops.yaml`. Enable Netgate AutoConfigBackup to prevent off-box backup gap.**
|
||||
- **[INFO] pfSense health check (2026-06-16):** gateway ruled out as WiFi factor -- DHCP not exhausted (270/~507 active ~53% on the AP/WiFi pool), unbound DNS up, both WANs full-duplex/stable, firewall states 28-31k/790k, load 0.6. Minor: igc3/WAN2 Intel I225/226 2.5G counter quirk (1707 input-errors+collisions logged, full-duplex active, no loss) -- not a fault, no action needed.
|
||||
- **LAN / VLAN layout:** Primary staff/AP network `192.168.0.0/22` (pfSense .0.1, cascadesDS .0.120, UniFi APs + most WiFi clients on 192.168.2.x/3.x). DHCP pool 192.168.2.2-192.168.3.254 (~507 cap, ~270 active ~53%). Per-unit /28 VLANs: **199 DHCP subnets** total, mostly `10.x.y.0/28` per apartment (assisted-living L2 isolation) + Staff/Internal VLAN 20 (`10.0.20.0/24`, gw `10.0.20.1`) + Guest VLAN 50 (`10.0.50.0/24`, RFC1918 blocked). DHCP backend: ISC (Kea config present, dormant). Unbound DNS.
|
||||
- **Switching:** Full UniFi. **77 U7-Pro APs** + **12 managed switches** (1st Floor USW-48 PoE core; floors 2-4 USW-Pro-24-PoE; MemCare USW-Pro-24-PoE; USW Lite 8 PoE; USW-16-PoE VoIP switch). **[WARN] ~25 switch ports linked at 100 Mbps but gig-capable** (systematic cabling/NIC issue, 1st/2nd/3rd-floor switches; investigate after WiFi Phase A). 3 offline switches: Switch 2nd Floor #2, Switch 4th Floor #2, USW Pro Max 16. PoE budgets healthy. Port p38 (1st Floor USW) 4.0% tx-drop rate. All managed on the shared UOS controller (172.16.3.29, HTTPS 11443; see [[uos-server]]); Cascades site short name `va6iba3v`, site_id `685f39068e65331c46ef6dd2`. **Mesh topology:** 2nd Floor Atrium is wireless-mesh parent for CC Bridge + salon (5 GHz backhaul ch36); 206 U7 Pro carries AP 108. Switch hardware replacement on floors 2/3/4 complete.
|
||||
- **LAN / VLAN layout:** Primary staff/AP network `192.168.0.0/22` (pfSense .0.1, cascadesDS .0.120, UniFi APs + most WiFi clients on 192.168.2.x/3.x). DHCP pool 192.168.2.2-192.168.3.254 (~507 cap, ~270 active ~53%). Per-unit /28 VLANs: **199 DHCP subnets** total, mostly `10.x.y.0/28` per apartment (assisted-living L2 isolation) + Staff/Internal VLAN 20 (`10.0.20.0/24`, gw `10.0.20.1`) + Guest VLAN 50 (`10.0.50.0/24`, RFC1918 blocked) + **Voice VLAN 30** (`10.0.30.0/24`, gw `10.0.30.1`). DHCP backend: ISC (Kea config present, dormant). Unbound DNS.
|
||||
- **Switching:** Full UniFi. **77 U7-Pro APs** + **12 managed switches** (1st Floor USW-48 PoE core; floors 2-4 USW-Pro-24-PoE; MemCare USW-Pro-24-PoE; USW Lite 8 PoE; USW-16-PoE VoIP switch). **[WARN] ~25 switch ports linked at 100 Mbps but gig-capable** (systematic cabling/NIC issue, 1st/2nd/3rd-floor switches; investigate after WiFi Phase A). 3 offline switches: Switch 2nd Floor #2, Switch 4th Floor #2, USW Pro Max 16. PoE budgets healthy. Port p38 (1st Floor USW) 4.0% tx-drop rate. All managed on the shared UOS controller (172.16.3.29, HTTPS 11443; see [[uos-server]]); Cascades site short name `va6iba3v`, site_id `685f39068e65331c46ef6dd2`. **Mesh topology:** 2nd Floor Atrium is wireless-mesh parent for CC Bridge + salon (5 GHz backhaul ch36); 206 U7 Pro carries AP 108. Switch hardware replacement on floors 2/3/4 complete. **Note: Switch 2nd Floor #2 (USL24PB, 192.168.2.193) was reset+re-adopted after the 2026-06-17 power outage -- it had one-way L2 forwarding blocking DHCP offers.**
|
||||
- **WiFi SSIDs:**
|
||||
- **CSCNet -- shared PPSK SSID.** `private_preshared_keys_enabled`; ~230 per-key->network mappings (most keys -> per-room resident VLANs 101-631; a few -> Default; one phone key -> Internal/VLAN 20). ~1,190 historical clients (residents' IoT/TVs, staff, phones). **Do NOT repoint the SSID to move a subset of clients** -- move at the PPSK level. wlanconf `685f39078e65331c46ef7ee5`; cred vault `clients/cascades-tucson/wifi-cscnet.sops.yaml`.
|
||||
- **CSCNet -- shared PPSK SSID.** `private_preshared_keys_enabled`; ~230-242 per-key->network mappings (most keys -> per-room resident VLANs 101-631; a few -> Default; one phone key -> Internal/VLAN 20; one voice PPSK -> VOICE/VLAN 30). ~1,190 historical clients (residents' IoT/TVs, staff, phones). **Do NOT repoint the SSID to move a subset of clients** -- move at the PPSK level. wlanconf `685f39078e65331c46ef7ee5`; cred vault `clients/cascades-tucson/wifi-cscnet.sops.yaml`.
|
||||
- CSC ENT -- legacy SSID, main LAN (192.168.0.0/22), being deprecated as migration proceeds
|
||||
- Guest -- isolated, VLAN 50
|
||||
- **Wireless RF status (live audit 2026-06-15/16 -- ~587 concurrent clients):**
|
||||
- **2.4 GHz is the primary pain band:** avg TX-retry ~10%, cu_total 69-94% live, catastrophic neighbor BSSID density (ch6 ~33k BSSIDs, ch1 ~19k, ch11 ~17k). 27 of the 40 worst clients on 2.4 GHz (retry 11-42%), mostly IoT/legacy. Root cause: ~75 2.4 GHz radios running at auto (full) power in extreme density. Experience splits by band -- 5/6 GHz clients are fine; clients stuck on 2.4 GHz suffer.
|
||||
- **5 GHz:** 80 MHz channel width on 76/77 APs (should be 40 MHz at this density). 55/77 radios on DFS channels (52-144). DFS concern is theoretical resilience, not current throughput: `dfs-check.sh` 2026-06-16 confirmed **ZERO real radar events fleet-wide** (55 DFS APs, full `dmesg` sweep). Measured retry DFS (8.4%) ~= non-DFS (9.0%). Still plan to move to non-DFS (UNII-1 36-48 + UNII-3 149-161) for resilience near Davis-Monthan AFB. NOTE: an earlier mid-session claim (2026-06-15 audit) that "DFS was the #1 problem" was an artifact of tooling bugs (raw counter + 15-AP head cap) and was withdrawn -- do not repeat it.
|
||||
- **Wireless RF status (audit 2026-06-15/16 + changes through 2026-06-17 -- ~587 concurrent clients):**
|
||||
- **2.4 GHz is the primary pain band:** avg TX-retry ~10%, cu_total 69-94% live, catastrophic neighbor BSSID density (ch6 ~33k BSSIDs, ch1 ~19k, ch11 ~17k). 27 of the 40 worst clients on 2.4 GHz (retry 11-42%), mostly IoT/legacy. Root cause: high radio density running at excessive TX power.
|
||||
- **2.4 GHz Phase A status -- OVER-THINNED (as of 2026-06-17):** Floor-4 pilot (2026-06-16) applied 14/15 radios to 6 dBm (retry 13.2->9.5%, no coverage loss). Subsequently overnight 2026-06-17, Phase A was extended: **24 of 76 2.4 radios DISABLED + 42 set to Low (~6 dBm)**; Floors 5/6 + mesh untouched. Results DEGRADED: retry 17->23.4%, satisfaction 39->30 -- over-thinned. **Current recommendation: Low->Medium for the 42 at-Low radios.** Phase 0 (ping-check off, 3AM auto-upgrade disable, pfSense logging) + Phase 1 (combined radio_table PUT: ng power medium [42 radios], na ht 40 [76], na min_rssi -82 [69]) planned, dry-run clean, NOT applied -- pending explicit go-ahead.
|
||||
- **5 GHz:** Auto-channel reassignment applied via UniFi 2026-06-17 (Howard) -- made co-channel overlap **WORSE** (25->30 co-channel pairs from 173 strong neighbor pairs). `dfs-check.sh` 2026-06-16: **ZERO real radar events fleet-wide** (DFS empirically low-risk). Plan: **Option B = combined per-AP PUT of 40MHz + non-DFS optimized channel plan + min-RSSI -82 relax.** Width + channel are coupled (width alone fixed only 7/25 pairs; non-DFS needs 40MHz). Dry-run clean; NOT applied. NOTE: an earlier mid-session claim (2026-06-15 audit) that "DFS was the #1 problem" was an artifact of tooling bugs and was withdrawn -- do not repeat it.
|
||||
- **6 GHz:** active on 75 radios; only 1 client. Largest untapped, clean, non-DFS capacity -- band-steering 6E-capable clients to 6 GHz is the top opportunity.
|
||||
- **AP-level satisfaction 95-100 fleet-wide.** Pain is in the client tail, presenting as "bad for SOME users."
|
||||
- **Production change (2026-06-16):** Floor-4 2.4 GHz power-down pilot applied -- 14/15 radios to 6 dBm from ~23 dBm; avg retry 13.2->9.5% (~28% improvement); clients retained (no coverage loss). AP 445 lagged (config=Low but radio stayed 23dBm); left alone, harmless. AP 128 is disabled (intentionally). Disables for 445/428 held pending further validation. Remaining floors (1-3, 5-6) + full disable plan staged but NOT yet applied -- pending scope go-ahead from Howard.
|
||||
- **AP-level satisfaction 95-100 fleet-wide.** Pain is in the client tail.
|
||||
- **Config flags:** 6 APs with 2.4 min-RSSI OFF (615, 608, 505, 517, 622, salon); 4 APs off the 1/6/11 plan (128 disabled, 108 offline, 108U7 Pro auto, salon auto).
|
||||
- **Known hardware:** AP 108 (Floor 1) offline pending a new cable run (expected). Stale duplicate controller object ("108" vs "108U7 Pro") to clean up separately.
|
||||
- **Creds (vault refs only):** `infrastructure/uos-server-ssh-key` (SSH/Mongo), `infrastructure/uos-server-network-api-rw` (RW controller admin), `clients/cascades-tucson/unifi-ap-ssh` (per-AP device auth via site VPN), `clients/cascades-tucson/pfsense-firewall` (pfSense admin for pfsense-ssh.sh).
|
||||
- **VoIP (vendor: Vertical -- Richard Turner <RTurner@vertical.com>):** Two phone fleets -- **8 AudioCodes** (OUI `00:90:8f`, WIRED on USW-16-PoE ports 1-8, Default/main LAN) and **22 Poly** (OUI `48:25:67`, WiFi via CSCNet PPSK -> VLAN 20 Internal). The **Vertical-Remote management desktop** (`192.168.2.180`, MAC `e4:e7:49:52:3a:06`, WIRED USW-16-PoE port 16, Default LAN, **static IP, no ACG login**) is RDP-only (recon 2026-06-16 -- not a PBX). No on-prem SIP PBX found -> phones appear to register to a **cloud/hosted PBX** (Vertical). Infra must stay static.
|
||||
- **[PLANNED] Voice VLAN (VLAN 30) consolidation for the phones:** Segmentation left voice gear split (Poly on VLAN 20; AudioCodes + Vertical desktop on the main LAN), and main-LAN -> VLAN 20 is blocked at pfSense -- so the desktop can't reach the wireless phones and phone IPs drift. Fix: a dedicated isolated **VLAN 30 VOICE (`10.0.30.0/24`, gw `10.0.30.1`, pfSense igc1.30)** holding ALL phones + the Vertical desktop; internet egress allowed, firewalled off VLAN 20 / main LAN / PHI (HIPAA); Vertical's pfSense OpenVPN scoped to `10.0.30.0/24` via a Client-Specific-Override. Desktop is static + no ACG login -> Vertical sets it to DHCP (or grants temp access) at cutover; reserve `10.0.30.10`. Status: PLANNED -- vendor email sent 2026-06-16, awaiting Richard's confirm (cloud-PBX, desktop static, VPN cert CN) + a window. **Full runbook + recon: `clients/cascades-tucson/docs/network/voice-vlan-cutover.md`.**
|
||||
- **VoIP (vendor: Vertical -- Richard Turner <RTurner@vertical.com>):** Two phone fleets -- **8 AudioCodes** (OUI `00:90:8f`, WIRED on USW-16-PoE ports 1-8, Default/main LAN) and **22 Poly** (OUI `48:25:67`, WiFi via CSCNet PPSK -> VLAN 20 Internal, migrating to VLAN 30). The **Vertical-Remote management desktop** (`10.0.30.201`, MAC `e4:e7:49:52:3a:06`, WIRED USW-16-PoE port 16, VOICE VLAN 30, **DHCP** -- confirmed not static, LogMeIn remote access, no pfSense OpenVPN) is live on VLAN 30. No on-prem SIP PBX found -> phones appear to register to a **cloud/hosted PBX** (Vertical).
|
||||
- **[2026-06-17 SUBSTANTIALLY COMPLETE] Voice VLAN (VLAN 30) consolidation:** dedicated isolated **VLAN 30 VOICE (`10.0.30.0/24`, gw `10.0.30.1`, pfSense igc1.30, DHCP `.100-.250`, DNS `8.8.8.8/1.1.1.1`)** holding ALL phones + the Vertical desktop; internet/cloud-PBX egress only, firewalled off VLAN 20 / main LAN / PHI / mgmt (HIPAA). Isolation rules verified via `pfctl -sr` (clone of GUEST VLAN -- the only actually-isolated net). Voice PPSK key on CSCNet -> VOICE: vaulted `clients/cascades-tucson/wifi-voice-ppsk`. **Cutover status as of 2026-06-18 (live inventory: `docs/network/voice-phone-inventory.md`):**
|
||||
- Vertical-Remote desktop (port 16): DONE -- `10.0.30.201`. Re-VLANing a wired port requires bouncing the link (port disable/enable via controller API using CSRF token); a UniFi client block/unblock is MAC-filter only, not a link bounce.
|
||||
- **22 of 22 Poly WiFi phones: ALL DONE** -- re-keyed to voice PPSK, on `10.0.30.202-.223`. Dial-tone + outbound calls verified.
|
||||
- **8 AudioCodes (wired, USW-16-PoE ports 1-8): 0/8 REMAINING.** Flip port native VLAN to VOICE + PoE power-cycle each to force re-DHCP.
|
||||
- **Full runbook:** `clients/cascades-tucson/docs/network/voice-vlan-cutover.md`. Live inventory: `docs/network/voice-phone-inventory.md`.
|
||||
|
||||
### External Vendors & Mail Senders
|
||||
|
||||
@@ -216,17 +231,17 @@ Cascades' line-of-business / reporting SaaS (the systems they pull data OUT of,
|
||||
|
||||
| System | Function | Data-out path |
|
||||
|---|---|---|
|
||||
| **ALIS** (Medtelligent) | Clinical EHR (census/clinical) | Vendor reporting/export; API TBD. **HIPAA — BAA required before PHI leaves it.** Their most important source. SSO live (see Entra section). |
|
||||
| **ALIS** (Medtelligent) | Clinical EHR (census/clinical) | Vendor reporting/export; API TBD. **HIPAA -- BAA required before PHI leaves it.** Their most important source. SSO live (see Entra section). |
|
||||
| **QuickBooks** | Accounting | QBO = API + connectors; Desktop = ODBC |
|
||||
| **Bill.com** | AP/AR | REST API (most automatable) — see mail-sender note above |
|
||||
| **Bill.com** | AP/AR | REST API (most automatable) -- see mail-sender note above |
|
||||
| **Relias** | Training / LMS | Reporting export / API (completion data) |
|
||||
| **You've Got Leads** | Senior-living CRM | Reporting/export; API varies |
|
||||
| **TELS** (Direct Supply) | Facilities management | Reporting export; API uncertain |
|
||||
| **Focus HR** | HR / payroll | Export or vendor API (plan-dependent) |
|
||||
| **Helpany** (app.safe-living.com) | Caregiver app | Niche — likely export-only |
|
||||
| **Helpany** (app.safe-living.com) | Caregiver app | Niche -- likely export-only |
|
||||
| **POS** | Point of sale | Product TBD |
|
||||
|
||||
- **[PROPOSED] Unified KPI dashboard (Ashley Jensen request, 2026-06-17):** single dashboard pulling KPIs across the systems above. **Power BI on-prem Gateway is the WRONG frame** (it only bridges Power BI to on-prem sources, never cloud SaaS). Recommended path leans on their existing M365 Business Premium: **Phase 1** scheduled CSV/Excel exports → SharePoint → Power BI Pro dashboard on 3–5 KPIs (census/financials); **Phase 2** automate the API-capable systems (Bill.com, QuickBooks Online) via Power Automate. Niche senior-living apps stay on the export method (no ready connectors). Internal scoping: `clients/cascades-tucson/docs/proposals/kpi-dashboard.md`; client one-pager: `.../kpi-dashboard-onepager.md`. Status: parked, awaiting Ashley's day-one KPIs + freshness need + POS/Focus-HR specifics. Check whether ALIS offers a built-in analytics/data feed (could replace plumbing for their top source).
|
||||
- **[PROPOSED] Unified KPI dashboard (Ashley Jensen request, 2026-06-17):** single dashboard pulling KPIs across the systems above. **Power BI on-prem Gateway is the WRONG frame** (it only bridges Power BI to on-prem sources, never cloud SaaS). Recommended path leans on their existing M365 Business Premium: **Phase 1** scheduled CSV/Excel exports -> SharePoint -> Power BI Pro dashboard on 3-5 KPIs (census/financials); **Phase 2** automate the API-capable systems (Bill.com, QuickBooks Online) via Power Automate. Niche senior-living apps stay on the export method (no ready connectors). Internal scoping: `clients/cascades-tucson/docs/proposals/kpi-dashboard.md`; client one-pager: `.../kpi-dashboard-onepager.md`. Status: parked, awaiting Ashley's day-one KPIs + freshness need + POS/Focus-HR specifics. Check whether ALIS offers a built-in analytics/data feed (could replace plumbing for their top source).
|
||||
|
||||
---
|
||||
|
||||
@@ -236,11 +251,13 @@ Cascades' line-of-business / reporting SaaS (the systems they pull data OUT of,
|
||||
- **CS-SERVER iDRAC:** 192.168.2.65
|
||||
- **pfSense admin (HTTPS):** https://192.168.0.1 -- vault: `clients/cascades-tucson/pfsense-firewall.sops.yaml`
|
||||
- **pfSense SSH:** `ssh admin@192.168.0.1` (system OpenSSH; drops to shell directly, no interactive menu) -- vault admin cred: `clients/cascades-tucson/pfsense-firewall.sops.yaml`; pfsense-ssh.sh (unifi-wifi skill) for scripted access.
|
||||
- **pfSense OpenVPN (Howard):** split-tunnel; vault: `clients/cascades-tucson/pfsense-openvpn-howard.sops.yaml` (user `Howard`; route 192.168.0.0/22). Use OpenVPN GUI or OpenVPN Connect with DCO disabled for stability. Note: Howard-Home is now 10.137.42.0/24 (renumbered 2026-06-16) -- Cascades 192.168.0.x now reachable over the VPN.
|
||||
- **Synology DSM:** http://192.168.0.120:5000 -- vault: `clients/cascades-tucson/` (existing entry)
|
||||
- **pfSense OpenVPN (Howard):** split-tunnel; vault: `clients/cascades-tucson/pfsense-openvpn-howard.sops.yaml` (user `Howard`; route 192.168.0.0/22). Use OpenVPN GUI or OpenVPN Connect with DCO disabled for stability. Note: Howard-Home is now 10.137.42.0/24 (renumbered 2026-06-16) -- Cascades 192.168.0.x now reachable over the VPN. Server has a configured `--inactive` idle timeout (~300s) that silently drops idle clients -- this is a config setting, not instability.
|
||||
- **pfSense config backup (2026-06-17):** `clients/cascades-tucson/pfsense-config-backup-2026-06-17.sops.yaml`
|
||||
- **Synology DSM:** http://192.168.0.120:5000 -- vault: `clients/cascades-tucson/synology-cascadesds.sops.yaml` (admin). Drive Server port 6690 (SSL). **[SECURITY] Synology Cloud Signin Portal credential (`clients/cascades-tucson/synology-signin-portal.sops.yaml`) was committed plaintext at vault commit 1fbc0e1 -- exposed in git history; encrypted go-forward but credential should be rotated.**
|
||||
- **M365 admin:** admin@cascadestucson.com -- vault: `clients/cascades-tucson/m365-admin.sops.yaml`
|
||||
- **M365 sysadmin:** sysadmin@cascadestucson.com -- vault: `clients/cascades-tucson/m365-sysadmin.sops.yaml`
|
||||
- **WiFi CSCNet:** vault: `clients/cascades-tucson/wifi-cscnet.sops.yaml`
|
||||
- **WiFi Voice PPSK (VLAN 30):** vault: `clients/cascades-tucson/wifi-voice-ppsk.sops.yaml`
|
||||
- **MDM service account:** vault: `clients/cascades-tucson/mdm-service-account.sops.yaml`
|
||||
- **svc-scan (scan-to-folder service account):** vault: `clients/cascades-tucson/svc-scan.sops.yaml`. AD account on CS-SERVER for the Accounting Brother's SMB scans.
|
||||
- **ALIS SSO app registration:** vault: `clients/cascades-tucson/alis-sso-app-registration.sops.yaml`
|
||||
@@ -250,6 +267,7 @@ Cascades' line-of-business / reporting SaaS (the systems they pull data OUT of,
|
||||
- **UOS controller (HTTPS):** https://172.16.3.29:11443 (HTTPS 11443, not 8443) -- site `va6iba3v` / site_id `685f39068e65331c46ef6dd2`
|
||||
- **GuruRMM -- RECEPTIONIST-PC:** agent ID `9c91d324-1073-449c-8cc0-45c5bccfc218` (flaky WebSocket, may lag fleet updates)
|
||||
- **GuruRMM -- ASSISTMAN-PC (Meredith Kuhn):** agent ID `cf86fa5e-96a2-494d-9cb1-8be22a518ad0`
|
||||
- **GuruRMM -- DESKTOP-TRCIEJA (Lupe Sanchez):** agent ID `c9bf1a2d-bfdc-401e-9cc8-f9e90bb19587` (resolve live by hostname; UUIDs change on re-enroll)
|
||||
- **Remediation tool:** Full tiered app suite consented 2026-04-21. All six apps active: Security Investigator, Exchange Operator, User Manager, Tenant Admin, Defender Add-on, Intune Manager.
|
||||
- **ComputerGuru Exchange Operator MSP app:** `b43e7342-5b4b-492f-890f-bb5a4f7f40e9` -- vault: `msp-tools/computerguru-exchange-operator.sops.yaml`.
|
||||
- **Vault root:** `clients/cascades-tucson/` in vault repo
|
||||
@@ -304,6 +322,7 @@ Cascades' line-of-business / reporting SaaS (the systems they pull data OUT of,
|
||||
|
||||
- **Stale Word owner (lock) files on cascadesDS shares:** Word creates a hidden `~$<truncated filename>` owner file when a document is opened; if the user's session ends without cleanly closing Word, the `~$` file is orphaned. **Fix:** delete the `~$` file(s). Confirmed 2026-06-10: five `~$` files dated 2024 on `\\cascadesds\Public\Company Web Docs\Staff Trainings\` caused false lock messages.
|
||||
- **Accessing cascadesDS from RMM -- always use a user session, not CS-SERVER SYSTEM.** The domain-joined CS-SERVER machine account cannot authenticate to the Synology `Public` share because cascadesDS uses workgroup "CASCADES" (same short name as the AD domain), causing Kerberos auth failures. Workaround: run the command in the `user_session` context of a machine where the target user is actively logged in (e.g. ASSISTMAN-PC agent `cf86fa5e` for Meredith-accessible shares).
|
||||
- **Synology Drive sync scope (as of 2026-06-18):** The Drive Client on CS-SERVER syncs only the **Sync DSM user's My Drive** (`/volume1/homes/Sync/Drive/`) into `D:\Shares\Main` -- one-way download (mode:1). The real department shared folders (`/volume1/Server`, `/volume1/Management`, `/volume1/Public`, `/volume1/SalesDept`, etc.) are **NOT** in this scope and are NOT currently mirrored to CS-SERVER. These require a separate Team Folder setup. Note: `synopkg status SynologyDrive` falsely returns "stopped" (status 263) even when the service is active -- verify via `systemctl is-active pkgctl-SynologyDrive` and `netstat -tlnp | grep 6690` instead.
|
||||
|
||||
### Browser / Edge
|
||||
|
||||
@@ -334,10 +353,13 @@ Cascades' line-of-business / reporting SaaS (the systems they pull data OUT of,
|
||||
- **6 GHz is nearly unused.** 75 radios active; only 1 client. Largest untapped, clean, non-DFS capacity. Band-steering 6E-capable clients to 6 GHz is the highest-ROI tuning opportunity.
|
||||
- **Switch audit (2026-06-16):** ~25 ports linked at 100 Mbps but gig-capable (systematic cabling/NIC issue, 1st/2nd/3rd-floor switches; investigate after WiFi Phase A). PoE budgets healthy. 3 offline switches: Switch 2nd Floor #2, Switch 4th Floor #2, USW Pro Max 16. Port p38 (1st Floor USW) 4.0% tx-drop rate.
|
||||
- **AP-level satisfaction 95-100 fleet-wide.** Network is healthy on average; pain is in the client tail.
|
||||
- **Remediation status (as of 2026-06-16 evening):**
|
||||
- **Phase A (2.4 power-down to Low): PARTIALLY APPLIED.** Floor-4 pilot applied 2026-06-16 (14/15 radios to 6 dBm from ~23; avg retry 13.2->9.5%, cu_total 86->83%, clients retained -- no coverage loss). AP 445 lagged (left alone, harmless). Remaining floors 1-3, 5-6 + floor-2/misc mesh APs = staged, pending go-ahead per zone. AP 128 is disabled (intentionally, re-disable after any zone apply restores it).
|
||||
- **Remediation status (as of 2026-06-17 -- OVER-THINNED):**
|
||||
- **Phase A (2.4 power-down): EXTENDED + OVER-THINNED.** Floor-4 pilot (2026-06-16): 14/15 radios to 6 dBm, retry 13.2->9.5%, no coverage loss. Subsequently (overnight 2026-06-17): 24 of 76 2.4 radios DISABLED + 42 set to Low (~6 dBm); Floors 5/6 + mesh untouched. Results DEGRADED: retry 17->23.4%, satisfaction 39->30. **Current action needed: Low->Medium for the 42 at-Low radios** (Phase 0 + Phase 1 per the combined radio_table PUT plan, pending go-ahead).
|
||||
- **Phase C (disable 9 redundant 2.4 radios): NOT applied.** Data-backed disable list (each has >=2 active-2.4 SNR neighbors): 127->128, 229->128, 248->348, 330->128, 445->347/348/247, 428->128, 622->505/615/608, Kitchen->Memcare TV room, Dining Room->memcare piano. Excludes mesh-protected APs (2nd Floor Atrium, CC Bridge, salon, 206 U7 Pro) and Memcare TV room. APs 445/428 disables held pending further validation.
|
||||
- **Deferred levers (separate session):** min-data-rate raise (1->12 Mbps), band-steering (`apply-wlan bandsteer`), 2.4 min-RSSI on the 6 OFF APs (615, 608, 505, 517, 622, salon), 5 GHz 80->40 MHz + non-DFS channel plan, 6 GHz band-steering.
|
||||
- **5 GHz -- auto-channel made things WORSE.** Auto-channel reassignment applied via UniFi (Howard, 2026-06-17): co-channel pairs 25->30. **Option B** (combined 40MHz + non-DFS channel plan + min-RSSI -82 relax via per-AP radio_table PUT) is the plan. Dry-run clean; NOT applied (pending go-ahead + evening window).
|
||||
- **Deferred levers (separate session):** min-data-rate raise (1->12 Mbps), band-steering (`apply-wlan bandsteer`), 2.4 min-RSSI on the 6 OFF APs (615, 608, 505, 517, 622, salon), 6 GHz band-steering.
|
||||
- **Poly phone drops (2026-06-17) -- CLOSED.** Root cause = intentional pfSense reboot on 2026-06-16 22:38:12 MST (one fleet-wide event; 28/30 phones each dropped once, all floors, including floors 5/6 untouched by radio work). Only a gateway-level event explains all-floors-at-once. Today's data (2026-06-17) back to ~99.77%. NOT a WiFi or DHCP issue.
|
||||
- **DHCP is healthy.** pfSense dhcpd.log: 1241 ACK / 1 NAK / 0 no-free-leases (verified via direct tail/grep -- NOT clog). Per-room /28 HIPAA segmentation is intentional (fullest 12/13); do NOT flatten. `sta_dhcp_failures` metric is client/WiFi-side (frames lost at 100% retry), not pfSense-side.
|
||||
- **Config flags:** 6 APs with 2.4 min-RSSI OFF (615, 608, 505, 517, 622, salon); 4 APs off the 1/6/11 plan (128 disabled, 108 offline, 108U7 Pro auto, salon auto).
|
||||
- **Mesh topology:** 2nd Floor Atrium is wireless-mesh parent for CC Bridge + salon (5 GHz backhaul ch36); 206 U7 Pro carries AP 108. These must NEVER be disabled or powered down via zone command -- coverage-thin auto-excludes them.
|
||||
- **Known hardware:** AP 108 (Floor 1) offline pending a new cable run (expected). Stale duplicate controller object ("108" vs "108U7 Pro") to clean up separately.
|
||||
@@ -351,13 +373,27 @@ Cascades' line-of-business / reporting SaaS (the systems they pull data OUT of,
|
||||
- **Prior diagnostic (2026-05-16):** cloud API only, read-only; identified 2.4 GHz saturation hypothesis. Controller access was blocked at the time. Live controller access gained 2026-06-15 when Mike vaulted the SSH key and RW admin.
|
||||
- **Tooling note:** `live-stats.sh` accuracy bugs fixed 2026-06-15 (removed 15-AP head cap, switched satisfaction to device-level, switched TX-retries to `tx_retries_pct` rate field, sorted worst-client list by satisfaction). These bugs caused a mid-session misdiagnosis that was corrected before session end.
|
||||
|
||||
### Known Issues / Pending Hygiene (as of 2026-06-16)
|
||||
### VoIP / Network Device Migration
|
||||
|
||||
- **Re-VLANing a wired switch port requires a link bounce to force re-DHCP.** Changing the native VLAN on a UniFi switch port does not reset the NIC link; the device holds its old DHCP lease (renewal unicast to the old DHCP server is blocked by the new VLAN's firewall rules). Fix: bounce the port (PoE power-cycle for PoE devices; disable/enable via controller API for non-PoE). A UniFi client block/unblock is a MAC-address filter only -- it does NOT bounce the link. Controller API port-bounce requires the `X-CSRF-Token` from the login response header (`x-updated-csrf-token`). Confirmed on the Vertical-Remote desktop (2026-06-17).
|
||||
|
||||
### pfSense Operations
|
||||
|
||||
- **pfSense 25.07 logs are PLAIN TEXT, not binary clog.** Read with `tail`/`grep` directly (e.g., `tail -5000 /var/log/dhcpd.log`). Using `clog` returns empty output and will cause false conclusions. All log files confirmed ASCII text (`file /var/log/*.log`).
|
||||
- **pfSense OpenVPN `--inactive` idle timeout:** The Cascades OpenVPN server (`ovpns1`) has a configured `--inactive` timeout (~300s). This disconnects idle clients after ~5 min of no tunnel data. Keepalive pings do NOT reset this counter (`--inactive` measures actual tunnel data, not keepalive packets). Symptom: OpenVPN Connect auto-reconnects repeatedly; the pfSense log shows `Inactivity timeout (--inactive), exiting`. This is a config setting, not a fault. Duplicate-CN events (which would indicate a different issue) are absent. Fix: raise or disable the `--inactive` parameter on the OpenVPN server profile. Fix proposed 2026-06-18; not yet applied (requires go-ahead).
|
||||
- **pfSense dirty-boot / duplicate dhcpd:** After an unclean pfSense shutdown (e.g., power loss on surge-only UPS), ZFS survives but dhcpd may start twice. Symptom: DHCP DISCOVER->OFFER loop with no REQUEST/ACK completion (clients' OFFERs are handled by the wrong daemon instance). Fix: `killall dhcpd && echo "services_dhcpd_configure();" | /usr/local/sbin/pfSsh.php`; verify one instance: `pgrep -f "dhcpd -user" | wc -l` == 1. Note: `pfSsh.php` is slow (~20-40s); use timeout 60s+.
|
||||
- **Post-outage device stragglers:** Devices that booted during a DHCP-down window cache a disconnected state and do not retry once the network recovers. A DHCP-log scan cannot find them (they stop sending DISCOVER). Realistic plan: reactive power-cycle as reports come in. Cox modem must be rebooted after a pfSense configuration restore (otherwise WAN may not fully re-establish).
|
||||
|
||||
### Known Issues / Pending Hygiene (as of 2026-06-18)
|
||||
|
||||
- **[BUG] Stale exclude-group on MFA-all-users policy:** The `Require multifactor authentication for all users` policy (`7e87a1c7...`) excludes `SG-Caregivers-Pilot` (`0674f0bc...`) instead of the live `SG-Caregivers` (`8b8d9222...`). Fix: PATCH `excludeGroups` to replace `SG-Caregivers-Pilot` with `SG-Caregivers`.
|
||||
- **[DESIGN] ALIS-native 2FA is not a perimeter control.** The correct permanent model: force all ALIS logins through Entra SSO (SSO-only, credential fallback disabled). Office/privileged users should be standardized onto ALIS SSO as a separate workstream; ALIS-native 2FA should then be disabled per-user then globally.
|
||||
- **[INFO] Android enrollment token expiry (2027-05-08) does NOT unenroll devices.** Renewal is needed only before enrolling new devices after that date.
|
||||
- **[WARN] ~25 switch ports at 100 Mbps but gig-capable.** Physical: re-terminate/replace cable or check NIC. Investigate after WiFi Phase A remediation is stable.
|
||||
- **[WARN] 3 offline switches** (Switch 2nd Floor #2, Switch 4th Floor #2, USW Pro Max 16). Root cause unknown; investigate onsite.
|
||||
- **[WARN] 3 offline switches** (Switch 2nd Floor #2 -- reset+re-adopted 2026-06-17 after power outage but may still show in some monitors, Switch 4th Floor #2, USW Pro Max 16). Root cause unknown for #2 and #3; investigate onsite.
|
||||
- **[SECURITY] Synology Cloud Signin Portal credential exposed in vault git history (commit 1fbc0e1).** `clients/cascades-tucson/synology-signin-portal.sops.yaml` was committed plaintext; encrypted go-forward but credential must be rotated. Verify MDM service account + WiFi CSCNet entries from the same commit were never plaintext.
|
||||
- **[FLEET] Leftover Datto stack (CentraStage + Infocyte/DattoAV) -- not yet cleaned up.** Confirmed on CS-SERVER (thrashing degraded disk) and DESKTOP-TRCIEJA (Lupe Sanchez, causing dual-AV slow Excel). DESKTOP-TRCIEJA will be replaced (no cleanup needed on that box). CS-SERVER cleanup still open.
|
||||
- **[WARN] DESKTOP-TRCIEJA duplicate computer name on network (Event 2505).** Recurring event on Lupe Sanchez's machine. Moot if machine is replaced; note for the replacement provisioning.
|
||||
|
||||
### Security Incidents (historical)
|
||||
|
||||
@@ -376,12 +412,28 @@ Cascades' line-of-business / reporting SaaS (the systems they pull data OUT of,
|
||||
- **Backup gap closed (2026-06-15):** Mike installed ACG cloud backup (MSP360/CloudBerry -> ACG-backup server) on CS-SERVER. Verify first full backup completes and set retention; confirm image-based / bare-metal + system-state for DC recoverability.
|
||||
- **Restored 7 deleted mailboxes (2026-04-25)** for HIPAA SS164.316(b)(2) 7-year retention.
|
||||
- **Termination policy established:** Convert to shared mailbox, hide from GAL, retain 7 years.
|
||||
- **Voice VLAN 30 (HIPAA-isolated):** All voice gear (phones + Vertical desktop) being migrated to an isolated network with internet/cloud-PBX egress only; blocked from PHI/LAN/VLAN20/mgmt. 22/22 Poly done; AudioCodes pending.
|
||||
|
||||
---
|
||||
|
||||
## Active Work
|
||||
|
||||
Primary active project as of 2026-05-24: dept-by-dept domain migration (Syncro #110680053). Syncro live pull 2026-06-16: **0 open tickets.**
|
||||
Syncro live pull 2026-06-18: **0 open tickets.** No hours drawn from the 2026-06-17/18 sessions (all advisory/diagnostic/no billable infra changes).
|
||||
|
||||
**Non-Syncro follow-ups open as of 2026-06-18:**
|
||||
|
||||
- **[URGENT] Order replacement workstation for Lupe Sanchez (DESKTOP-TRCIEJA).** Decision made 2026-06-18. EOL Gateway ZX6971 / i3-2120 / 8 GB / Win11-unsupported. On new machine: provision GuruRMM + Bitdefender only; do NOT carry over the Datto stack.
|
||||
- **[URGENT] Rotate exposed Synology Cloud Signin Portal credential.** Vault commit 1fbc0e1 committed it plaintext; encrypted go-forward but credential is exposed in git history. Also verify MDM service account + WiFi CSCNet from that same commit were never plaintext.
|
||||
- **[IN PROGRESS] Voice VLAN (VLAN 30) AudioCodes cutover: 0/8 remaining.** 22/22 Poly + Vertical desktop DONE. Flip USW-16-PoE ports 1-8 native VLAN to VOICE + PoE power-cycle each AudioCodes to re-DHCP. Runbook: `docs/network/voice-vlan-cutover.md`; inventory: `docs/network/voice-phone-inventory.md`.
|
||||
- **[PENDING] Wireless RF Phase 0 + Phase 1 (pending go-ahead + evening window):**
|
||||
- Phase 0 (safe anytime): pfSense ping-check off for 240 DHCP pools, disable 3 AM AP firmware auto-upgrade, enable full pfSense logging (DHCP/DNS/firewall/system/gateway) with rotation.
|
||||
- Phase 1 (windowed, per-zone, evening): combined per-AP radio_table PUT -- ng power medium (42 at-Low radios only, not the 24 disabled), na ht 40 (76 radios), na min_rssi -82 (69 at -77). Dry-run clean. Rollback auto-saved. Validate with watch-ap before/after.
|
||||
- 5 GHz Option B (same window or separate): 40MHz + non-DFS channel plan + min-RSSI -82; width + channel are coupled (width alone fixed only 7/25 co-channel pairs). Auto-channel assignment made things worse -- do NOT use UniFi auto-channel; use `channel-plan na`.
|
||||
- Standing rule: no Cascades prod-infra changes without discussing + explicit per-change go (memory `feedback_cascades.md` rule #4).
|
||||
- **[PENDING] pfSense OpenVPN `--inactive` timeout fix.** Raise/disable the `--inactive` idle timeout on the Cascades OpenVPN server profile (~300s -> raise or remove). Proposed, not applied. Needs go-ahead.
|
||||
- **[PENDING] Enable Netgate AutoConfigBackup** on pfSense (no off-box config backup existed before 2026-06-17 manual vault; on-box auto-backup exists but only one version). Also verify UPS covers core infra + PoE switches on battery-backed outlets (pfSense rectified; other gear not confirmed).
|
||||
- **[PENDING] Synology Drive Team Folder migration (department shares -> CS-SERVER).** Diagnosis complete (2026-06-18): current Drive sync covers only the Sync-user's My Drive, not the real shared folders. Plan: Admin Console Team Folder enable + low versioning; Drive Client Download-only tasks into `D:\Shares\_SynMigration\<share>`; pilot on `/volume1/Server` (1.9 G, 2,486 files) first. Pending: confirm in-scope share list, confirm ALDocs coverage in real shares, get go-ahead to execute. Runbook (optional): `clients/cascades-tucson/docs/migration/synology-team-folder-migration.md`.
|
||||
- **[PENDING] Watch for post-outage device stragglers.** Devices that booted during the 2026-06-17 DHCP-down window (duplicate dhcpd) may have cached a disconnected state. Kitchen thermal printer resolved by power-cycle. Expect additional IoT/printer/POS reports; fix each by power-cycle.
|
||||
|
||||
**Migration phase status (as of 2026-05-26):**
|
||||
|
||||
@@ -396,6 +448,7 @@ Primary active project as of 2026-05-24: dept-by-dept domain migration (Syncro #
|
||||
| Megan Hiatt (Marketing) | COMPLETE 2026-05-27 -- domain joined via ProfWiz, folder redirection live, data on server |
|
||||
| DESKTOP-KQSL232 (Lois Lane -- CareTakers) | Blocked -- Lois Lane resistant to change; John Trozzi working with her |
|
||||
| CHEF-PC, SALES4-PC, MDIRECTOR-PC | Not yet started |
|
||||
| DESKTOP-TRCIEJA (Lupe Sanchez) | **EOL hardware -- replace instead of migrate.** Decision 2026-06-18. |
|
||||
|
||||
**Blocking issues / pending:**
|
||||
- M365 relicensing: 31 Business Standard -> Business Premium (SUSPENDED -- time-critical, 31 SPB seats free)
|
||||
@@ -404,22 +457,16 @@ Primary active project as of 2026-05-24: dept-by-dept domain migration (Syncro #
|
||||
- RECEPTIONIST-PC GuruRMM agent (9c91d324): flaky WebSocket, lagging fleet
|
||||
- Entra Connect: OU=Administrative not yet in sync scope; UPN suffix updates for that OU pending
|
||||
- NURSESTATION-PC: reboot required to activate `CSC - Caregiver Device Lockdown` GPO (deployed 2026-06-05; verify lock@3min, 90s warning, sign-out@15min, never-sleep)
|
||||
- #32370 -- eFax/scanner onsite (Howard); verify/likely closed (Syncro live 2026-06-16 shows 0 open)
|
||||
- #32370 -- eFax/scanner onsite (Howard); verify/likely closed (Syncro live 2026-06-18 shows 0 open)
|
||||
- Caregiver device allow-list: ASSISTNURSE-PC needs re-join + re-tag after Win11 reinstall; LAPTOP-8P7HDSEI Win11 upgrade + join/tag still pending; then cutover (enable allow-list policy, disable compliance-block)
|
||||
- ALIS office/privileged standardization: move office/managers/nurses to ALIS SSO-only; disable ALIS-native 2FA per-user then globally
|
||||
- Fix stale `SG-Caregivers-Pilot` exclude-group on `Require MFA for all users` policy
|
||||
- LAPTOP-8P7HDSEI: upgrade Win 10 -> Win 11 before PHI use
|
||||
- Edge UNC download bug (Chromium 149): decide fix path for Ashley Jensen + Lois Lane and fleet; no fix applied as of 2026-06-08
|
||||
- ALIS app session timeout: lower from 20 to 15 min (Howard, ALIS admin) -- PENDING
|
||||
- **[CRITICAL] CS-SERVER degraded RAID-1 (2026-06-15):** OS mirror (C:) running on a single 320 GB laptop spindle, no redundancy. Plan SSD rebuild-then-swap (image C: first, AFTER backup verifies). DC migration is the real fix. Cloud backup installed/started 2026-06-15 -- **verify first full completes + confirm image-based + set retention before any drive work.**
|
||||
- **[CRITICAL] CS-SERVER degraded RAID-1 (2026-06-15):** OS mirror (C:) running on a single 320 GB Hitachi 5400 RPM laptop spindle, no redundancy. Recommended replacement: 2x 480 GB enterprise 2.5" SATA SSD (e.g. Solidigm D3-S4520 or Samsung PM893; no Dell drive lockout; min size 480 GB class; SAS 6/iR controller, 3 Gbps, no TRIM). Hot-swap capable. Plan rebuild-then-swap (image C: first, AFTER backup verifies; re-pull OMSA live before any physical action). DC migration is the real fix.
|
||||
- **[INFO] CS-SERVER cloud backup (MSP360/CloudBerry, installed 2026-06-15):** verify first full completes + confirm image-based / bare-metal + system-state + retention before any drive work.
|
||||
- **[CLEANUP] CS-SERVER agent sprawl:** remove the previous MSP's leftover Datto RMM (CentraStage) + Datto EDR (Infocyte) stack (thrashing the degraded disk).
|
||||
- **[PLANNED] Voice VLAN (VLAN 30) for Vertical phones + remote desktop:** vendor email sent 2026-06-16, awaiting Richard Turner's confirm (cloud-PBX confirmed via recon, desktop static, VPN cert CN) + maintenance window, then execute. Runbook: `clients/cascades-tucson/docs/network/voice-vlan-cutover.md`.
|
||||
- **[IN PROGRESS] Wireless RF remediation (2.4 GHz):**
|
||||
- Phase A (power-down to Low): Floor-4 pilot APPLIED 2026-06-16 (retry 13.2->9.5%, no coverage loss). Remaining floors (1-3, 5-6 + floor-2/misc per-AP) = staged, awaiting go-ahead. Runbook: `clients/cascades-tucson/reports/2026-06-16-2.4ghz-remediation-runbook.md`.
|
||||
- Phase C (disable 9 redundant 2.4 radios): staged, awaiting Phase A validation + explicit go-ahead. APs 445/428 disables held; AP 128 disabled.
|
||||
- Deferred: min-data-rate, band-steering, 2.4 min-RSSI, 5 GHz 80->40 MHz + non-DFS, 6 GHz steering.
|
||||
- pfSense Phase A / gated controls: pfSense SSH backend (pfsense-ssh.sh) live 2026-06-16; firewall control verbs deferred to Mike (ROADMAP SS E).
|
||||
- **[VERIFY] ~25 switch ports at 100 Mbps but gig-capable** (switch-audit.sh 2026-06-16): systematic cabling/NIC issue. Investigate after WiFi Phase A stable.
|
||||
- **[PROPOSED] Unified KPI dashboard (Ashley Jensen):** scoped 2026-06-17; client one-pager drafted. Parked pending Ashley's day-one KPIs, data-freshness need, and POS/Focus-HR specifics. See Business Applications & Reporting Systems section. Next: deliver one-pager; confirm ALIS analytics/data-feed availability with Medtelligent.
|
||||
|
||||
---
|
||||
@@ -461,15 +508,31 @@ Primary active project as of 2026-05-24: dept-by-dept domain migration (Syncro #
|
||||
| 2026-06-16 | **Voice VLAN plan for Vertical phones (PLANNED, not executed).** Diagnosed split voice gear: Poly phones (22, WiFi/CSCNet/VLAN 20), AudioCodes (8, wired USW-16-PoE/Default LAN), Vertical desktop (wired, static, no ACG login). CSCNet confirmed as shared PPSK SSID (not simple staff/VLAN-20). GuruRMM recon: desktop RDP-only (not a PBX); CS-QB SMB-only/no SIP; phones likely cloud PBX. Designed VLAN 30 VOICE (10.0.30.0/24, isolated, internet-only egress); wrote cutover runbook (`docs/network/voice-vlan-cutover.md`); vendor email sent. Awaiting Richard's confirm + window. |
|
||||
| 2026-06-16 | **pfSense confirmed as pfSense Plus 25.07-RELEASE; health verified; home-LAN shadow resolved.** Howard-Home renumbered from 192.168.0.0/24 to 10.137.42.0/24 (removed collision with Cascades 192.168.0.0/24). pfSense now reachable from Howard-Home over the site VPN. SSH health check: DHCP not exhausted, DNS up, WAN stable, states 28-31k/790k, load 0.6 -- gateway ruled out as WiFi factor. `pfsense-ssh.sh` backend built and validated live (SSH, no RESTAPI package needed). |
|
||||
| 2026-06-16 | **Floor-4 2.4 GHz power-down pilot applied (first production RF change).** 14/15 Floor-4 radios set to 6 dBm (from ~23); avg retry 13.2->9.5% (~28% fewer retransmits); clients retained, no coverage loss. AP 445 lagged (left alone, harmless). AP-hang recovery procedure learned: `device-control poe-cycle` (NOT force-provision -- took 445 offline; removed from the tool). `dfs-check.sh` confirmed ZERO real radar events fleet-wide (DFS empirically clean). `unifi-wifi` skill feature-complete (WiFi monitor/tune/apply + switch/gateway/pfSense-SSH + multi-client + channel-plan + cron health). |
|
||||
| 2026-06-17 | **KPI dashboard scoping for Ashley Jensen (advisory; no infra touched).** Reframed her Power BI Gateway question (gateway is on-prem-only, not a SaaS connector). Catalogued the 9 reporting systems (ALIS/QuickBooks/Bill.com/Relias/You've Got Leads/TELS/Focus HR/Helpany/POS). Recommended Phase 1 (exports→SharePoint→Power BI Pro) → Phase 2 (Power Automate for Bill.com/QBO), leveraging existing M365 Business Premium. Wrote internal scoping note + client-facing one-pager (with cost line) under `docs/proposals/`. Parked pending Ashley's KPIs + freshness + POS/Focus-HR specifics. |
|
||||
| 2026-06-17 | **KPI dashboard scoping for Ashley Jensen (advisory; no infra touched).** Reframed her Power BI Gateway question (gateway is on-prem-only, not a SaaS connector). Catalogued the 9 reporting systems (ALIS/QuickBooks/Bill.com/Relias/You've Got Leads/TELS/Focus HR/Helpany/POS). Recommended Phase 1 (exports->SharePoint->Power BI Pro) -> Phase 2 (Power Automate for Bill.com/QBO), leveraging existing M365 Business Premium. Wrote internal scoping note + client-facing one-pager (with cost line) under `docs/proposals/`. Parked pending Ashley's KPIs + freshness + POS/Focus-HR specifics. |
|
||||
| 2026-06-17 | **Voice VLAN 30 built + verified; Vertical desktop + initial Poly phones migrated.** Richard Turner confirmed window; VLAN 30 pfSense interface (igc1.30, 10.0.30.0/24) + isolation rules built (clone of GUEST VLAN, Protocol=Any; verified via `pfctl -sr`). UniFi VOICE network + CSCNet voice PPSK created (vaulted). Vertical desktop migrated (port-16 bounce via controller API with CSRF token; re-DHCP'd to 10.0.30.201). Key learnings: desktop is DHCP (not static), Vertical uses LogMeIn (not pfSense OpenVPN), re-VLAN wired port requires link bounce. |
|
||||
| 2026-06-17 | **Poly phone drops root-caused and closed; whole-network smoothing plan built (dry-run only).** Phone drops = intentional pfSense reboot on 2026-06-16 22:38 MST (transient, one-time, resolved). DHCP server healthy (1241 ACK/1 NAK/0 no-free-leases, read directly from plain-text log). Per-room /28 HIPAA segmentation confirmed intentional + healthy. Produced prioritized smoothing plan (Phase 0 + Phase 1 radio_table PUT). Nothing applied. Hard rule established: no Cascades prod-infra changes without discussing + explicit per-change go. |
|
||||
| 2026-06-17 | **CS-SERVER drive review (advisory).** Confirmed surviving drive: Hitachi HTS545032B9A300 (0:0:2, 320 GB SATA, 5400 RPM). Recommended replacement drives: 2x 480 GB enterprise SATA SSD (e.g. Solidigm D3-S4520 or Samsung PM893), hot-swap on R610 backplane. No server-side commands run; gated on backup verification. |
|
||||
| 2026-06-17 | **Power outage -- full site down + recovery.** All 77 APs + 12 switches disconnected. Root cause: pfSense on UPS surge-only side (no battery) -> unclean shutdown -> ZFS OK, but duplicate dhcpd + 2nd-floor switch one-way L2 forwarding. Howard: killed duplicate dhcpd + clean restart remotely. Mike: moved pfSense to battery outlets, restored config from on-box auto-backup (VLAN30 intact), reset+re-adopted Switch 2nd Floor #2 (USL24PB), rebooted Cox modem. Network fully restored. Separately: 5GHz auto-channel applied (co-channel 25->30, worse). pfSense config vaulted. Pre-existing plaintext Synology signin credential found in vault history (commit 1fbc0e1) -- encrypted go-forward; needs rotation. Incident report: `clients/cascades-tucson/reports/2026-06-17-power-outage-incident.md`. |
|
||||
| 2026-06-18 | **Power outage follow-ups: OpenVPN flapping root-caused; kitchen printer casualty resolved.** OpenVPN disconnect/reconnect cycle = configured `--inactive` idle timeout (~300s) on the pfSense server, not a fault. Fix proposed (raise/disable); not applied. Kitchen thermal printer (iPad POS) would not print post-outage -- booted during DHCP-down window, cached disconnected state; fixed by power-cycle. DHCP straggler sweep: 13/13 active senders completing, 0 stuck. |
|
||||
| 2026-06-18 | **Synology Drive sync architecture diagnosed; Team Folder migration plan produced.** Current Drive sync is Sync-user My Drive only (not the real shared folders). Real NAS shares (Server 1.9 G, Management 5.5 G, Public ~50 G, SalesDept ~23 G) are not mirrored. Plan: Team Folder Download-only tasks into `D:\Shares\_SynMigration\<share>` staging; pilot on `/volume1/Server`. No changes made. |
|
||||
| 2026-06-18 | **DESKTOP-TRCIEJA (Lupe Sanchez) performance diagnosed; replace-not-remediate decision.** Root causes: (a) EOL hardware -- Gateway ZX6971 AIO, Intel i3-2120 (2011, 2C/4T), 8 GB RAM, Win11 unsupported; (b) dual real-time AV -- ACG Bitdefender (keep) + leftover Datto stack (Datto RMM/CentraStage + Datto EDR/Infocyte + bundled DattoAV) both scanning every file on a 2-core CPU under memory pressure. OneDrive ruled out (desktop is local). Howard decided: no remediation; order replacement. Another instance of the fleet-wide leftover-Datto-stack cleanup. |
|
||||
|
||||
---
|
||||
|
||||
## Compilation Notes
|
||||
|
||||
**Session logs read:** all prior sessions + 2026-06-15/16 logs (wireless RF audit, CS-SERVER RAID + VPN reset, voice VLAN plan) + 2026-06-17 KPI-dashboard-scoping log + 2 proposal docs (kpi-dashboard, kpi-dashboard-onepager) + 2 reports (unifi-full-audit, 2.4ghz-remediation-runbook) + 9 memory files. Date range: 2026-03-06 through 2026-06-17.
|
||||
**Session logs read:** all prior sessions + 2026-06-17/18 logs: voice VLAN 30 build + Poly cutover, Poly phone-drop root cause + wireless smoothing plan, power-outage recovery + 5GHz option analysis, CS-SERVER drive review, KPI dashboard scoping, power-outage follow-up (OpenVPN + printer), Synology Drive sync diagnosis, DESKTOP-TRCIEJA (Lupe Sanchez) perf diagnosis. Date range: 2026-03-06 through 2026-06-18.
|
||||
|
||||
**New this compile (2026-06-17):** added Business Applications & Reporting Systems section (9 LOB/reporting SaaS catalogued) + the proposed unified KPI dashboard (Ashley Jensen). Advisory-only session; no infrastructure changed. All RF / migration / HIPAA state unchanged from the 2026-06-16 compile.
|
||||
**New this compile (2026-06-18):**
|
||||
- Voice VLAN 30: status updated to 22/22 Poly + desktop DONE; AudioCodes 0/8 still pending. PPSK vaulted. Wired-port link-bounce pattern documented.
|
||||
- Power outage (2026-06-17): full incident documented. pfSense UPS placement rectified. Duplicate dhcpd, 2nd-floor switch L2 failure, Cox modem reboot step. Post-outage straggler pattern (power-cycle) documented. pfSense config vaulted. Synology signin credential exposure flagged (vault commit 1fbc0e1).
|
||||
- Wireless: Phase A extended overnight 2026-06-17 and over-thinned (retry 17->23.4%, satisfaction 39->30). 5GHz auto-channel made co-channel overlap worse. Both corrective plans staged (Low->Medium, Option B) but not applied. Phone drop mystery closed (intentional pfSense reboot).
|
||||
- DESKTOP-TRCIEJA (Lupe Sanchez): added to key contacts and migration table; EOL hardware + dual-AV root cause; replace decision.
|
||||
- pfSense patterns: plain-text logs (not clog), --inactive OpenVPN timeout, dirty-boot/duplicate-dhcpd recovery, post-outage stragglers.
|
||||
- Synology Drive sync architecture documented (current scope is Sync-user My Drive only; Team Folder migration plan for department shares).
|
||||
- Active Work: updated with all new non-Syncro follow-ups (Lupe replacement, Synology credential rotation, AudioCodes cutover, Phase 0+1 wireless, OpenVPN fix, AutoConfigBackup, Team Folder migration, outage stragglers).
|
||||
- Sources: 7 new session-log paths appended (2026-06-17-poly-phone-drops, 2026-06-17-power-outage, 2026-06-17-cs-server-drive-review, 2026-06-17-voice-vlan30-build, 2026-06-18-outage-followup-openvpn-printer, 2026-06-18-synology-drive-sync, 2026-06-18-lupesanchez-perf-diag); voice-phone-inventory.md added.
|
||||
- Billing: hours 55.75 (unchanged; no draws from 2026-06-17/18 sessions); date updated to 2026-06-18.
|
||||
|
||||
**Client folder:** `clients/cascades-tucson/` (NOT `clients/cascades/` -- that directory does not exist).
|
||||
|
||||
@@ -478,29 +541,28 @@ Primary active project as of 2026-05-24: dept-by-dept domain migration (Syncro #
|
||||
- Audit retention infra -- approved 2026-04-29, not yet built
|
||||
- dunedolly21@gmail.com guest invite -- confirm with Lauren
|
||||
- Windows MDM auto-enroll scope -- confirm in portal (Entra -> Devices -> Mobility -> Microsoft Intune -> MDM user scope)
|
||||
- #32370 -- verify/likely closed; Syncro live 2026-06-16 shows 0 open tickets
|
||||
- #32370 -- verify/likely closed; Syncro live 2026-06-18 shows 0 open tickets
|
||||
- Edge UNC download bug fix path -- no fix applied as of 2026-06-08; decision pending Howard
|
||||
- ALIS BAA with Medtelligent -- not yet verified; confirm with Meredith (also: does ALIS offer a built-in analytics / data feed? relevant to the KPI dashboard)
|
||||
- KPI dashboard (Ashley Jensen) -- parked; need day-one KPIs, data-freshness need, POS product + Focus HR plan before scoping a build
|
||||
- JD Martin (jd.martin@cascadestucson.com) -- confirmed Syncro contact; role not yet documented
|
||||
- CS-SERVER cloud backup: verify first full completes, confirm image-based / bare-metal + system-state, set retention; only then proceed with RAID remediation
|
||||
- NURSESTATION-PC: verify `CSC - Caregiver Device Lockdown` GPO activated (requires reboot; verify lock@3min, 90s warning, sign-out@15min, never-sleep)
|
||||
- Wireless RF: Floors 1-3, 5-6 power-down + Phase C disables pending scope go-ahead from Howard
|
||||
- Wireless RF: Phase 0 + Phase 1 (Low->Medium + Option B) pending scope go-ahead from Howard; windowed evening session needed
|
||||
- Christine (room 515, Poly phone 10.0.30.220) -- last name noted as "~Nyuda -- VERIFY"
|
||||
- UPS coverage: confirm all core infra + PoE switches are battery-backed (pfSense rectified; others not confirmed)
|
||||
- Netgate AutoConfigBackup: not yet enabled
|
||||
- Synology signin credential rotation: exposed in vault history commit 1fbc0e1; encrypted go-forward but must rotate
|
||||
|
||||
**Resolved since last compile (2026-06-15 -> 2026-06-16):**
|
||||
- Howard-Home LAN shadow: resolved 2026-06-16 (renumbered to 10.137.42.0/24; Cascades 192.168.0.x now reachable over VPN)
|
||||
- pfSense version: confirmed pfSense Plus 25.07-RELEASE (was listed as "pfSense 24.0")
|
||||
- pfSense gateway: ruled out as WiFi factor (health check 2026-06-16)
|
||||
- DFS empirically clean: dfs-check.sh confirmed ZERO radar events fleet-wide (was theoretical concern)
|
||||
- Floor-4 2.4 GHz power-down: applied (first production RF change; retry 13.2->9.5%)
|
||||
- unifi-wifi skill: feature-complete as of 2026-06-16 (WiFi/switch/gateway/pfSense-SSH, all gated writes validated)
|
||||
|
||||
**Carried forward from prior compile:**
|
||||
- Wireless controller access unblocked (2026-06-15): SSH/Mongo + RW API + AP creds all vaulted; live RF audit completed; tuning plan staged
|
||||
- CS-SERVER RAID degraded + cloud backup installed (2026-06-15)
|
||||
- Voice VLAN VLAN 30 plan + runbook (2026-06-16); vendor email sent; awaiting confirm
|
||||
- CSCNet SSID correction: shared PPSK SSID (~230 per-key->network mappings), not "staff/VLAN-20"
|
||||
- Shared mailboxes grievances@ + Surveys@ created and delegated (2026-06-12): ticket #32417 Invoiced; prepay block 55.75h
|
||||
**Resolved since last compile (2026-06-17 -> 2026-06-18):**
|
||||
- Poly phone drops: closed (intentional 2026-06-16 pfSense reboot; transient)
|
||||
- Voice VLAN 30: built, verified, Poly cutover complete (22/22); Vertical desktop done
|
||||
- pfSense 25.07 log format: documented (plain text, not clog)
|
||||
- pfSense config backup: vaulted post-restore (2026-06-17)
|
||||
- pfSense on battery-backed UPS: rectified (Mike, 2026-06-17)
|
||||
- Kitchen printer / post-outage straggler: resolved by power-cycle (2026-06-18)
|
||||
- Synology Drive sync architecture: diagnosed (2026-06-18; Team Folder plan produced)
|
||||
- DESKTOP-TRCIEJA root cause: identified (dual AV + EOL hardware); decision made (replace)
|
||||
|
||||
## Backlinks
|
||||
|
||||
|
||||
55
wiki/clients/darrell-delphen.md
Normal file
55
wiki/clients/darrell-delphen.md
Normal file
@@ -0,0 +1,55 @@
|
||||
---
|
||||
type: client
|
||||
name: darrell-delphen
|
||||
display_name: Darrell Delphen
|
||||
last_compiled: 2026-06-18
|
||||
compiled_by: GURU-5070/claude-main
|
||||
sources:
|
||||
- clients/darrell-delphen/session-logs/2026-06/2026-06-18-mike-email-link-sni-block.md
|
||||
backlinks: []
|
||||
---
|
||||
|
||||
# Darrell Delphen
|
||||
|
||||
## Profile
|
||||
- **Contract type:** Break-fix (no prepaid block)
|
||||
- **Key contacts:** Darrell Delphen
|
||||
- **Billing rate:** $150/hr remote (most recent labor)
|
||||
- **Location:** Yantis, TX
|
||||
- **Syncro customer ID:** 35996725
|
||||
- **Managed devices:** GuruRMM agent on DDDOffice072023 (0 assets recorded in Syncro)
|
||||
|
||||
## Infrastructure
|
||||
|
||||
### Servers & Services
|
||||
| Host | IP | Role | OS | Notes |
|
||||
|---|---|---|---|---|
|
||||
| DDDOffice072023 | LAN behind 192.168.1.1 | Workstation | Windows | GuruRMM agent `000ed57d-fd05-4001-871c-244f43155c16` (v0.6.66); ISP egress 167.89.210.225 |
|
||||
|
||||
### Email & Identity
|
||||
- **Mail security:** Intermedia Email Protection (rewrites links to `url.emailprotection.link`). Mailbox accessed via Outlook. (verify tenant/host details)
|
||||
|
||||
### Network
|
||||
- **ISP / WAN:** ISP-provided/managed **Extreme EXOS** gateway (LAN gateway 192.168.1.1, egress 167.89.210.225). Client has no login to this device — changes require ISP escalation.
|
||||
- **Firewall:** the EXOS gateway runs ISP-side web/URL filtering (the "NetIQ" feature) capable of SNI-based TLS inspection.
|
||||
- **VPN:** (verify — none known)
|
||||
|
||||
## Access
|
||||
- Remote management: GuruRMM (agent on DDDOffice072023)
|
||||
- Vault path: (verify — no client credentials vaulted)
|
||||
|
||||
## Patterns & Known Issues
|
||||
|
||||
- **ISP gateway SNI filtering breaks Intermedia/Outlook email links.** The ISP-managed Extreme EXOS gateway's "NetIQ" URL-filtering feature has SNI-intercepted and reset TLS to `url.emailprotection.link` (Intermedia's link-rewriter), producing `ERR_CONNECTION_ABORTED` / SChannel `0x80090326` (SEC_E_ILLEGAL_MESSAGE) on Outlook links while Gmail links (Google redirector) worked. The client cannot change this device — resolution requires an ISP ticket. If it recurs, suspect the "NetIQ" feature being re-enabled and confirm with an SNI-varied handshake test against `199.193.205.140`.
|
||||
|
||||
## Active Work
|
||||
|
||||
*No open tickets in Syncro as of 2026-06-18.*
|
||||
|
||||
## History Highlights
|
||||
|
||||
- **2026-06-18** — Diagnosed Outlook email links failing (`ERR_CONNECTION_ABORTED`). Root cause: ISP EXOS gateway "NetIQ" feature SNI-blocking Intermedia's `url.emailprotection.link` rewriter. Deployed Cloudflare WARP as interim bypass; ISP disabled the feature for the permanent fix; verified native path and removed WARP. Billed 1.0h remote ($150, Syncro #32437 / invoice #67853).
|
||||
|
||||
## Backlinks
|
||||
|
||||
*(none yet)*
|
||||
91
wiki/clients/patriot-internal-medicine.md
Normal file
91
wiki/clients/patriot-internal-medicine.md
Normal file
@@ -0,0 +1,91 @@
|
||||
---
|
||||
type: client
|
||||
name: patriot-internal-medicine
|
||||
display_name: Patriot Internal Medicine
|
||||
last_compiled: 2026-06-18
|
||||
compiled_by: Mikes-MacBook-Air/claude-main
|
||||
sources:
|
||||
- session-logs/2026-06/2026-06-18-mike-acg-website-phase3b-pim-onboarding.md
|
||||
- clients/patriot-internal-medicine/gururmm-site-tucson.sops.yaml (vault)
|
||||
- clients/patriot-internal-medicine/gururmm-site-sonoita.sops.yaml (vault)
|
||||
backlinks:
|
||||
- projects/gururmm
|
||||
---
|
||||
|
||||
# Patriot Internal Medicine
|
||||
|
||||
Medical practice with two locations: Tucson (primary) and Sonoita. Onboarded to GuruRMM on 2026-06-18. No agents deployed yet — enrollment pending.
|
||||
|
||||
---
|
||||
|
||||
## Profile
|
||||
|
||||
- **Contract type:** [unverified — not documented]
|
||||
- **Key contacts:** [not yet captured]
|
||||
- **Billing rate:** [unverified — not documented]
|
||||
- **Syncro customer ID:** [not yet documented]
|
||||
- **Active tickets:** None
|
||||
|
||||
---
|
||||
|
||||
## Infrastructure
|
||||
|
||||
### Servers & Services
|
||||
|
||||
No servers or infrastructure documented yet. Medical practice with two physical locations.
|
||||
|
||||
### Network
|
||||
|
||||
No network details captured yet.
|
||||
|
||||
---
|
||||
|
||||
## GuruRMM Enrollment
|
||||
|
||||
- **Client name in RMM:** Patriot Internal Medicine
|
||||
- **Client ID:** `9c69dc3c-5fa1-4878-8b9d-f69c36df2a67`
|
||||
|
||||
**Sites:**
|
||||
|
||||
| Site Name | Site ID | Site Code | Enrollment URL | Vault Path |
|
||||
|---|---|---|---|---|
|
||||
| Tucson | `8c4fcf01-5fca-4a40-83e7-75b87f658b72` | `NORTH-WOLF-6270` | https://rmm.azcomputerguru.com/install/NORTH-WOLF-6270 | `clients/patriot-internal-medicine/gururmm-site-tucson.sops.yaml` |
|
||||
| Sonoita | `2ec7dcc5-91c3-4182-bfba-cd2cf9a43349` | `LIGHT-HARBOR-9617` | https://rmm.azcomputerguru.com/install/LIGHT-HARBOR-9617 | `clients/patriot-internal-medicine/gururmm-site-sonoita.sops.yaml` |
|
||||
|
||||
**Enrolled agents:** None — no agents deployed yet as of 2026-06-18.
|
||||
|
||||
---
|
||||
|
||||
## Access
|
||||
|
||||
- **GuruRMM (external):** https://rmm.azcomputerguru.com
|
||||
- **Vault paths:**
|
||||
- `clients/patriot-internal-medicine/gururmm-site-tucson.sops.yaml` — Tucson site enrollment key and API credentials
|
||||
- `clients/patriot-internal-medicine/gururmm-site-sonoita.sops.yaml` — Sonoita site enrollment key and API credentials
|
||||
|
||||
---
|
||||
|
||||
## Active Work
|
||||
|
||||
As of 2026-06-18:
|
||||
|
||||
- **[OPEN] Deploy agents:** Install GuruRMM agents at Tucson and Sonoita endpoints using the enrollment URLs above
|
||||
- **[OPEN] Run onboarding diagnostics:** After first agents enroll, run baseline security and infrastructure scans
|
||||
- **[OPEN] Capture contact information:** Document primary contacts, phone numbers, and escalation paths
|
||||
- **[OPEN] Determine contract type:** Verify prepaid block, managed services, or break-fix arrangement
|
||||
- **[OPEN] Infrastructure discovery:** Document servers, network topology, line-of-business applications, backup systems
|
||||
- **[OPEN] M365 tenant verification:** Check if practice has Microsoft 365 tenant and onboard remediation tools if present
|
||||
|
||||
---
|
||||
|
||||
## History Highlights
|
||||
|
||||
| Date | Event |
|
||||
|---|---|
|
||||
| 2026-06-18 | Client "Patriot Internal Medicine" created in GuruRMM (ID: 9c69dc3c). Two sites provisioned: Tucson (NORTH-WOLF-6270) and Sonoita (LIGHT-HARBOR-9617). Enrollment keys vaulted. Discord alerts posted to #dev-alerts. |
|
||||
|
||||
---
|
||||
|
||||
## Backlinks
|
||||
|
||||
- [[projects/gururmm]] — Client and sites provisioned; no agents enrolled yet
|
||||
@@ -1,7 +1,7 @@
|
||||
# Wiki Index
|
||||
|
||||
Last updated: 2026-06-17
|
||||
Compiled by: HOWARD-HOME/claude-main
|
||||
Last updated: 2026-06-18
|
||||
Compiled by: GURU-5070/claude-main
|
||||
|
||||
This wiki is LLM-maintained. Do not edit articles manually — run `/wiki-compile` to update.
|
||||
Run `/wiki-lint` to check for stale entries and broken backlinks.
|
||||
@@ -18,19 +18,21 @@ Run `/wiki-lint` to check for stale entries and broken backlinks.
|
||||
|
||||
| Article | Summary | Last Compiled |
|
||||
|---|---|---|
|
||||
| [Cascades of Tucson](clients/cascades-tucson.md) | Prepaid block $175/hr, **55.75 hrs remaining** (live 2026-06-16); senior living; active domain migration + HIPAA compliance project; single DC on aging R610 hardware; caregiver restricted-access model PROVEN 2026-06-05: Hybrid Entra Join + CA allow-list + ALIS SSO validated on NURSESTATION-PC/pilot.test; GPO `CSC - Caregiver Workstation` (shortcuts + printers) built + validated; GPO `CSC - Caregiver Device Lockdown` deployed (HIPAA auto-logoff, activates on reboot); INTUNE_A PendingInput tenant-wide (MS case open; GPO path used instead); folder-redirection root cause fixed 2026-06-08 (fdeploy.ini); shared mailboxes grievances@/Surveys@ created + delegated 2026-06-12 (#32417); Monday cutover to real caregivers pending; #32383 (bill.com/BOK chris.knight) Resolved; UniFi wifi RF (77 U7-Pro APs/~587 clients via UOS controller): 2.4GHz over-coverage = primary pain; pfSense ruled out as cause; Floor-4 power-down pilot applied 2026-06-16 (retry 13.2->9.5%); coverage-thin disable plan + 2.4 remediation runbook staged; DFS empirically clean; 6GHz untapped; CS-SERVER OS RAID-1 degraded 2026-06-15 (data-loss risk; cloud backup now started); Voice VLAN (VLAN 30) consolidation planned 2026-06-16 for Vertical phones + remote desktop (CSCNet confirmed a shared PPSK SSID); KPI dashboard for Ashley Jensen scoped 2026-06-17 (Power BI + SharePoint phased plan, parked); Syncro 0 open tickets | 2026-06-17 |
|
||||
| [Cascades of Tucson](clients/cascades-tucson.md) | Prepaid block $175/hr, **55.75 hrs remaining** (live 2026-06-18); senior living; active domain migration + HIPAA compliance project; single DC on aging R610 hardware; caregiver restricted-access model PROVEN 2026-06-05: Hybrid Entra Join + CA allow-list + ALIS SSO validated on NURSESTATION-PC/pilot.test; GPO `CSC - Caregiver Workstation` (shortcuts + printers) built + validated; GPO `CSC - Caregiver Device Lockdown` deployed (HIPAA auto-logoff, activates on reboot); INTUNE_A PendingInput tenant-wide (MS case open; GPO path used instead); folder-redirection root cause fixed 2026-06-08 (fdeploy.ini); shared mailboxes grievances@/Surveys@ created + delegated 2026-06-12 (#32417); Monday cutover to real caregivers pending; #32383 (bill.com/BOK chris.knight) Resolved; UniFi wifi RF (77 U7-Pro APs/~587 clients via UOS controller): 2.4GHz over-coverage = primary pain; pfSense ruled out as cause; Floor-4 power-down pilot applied 2026-06-16 (retry 13.2->9.5%); coverage-thin disable plan + 2.4 remediation runbook staged; DFS empirically clean; 6GHz untapped; CS-SERVER OS RAID-1 degraded 2026-06-15 (data-loss risk; cloud backup now started); Voice VLAN (VLAN 30) consolidation planned 2026-06-16 for Vertical phones + remote desktop (CSCNet confirmed a shared PPSK SSID); KPI dashboard for Ashley Jensen scoped 2026-06-17 (Power BI + SharePoint phased plan, parked); Voice VLAN 30 built + 22/22 Poly cut over 2026-06-17 (AudioCodes 0/8 pending); building power outage 2026-06-17 (pfSense on UPS surge-only side) full site down + recovered; DESKTOP-TRCIEJA (Lupe Sanchez) slow Excel diagnosed 2026-06-18 = EOL i3-2120 hardware + dual real-time AV (leftover Datto stack) -> replace machine; Syncro 0 open tickets | 2026-06-18 |
|
||||
| [Dataforth Corporation](clients/dataforth.md) | Prepaid block ~$2,099/mo, 34.5 hrs remaining; signal conditioning manufacturer; 64 DOS test stations; 2025 crypto attack recovery + incomplete restore (files dropped across shares — migration-gap audit in progress); 2026-03-27 phishing incident + MFA rollout; active test datasheet pipeline project; Neptune Exchange colocated at D2; 2026-06-04 SP1366 file recovery (19/20 PDFs restored from HGHAUBNER pre-attack backup); GuruRMM fleet 13→45 agents; 2026-06-02 Syncro asset reconciliation (78→20 keep/21 flag/28 remove/9 verify); fleet-wide Syncro agent break ~2025-10-06; Bitdefender phase-off in progress | 2026-06-04 |
|
||||
| [Instrumental Music Center](clients/instrumental-music-center.md) | Prepaid block $175/hr, 12.5 hrs remaining; music retail/repair; AIMsi POS on SQL Server 2019; phantom DC causing slow logons; GuruRMM enrolled (IMC1) | 2026-05-24 |
|
||||
| [Valley Wide Plastering](clients/valleywide.md) | Prepaid block, 10 hrs remaining; plastering/stucco contractor; HP DL360 Gen10 + XenServer; VB6 app modernization project; RDWeb brute-force incident; 11 Yealink phones pending | 2026-06-14 |
|
||||
| [ACG Internal Infrastructure](clients/internal-infrastructure.md) | ACG's own hosting infra — Neptune Exchange (cert expires 2026-05-31, DkimSigner disabled), IX server, Cloudflare tunnel workaround, ACG M365 tenant gaps | 2026-05-24 |
|
||||
| [BirthBiologic](clients/birth-biologic.md) | Bio/healthcare; BB-SERVER (WS2016) GuruRMM enrolled; Datto→SharePoint migration incomplete; M365 apps partially consented | 2026-05-24 |
|
||||
| [CryoWeave](clients/cryoweave.md) | Custom cryogenic cable assemblies; cPanel on IX; website redesign + SEO project in progress; Syncro ID not documented | 2026-05-24 |
|
||||
| [Darrell Delphen](clients/darrell-delphen.md) | Break-fix residential (Yantis, TX); single Windows workstation DDDOffice072023 (GuruRMM); 2026-06-18 Outlook email links failing = ISP-managed Extreme EXOS gateway "NetIQ" SNI-filtering of Intermedia's url.emailprotection.link rewriter (WARP interim bypass, ISP disabled the feature for permanent fix); Syncro #35996725 | 2026-06-18 |
|
||||
| [Glaz-Tech Industries](clients/glaztech.md) | ~200 users, 9 locations; prepaid ~22.25 hrs; web server WWW (192.168.8.72 / 65.113.52.88) — IIS 10/VB.NET e-commerce; CRITICAL security posture: website connects to GTI-INV-SQL as sysadmin (login `tom`, named SQL login, C0 top finding) + plaintext PANs+CVV (stored by GTIware PSA, not website) + plaintext passwords + SQLi via `quo()` + XSS; apex 404 fixed + payment TLS fixed 2026-06-03; intrusion/brute-force log review 2026-06-04 (no attacker found; H5 detection blind spot confirmed — HTTP 200 on both success/failure + no failed-login logging); #32378 Waiting on Customer (assessment + reports + Appendix A delivered); M365 no MFA; SCL bypass rules for vendor DMARC + MailProtector digests | 2026-06-04 |
|
||||
| [Grabb & Durando Law Office](clients/grabb-durando.md) | Personal injury law firm; GND-SERVER GuruRMM enrolled; AI demand review app scoped ($4K–$7K); website migration pending; plaintext DB password in README needs vaulting | 2026-05-24 |
|
||||
| [Pavon](clients/pavon.md) | Former/archive client; GeoVision NVR surveillance; OwnCloud at 172.16.3.22 backed by Uranus; cron stacking fixed; Nextcloud migration deferred 3–6 months | 2026-05-24 |
|
||||
| [Rieusset Corp (Tom Sorensen)](clients/rieusset-corp.md) | Small business; email hosted on Neptune Exchange (4 mailboxes: tsorensen, tomrc, ojodeagua, csorensen @rieussetcorp.com); Mailprotector domain ID 57833; outbound via SBR Outbound.Sorensen connector; clipto.com allow rule added 2026-06-08 | 2026-06-08 |
|
||||
| [Rednour Law Offices](clients/rednour.md) | Law firm; M365 rednourlaw.com (tenant 4a4ca18a) fully onboarded 2026-05-31; all 5 ComputerGuru SPs consented; no MDE license; 3 workstations GuruRMM enrolled (FRONTDESKRECEPT/LEGALASST/REDNOURCARRIEVI); Carla Skinner renamed from Emma; prior MSP agents (ScreenConnect/Splashtop/Datto) still present; shared-drive access for Nick Pafford deferred | 2026-06-02 |
|
||||
| [Peaceful Spirit Therapeutic Massage](clients/peaceful-spirit.md) | Massage therapy practice; PST-SERVER (192.168.0.2) + 5 GuruRMM agents; L2TP/IPsec RRAS VPN complete; 2026-06-04 site-wide outage resolved (UDR Ultra reboot dropped VPN port-forward, re-added in controller); BridgettePSHomeComputer re-enrolled (new UUID 01160fc8); vault drift open (pst-admin password); Syncro 278525 (Peaceful Spirit Massage) | 2026-06-04 |
|
||||
| [Patriot Internal Medicine](clients/patriot-internal-medicine.md) | Medical practice, two locations (Tucson + Sonoita); GuruRMM client+sites provisioned 2026-06-18 (Tucson: NORTH-WOLF-6270, Sonoita: LIGHT-HARBOR-9617); no agents deployed yet; enrollment keys vaulted; infrastructure discovery pending | 2026-06-18 |
|
||||
| [Sombra Residential LLC](clients/sombra-residential.md) | Property management; Server2013 (actually WS2012 EOL, unpatched) + DESKTOP-UQRN4K3 GuruRMM enrolled; Transwiz migration artifacts cause Office credential prompts | 2026-05-24 |
|
||||
| [Stamback Septic](clients/stamback-septic.md) | Septic services; prepaid block ~3.5 hrs remaining; DESKTOP-BTR2AM3 + StambackLaptopNew GuruRMM enrolled; OneDrive identity wipe pattern documented | 2026-05-24 |
|
||||
| [BG Builders LLC](clients/bg-builders.md) | Construction; M365 bgbuildersllc.com (CIPP: sonorangreenllc.com); terminated employee (Lesley Roth) — account disabled, litigation hold, device wipes pending; no Intune | 2026-05-24 |
|
||||
@@ -107,6 +109,7 @@ Run `/wiki-lint` to check for stale entries and broken backlinks.
|
||||
| Grabb & Durando Law Office | GND-SERVER (WS2019, GuruRMM enrolled) | GuruRMM; AI demand review app (scoped) |
|
||||
| Pavon | OwnCloud VM (172.16.3.22), Uranus /Archive storage | — |
|
||||
| Peaceful Spirit | PST-SERVER (192.168.0.2, agent 87293069), UCG-PST-CC UDR Ultra (192.168.0.10 / 98.190.129.150), 4 workstations | GuruRMM |
|
||||
| Patriot Internal Medicine | Two locations (Tucson, Sonoita); no systems documented yet | GuruRMM (client+sites provisioned, no agents deployed yet) |
|
||||
| Sombra Residential LLC | Server2013 (WS2012 EOL) + DESKTOP-UQRN4K3, GuruRMM enrolled | GuruRMM |
|
||||
| Stamback Septic | DESKTOP-BTR2AM3 + StambackLaptopNew, GuruRMM enrolled | GuruRMM |
|
||||
| BG Builders LLC | M365 bgbuildersllc.com; no on-prem infra documented | — |
|
||||
|
||||
Reference in New Issue
Block a user