Session log + DFWDS Node port + Hoffman API uploader pipeline

Built the missing piece between the test datasheet pipeline and Dataforth's
new product API. End-to-end:

- Pulled DFWDS (Dataforth Web Datasheet System) VB6 source from
  AD1\Engineering\ENGR\ATE\Test Datasheets\DFWDS to local for analysis
- Decoded its filename validation: A-J prefix decodes (A=10..J=19), all-
  numeric WO# valid (no leading 0), anything else bad
- Ported the validation + move logic to Node (dfwds-process.js)
- Built bulk uploader (upload-delta.js) for Hoffman's Swagger API
  (POST /api/v1/TestReportDataFiles/bulk with OAuth client_credentials)

Sanitized 3 prior reference scripts (fetch-server-inventory, test-scenarios,
test-upload-two) to read CF_* env vars instead of hardcoded creds.

Live drain results:
- 897 files moved Test_Datasheets -> For_Web (all valid, no renames, no
  bad), DFWDS port summary in 1.1s
- Pushed entire For_Web (7,061 files) to Hoffman API in 49.7s @ 142/s:
  Created=803 Updated=114 Unchanged=6,144 Errors=0
- Server count: 489,579 -> 490,382 (+803 net new)

Also:
- Added clients/dataforth/.gitignore to exclude plaintext Oauth.txt note
- Added clients/instrumental-music-center/docs/2026-04-13-ticket-notes.md
  (ticket write-up of 2026-04-11/12/13 IMC1 RDS removal/SQL migration work)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-14 21:06:20 -07:00
parent 72105233a2
commit dd5c5afd4b
80 changed files with 13466 additions and 0 deletions

View File

@@ -0,0 +1,34 @@
"""Probe DFWDS folder on AD1 via AD2."""
import base64, paramiko, subprocess, yaml
pwd_raw = yaml.safe_load(subprocess.run(
['sops','-d','D:/vault/clients/dataforth/ad2.sops.yaml'],
capture_output=True, text=True, timeout=30, check=True,
).stdout)['credentials']['password']
PWD = pwd_raw.replace('\\', '')
c = paramiko.SSHClient()
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
c.connect('192.168.0.6', username='sysadmin', password=PWD,
timeout=30, banner_timeout=45, look_for_keys=False, allow_agent=False)
def ps(cmd, to=180):
enc = base64.b64encode(cmd.encode('utf-16-le')).decode()
_, o, e = c.exec_command(f'powershell -NoProfile -EncodedCommand {enc}', timeout=to)
return o.read().decode('utf-8','replace'), e.read().decode('utf-8','replace')
BASE = r'\\AD1\Engineering\ENGR\ATE\Test Datasheets\DFWDS'
print(f'=== top-level: {BASE} ===')
out, err = ps(f'Get-ChildItem -LiteralPath "{BASE}" -Force | Select Name,Mode,Length,LastWriteTime | Format-Table -AutoSize | Out-String')
print(out)
print('\n=== recursive count + total bytes ===')
out, err = ps(f'$f = Get-ChildItem -LiteralPath "{BASE}" -Recurse -File -ErrorAction SilentlyContinue; "files: $($f.Count)"; "bytes: $(($f | Measure-Object Length -Sum).Sum)"')
print(out)
print('\n=== full recursive listing (first 100) ===')
out, err = ps(f'Get-ChildItem -LiteralPath "{BASE}" -Recurse -File -ErrorAction SilentlyContinue | Select-Object -First 100 FullName,Length,LastWriteTime | Format-Table -AutoSize | Out-String')
print(out[:6000])
c.close()