Mike Swanson
733d87f20e
Dataforth (projects/dataforth-dos/): - UI feature: row coloring + PUSH/RE-PUSH buttons + Website Status filter - Database dedup to one row per SN (2.89M -> 469K rows, UNIQUE constraint added) - Import logic handles FAIL -> PASS retest transition - Refactored upload-to-api.js to render datasheets in-memory (dropped For_Web filesystem dep) - Bulk pushed 170,984 records to Hoffman API - Statistical sanity check: 100/100 stamped SNs verified on Hoffman GuruRMM (projects/msp-tools/guru-rmm/): - ROADMAP.md: added Terminology (5-tier hierarchy), Tunnel Channels Phase 2, Logging/Audit/Observability, Multi-tenancy, Modular Architecture, Protocol Versioning, Certificates sections + Decisions Log - CONTEXT.md: hierarchy table, new anti-patterns (bootstrap sacred, no cross-module imports), revised next-steps priorities Session logs for both projects. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
83 lines
4.5 KiB
Python
83 lines
4.5 KiB
Python
"""Deploy the testdatadb upload integration to AD2 + convert scheduled task from hourly -> daily.
|
|
|
|
Steps:
|
|
1. Backup current C:\\Shares\\testdatadb\\database\\import.js on AD2
|
|
2. SFTP new upload-to-api.js + updated import.js
|
|
3. node --check both on AD2 to be safe
|
|
4. Restart testdatadb service to reload
|
|
5. Re-register DataforthTestDatasheetUploader task as DAILY (was hourly)
|
|
6. Verify task definition + show next run
|
|
"""
|
|
import base64, paramiko, subprocess, time, yaml
|
|
|
|
pwd_raw = yaml.safe_load(subprocess.run(['sops','-d','D:/vault/clients/dataforth/ad2.sops.yaml'],
|
|
capture_output=True, text=True, timeout=30, check=True).stdout)['credentials']['password']
|
|
PWD = pwd_raw # Vault has been fixed — no more `.replace('\\','')` needed
|
|
|
|
LOCAL = r'D:\claudetools\projects\dataforth-dos\datasheet-pipeline\implementation-upload\database'
|
|
REMOTE = 'C:/Shares/testdatadb/database'
|
|
|
|
c = paramiko.SSHClient(); c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
|
c.connect('192.168.0.6', username='sysadmin', password=PWD,
|
|
timeout=30, banner_timeout=45, look_for_keys=False, allow_agent=False)
|
|
|
|
def ps(cmd, to=120):
|
|
enc = base64.b64encode(cmd.encode('utf-16-le')).decode()
|
|
_, o, e = c.exec_command(f'powershell -NoProfile -EncodedCommand {enc}', timeout=to)
|
|
return o.read().decode('utf-8','replace'), e.read().decode('utf-8','replace')
|
|
|
|
print('[1] backup import.js on AD2')
|
|
out, _ = ps(f'Copy-Item -LiteralPath "{REMOTE.replace("/","\\")}\\import.js" -Destination "{REMOTE.replace("/","\\")}\\import.js.bak-$(Get-Date -Format yyyyMMdd-HHmmss)" -Force; Get-ChildItem "{REMOTE.replace("/","\\")}" -Filter "import.js*" | Select Name,Length | Format-Table -AutoSize | Out-String')
|
|
print(out.rstrip())
|
|
|
|
print('\n[2] SFTP updated files')
|
|
sftp = c.open_sftp()
|
|
sftp.put(f'{LOCAL}/upload-to-api.js', f'{REMOTE}/upload-to-api.js')
|
|
sftp.put(f'{LOCAL}/import.js', f'{REMOTE}/import.js')
|
|
sftp.close()
|
|
out, _ = ps(f'Get-ChildItem "{REMOTE.replace("/","\\")}" -Filter "upload-to-api.js","import.js" | Select Name,Length | Format-Table -AutoSize | Out-String')
|
|
print(out.rstrip())
|
|
|
|
print('\n[3] node --check on both')
|
|
out, err = ps(f'cd "{REMOTE.replace("/","\\")}"; & node --check upload-to-api.js 2>&1; echo "---"; & node --check import.js 2>&1; echo "---end"')
|
|
print(out.rstrip())
|
|
if err.strip() and 'CLIXML' not in err: print('[stderr]', err[:300])
|
|
|
|
print('\n[4] restart testdatadb service')
|
|
out, err = ps('Restart-Service testdatadb; Start-Sleep 3; Get-Service testdatadb | Select Name,Status | Format-Table -AutoSize | Out-String', to=60)
|
|
print(out.rstrip())
|
|
if err.strip() and 'CLIXML' not in err: print('[stderr]', err[:300])
|
|
|
|
print('\n[5] re-register scheduled task as DAILY (was hourly)')
|
|
REG = r'''
|
|
$taskName = 'DataforthTestDatasheetUploader'
|
|
$scriptPath = 'C:\ProgramData\dataforth-uploader\run-pipeline.ps1'
|
|
Unregister-ScheduledTask -TaskName $taskName -Confirm:$false -ErrorAction SilentlyContinue | Out-Null
|
|
|
|
$argStr = '-NoProfile -ExecutionPolicy Bypass -File ' + '"' + $scriptPath + '"'
|
|
$action = New-ScheduledTaskAction -Execute 'powershell.exe' -Argument $argStr -WorkingDirectory 'C:\ProgramData\dataforth-uploader'
|
|
# Daily at 02:30 server time (quiet hours)
|
|
$trigger = New-ScheduledTaskTrigger -Daily -At (Get-Date -Hour 2 -Minute 30 -Second 0)
|
|
$principal = New-ScheduledTaskPrincipal -UserId 'SYSTEM' -LogonType ServiceAccount -RunLevel Highest
|
|
$settings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries -StartWhenAvailable -ExecutionTimeLimit (New-TimeSpan -Minutes 30)
|
|
Register-ScheduledTask -TaskName $taskName -Action $action -Trigger $trigger -Principal $principal -Settings $settings -Description 'Dataforth Test Datasheet Uploader daily fallback (primary path is import.js post-export hook)' | Out-Null
|
|
|
|
Write-Host '=== registered task ==='
|
|
(Get-ScheduledTask -TaskName $taskName).Triggers | Format-List
|
|
(Get-ScheduledTask -TaskName $taskName).Actions | Format-List
|
|
Write-Host '=== next run ==='
|
|
Get-ScheduledTaskInfo -TaskName $taskName | Select LastRunTime,LastTaskResult,NextRunTime | Format-List
|
|
'''
|
|
# Write to file on AD2, run it
|
|
sftp = c.open_sftp()
|
|
with sftp.open('C:/ProgramData/dataforth-uploader/register-daily.ps1', 'w') as fh:
|
|
fh.write(REG)
|
|
sftp.close()
|
|
_, o, e = c.exec_command(r'powershell -NoProfile -ExecutionPolicy Bypass -File "C:\ProgramData\dataforth-uploader\register-daily.ps1"', timeout=60)
|
|
print(o.read().decode('utf-8','replace'))
|
|
err = e.read().decode('utf-8','replace')
|
|
if err.strip() and 'CLIXML' not in err: print('[stderr]', err[:300])
|
|
|
|
c.close()
|
|
print('\n[OK] deploy complete')
|