Dataforth UI push + dedup + refactor, GuruRMM roadmap evolution, Azure signing setup

Dataforth (projects/dataforth-dos/):
- UI feature: row coloring + PUSH/RE-PUSH buttons + Website Status filter
- Database dedup to one row per SN (2.89M -> 469K rows, UNIQUE constraint added)
- Import logic handles FAIL -> PASS retest transition
- Refactored upload-to-api.js to render datasheets in-memory (dropped For_Web filesystem dep)
- Bulk pushed 170,984 records to Hoffman API
- Statistical sanity check: 100/100 stamped SNs verified on Hoffman

GuruRMM (projects/msp-tools/guru-rmm/):
- ROADMAP.md: added Terminology (5-tier hierarchy), Tunnel Channels Phase 2,
  Logging/Audit/Observability, Multi-tenancy, Modular Architecture,
  Protocol Versioning, Certificates sections + Decisions Log
- CONTEXT.md: hierarchy table, new anti-patterns (bootstrap sacred,
  no cross-module imports), revised next-steps priorities

Session logs for both projects.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

projects/dataforth-dos/datasheet-pipeline/deploy_upload_integration.py

@@ -0,0 +1,82 @@
+"""Deploy the testdatadb upload integration to AD2 + convert scheduled task from hourly -> daily.
+
+Steps:
+ 1. Backup current C:\\Shares\\testdatadb\\database\\import.js on AD2
+ 2. SFTP new upload-to-api.js + updated import.js
+ 3. node --check both on AD2 to be safe
+ 4. Restart testdatadb service to reload
+ 5. Re-register DataforthTestDatasheetUploader task as DAILY (was hourly)
+ 6. Verify task definition + show next run
+"""
+import base64, paramiko, subprocess, time, yaml
+
+pwd_raw = yaml.safe_load(subprocess.run(['sops','-d','D:/vault/clients/dataforth/ad2.sops.yaml'],
+    capture_output=True, text=True, timeout=30, check=True).stdout)['credentials']['password']
+PWD = pwd_raw  # Vault has been fixed — no more `.replace('\\','')` needed
+
+LOCAL = r'D:\claudetools\projects\dataforth-dos\datasheet-pipeline\implementation-upload\database'
+REMOTE = 'C:/Shares/testdatadb/database'
+
+c = paramiko.SSHClient(); c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+c.connect('192.168.0.6', username='sysadmin', password=PWD,
+          timeout=30, banner_timeout=45, look_for_keys=False, allow_agent=False)
+
+def ps(cmd, to=120):
+    enc = base64.b64encode(cmd.encode('utf-16-le')).decode()
+    _, o, e = c.exec_command(f'powershell -NoProfile -EncodedCommand {enc}', timeout=to)
+    return o.read().decode('utf-8','replace'), e.read().decode('utf-8','replace')
+
+print('[1] backup import.js on AD2')
+out, _ = ps(f'Copy-Item -LiteralPath "{REMOTE.replace("/","\\")}\\import.js" -Destination "{REMOTE.replace("/","\\")}\\import.js.bak-$(Get-Date -Format yyyyMMdd-HHmmss)" -Force; Get-ChildItem "{REMOTE.replace("/","\\")}" -Filter "import.js*" | Select Name,Length | Format-Table -AutoSize | Out-String')
+print(out.rstrip())
+
+print('\n[2] SFTP updated files')
+sftp = c.open_sftp()
+sftp.put(f'{LOCAL}/upload-to-api.js', f'{REMOTE}/upload-to-api.js')
+sftp.put(f'{LOCAL}/import.js', f'{REMOTE}/import.js')
+sftp.close()
+out, _ = ps(f'Get-ChildItem "{REMOTE.replace("/","\\")}" -Filter "upload-to-api.js","import.js" | Select Name,Length | Format-Table -AutoSize | Out-String')
+print(out.rstrip())
+
+print('\n[3] node --check on both')
+out, err = ps(f'cd "{REMOTE.replace("/","\\")}"; & node --check upload-to-api.js 2>&1; echo "---"; & node --check import.js 2>&1; echo "---end"')
+print(out.rstrip())
+if err.strip() and 'CLIXML' not in err: print('[stderr]', err[:300])
+
+print('\n[4] restart testdatadb service')
+out, err = ps('Restart-Service testdatadb; Start-Sleep 3; Get-Service testdatadb | Select Name,Status | Format-Table -AutoSize | Out-String', to=60)
+print(out.rstrip())
+if err.strip() and 'CLIXML' not in err: print('[stderr]', err[:300])
+
+print('\n[5] re-register scheduled task as DAILY (was hourly)')
+REG = r'''
+$taskName = 'DataforthTestDatasheetUploader'
+$scriptPath = 'C:\ProgramData\dataforth-uploader\run-pipeline.ps1'
+Unregister-ScheduledTask -TaskName $taskName -Confirm:$false -ErrorAction SilentlyContinue | Out-Null
+
+$argStr = '-NoProfile -ExecutionPolicy Bypass -File ' + '"' + $scriptPath + '"'
+$action = New-ScheduledTaskAction -Execute 'powershell.exe' -Argument $argStr -WorkingDirectory 'C:\ProgramData\dataforth-uploader'
+# Daily at 02:30 server time (quiet hours)
+$trigger = New-ScheduledTaskTrigger -Daily -At (Get-Date -Hour 2 -Minute 30 -Second 0)
+$principal = New-ScheduledTaskPrincipal -UserId 'SYSTEM' -LogonType ServiceAccount -RunLevel Highest
+$settings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries -StartWhenAvailable -ExecutionTimeLimit (New-TimeSpan -Minutes 30)
+Register-ScheduledTask -TaskName $taskName -Action $action -Trigger $trigger -Principal $principal -Settings $settings -Description 'Dataforth Test Datasheet Uploader daily fallback (primary path is import.js post-export hook)' | Out-Null
+
+Write-Host '=== registered task ==='
+(Get-ScheduledTask -TaskName $taskName).Triggers | Format-List
+(Get-ScheduledTask -TaskName $taskName).Actions | Format-List
+Write-Host '=== next run ==='
+Get-ScheduledTaskInfo -TaskName $taskName | Select LastRunTime,LastTaskResult,NextRunTime | Format-List
+'''
+# Write to file on AD2, run it
+sftp = c.open_sftp()
+with sftp.open('C:/ProgramData/dataforth-uploader/register-daily.ps1', 'w') as fh:
+    fh.write(REG)
+sftp.close()
+_, o, e = c.exec_command(r'powershell -NoProfile -ExecutionPolicy Bypass -File "C:\ProgramData\dataforth-uploader\register-daily.ps1"', timeout=60)
+print(o.read().decode('utf-8','replace'))
+err = e.read().decode('utf-8','replace')
+if err.strip() and 'CLIXML' not in err: print('[stderr]', err[:300])
+
+c.close()
+print('\n[OK] deploy complete')