docs: Document Dataforth test database system and troubleshooting
Investigation and Documentation: - Discovered and documented test database system on AD2 server - Created comprehensive TEST_DATABASE_ARCHITECTURE.md with full system details - Retrieved all key database files from AD2 (import.js, schema.sql, server configs) - Documented data flow: DOS machines → NAS → AD2 → SQLite → Web interface - Verified database health: 1,027,517 records, 1075 MB, dates back to 1990 Database System Architecture: - SQLite database with Node.js/Express.js web server (port 3000) - Automated import via Sync-FromNAS.ps1 (runs every 15 minutes) - 8 log types supported: DSCLOG, 5BLOG, 7BLOG, 8BLOG, PWRLOG, SCTLOG, VASLOG, SHT - FTS5 full-text search, comprehensive indexes for performance - API endpoints: search, stats, export, datasheet generation Troubleshooting Scripts Created: - Database diagnostics: check-db-simple.ps1, test-db-directly.ps1 - Server status checks: check-node-running.ps1, check-db-server.ps1 - Performance analysis: check-db-performance.ps1, check-wal-files.ps1 - API testing: test-api-endpoint.ps1, test-query.js - Import monitoring: check-new-records.ps1 - Database optimization attempts: api-js-optimized.js, api-js-fixed.js - Deployment scripts: deploy-db-optimization.ps1, deploy-db-fix.ps1, restore-original.ps1 Key Findings: - Database file healthy and queryable (verified with test-query.js) - Node.js server not running (port 3000 closed) - root cause of web interface issues - Database last updated 8 days ago (01/13/2026) - automated sync may be broken - Attempted performance optimizations (WAL mode) incompatible with readonly connections - Original api.js restored from backup after optimization conflicts Retrieved Documentation: - QUICKSTART-retrieved.md: Quick start guide for database server - SESSION_NOTES-retrieved.md: Complete session notes from database creation - Sync-FromNAS-retrieved.ps1: Full sync script with database import logic - import-js-retrieved.js: Node.js import script (12,774 bytes) - schema-retrieved.sql: SQLite schema with FTS5 triggers - server-js-retrieved.js: Express.js server configuration - api-js-retrieved.js: API routes and endpoints - package-retrieved.json: Node.js dependencies Action Items Identified: 1. Start Node.js server on AD2 to restore web interface functionality 2. Investigate why automated sync hasn't updated database in 8 days 3. Check Windows Task Scheduler for Sync-FromNAS.ps1 scheduled task 4. Run manual import to catch up on 8 days of test data if needed Technical Details: - Database path: C:\Shares\testdatadb\database\testdata.db - Web interface: http://192.168.0.6:3000 (when running) - Database size: 1075.14 MB (1,127,362,560 bytes) - Total records: 1,027,517 (slight variance from original 1,030,940) - Pass rate: 99.82% (1,029,046 passed, 1,888 failed) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
107
deploy-db-optimization.ps1
Normal file
107
deploy-db-optimization.ps1
Normal file
@@ -0,0 +1,107 @@
|
||||
# Deploy Database Performance Optimizations to AD2
|
||||
$password = ConvertTo-SecureString 'Paper123!@#' -AsPlainText -Force
|
||||
$cred = New-Object System.Management.Automation.PSCredential('INTRANET\sysadmin', $password)
|
||||
|
||||
Write-Host "========================================" -ForegroundColor Cyan
|
||||
Write-Host "Test Database Performance Optimization" -ForegroundColor Cyan
|
||||
Write-Host "========================================`n" -ForegroundColor Cyan
|
||||
|
||||
# Step 1: Mount AD2 share
|
||||
Write-Host "[1/6] Mounting AD2 C$ share..." -ForegroundColor Green
|
||||
New-PSDrive -Name AD2 -PSProvider FileSystem -Root "\\192.168.0.6\C$" -Credential $cred -ErrorAction Stop | Out-Null
|
||||
Write-Host " [OK] Share mounted" -ForegroundColor Green
|
||||
|
||||
# Step 2: Backup existing api.js
|
||||
Write-Host "`n[2/6] Backing up existing api.js..." -ForegroundColor Green
|
||||
$timestamp = Get-Date -Format "yyyy-MM-dd-HHmmss"
|
||||
$backupPath = "AD2:\Shares\testdatadb\routes\api.js.backup-$timestamp"
|
||||
Copy-Item "AD2:\Shares\testdatadb\routes\api.js" $backupPath
|
||||
Write-Host " [OK] Backup created: api.js.backup-$timestamp" -ForegroundColor Green
|
||||
|
||||
# Step 3: Deploy optimized api.js
|
||||
Write-Host "`n[3/6] Deploying optimized api.js..." -ForegroundColor Green
|
||||
$optimizedContent = Get-Content "D:\ClaudeTools\api-js-optimized.js" -Raw
|
||||
$optimizedContent | Set-Content "AD2:\Shares\testdatadb\routes\api.js" -Encoding UTF8
|
||||
Write-Host " [OK] Optimized api.js deployed" -ForegroundColor Green
|
||||
|
||||
# Step 4: Stop Node.js server
|
||||
Write-Host "`n[4/6] Stopping Node.js server..." -ForegroundColor Yellow
|
||||
try {
|
||||
Invoke-Command -ComputerName 192.168.0.6 -Credential $cred -ScriptBlock {
|
||||
$nodeProcs = Get-Process node -ErrorAction SilentlyContinue
|
||||
if ($nodeProcs) {
|
||||
$nodeProcs | ForEach-Object {
|
||||
Write-Host " Stopping process ID: $($_.Id)"
|
||||
Stop-Process -Id $_.Id -Force
|
||||
}
|
||||
Start-Sleep -Seconds 2
|
||||
Write-Host " [OK] Node.js processes stopped"
|
||||
} else {
|
||||
Write-Host " [INFO] No Node.js process found"
|
||||
}
|
||||
} -ErrorAction Stop
|
||||
} catch {
|
||||
Write-Host " [WARNING] Could not stop via WinRM: $($_.Exception.Message)" -ForegroundColor Yellow
|
||||
Write-Host " [ACTION] You may need to stop the server manually on AD2" -ForegroundColor Yellow
|
||||
}
|
||||
|
||||
# Step 5: Start Node.js server
|
||||
Write-Host "`n[5/6] Starting Node.js server..." -ForegroundColor Green
|
||||
try {
|
||||
Invoke-Command -ComputerName 192.168.0.6 -Credential $cred -ScriptBlock {
|
||||
Set-Location "C:\Shares\testdatadb"
|
||||
|
||||
# Start Node.js in background
|
||||
$startInfo = New-Object System.Diagnostics.ProcessStartInfo
|
||||
$startInfo.FileName = "node"
|
||||
$startInfo.Arguments = "server.js"
|
||||
$startInfo.WorkingDirectory = "C:\Shares\testdatadb"
|
||||
$startInfo.UseShellExecute = $false
|
||||
$startInfo.RedirectStandardOutput = $true
|
||||
$startInfo.RedirectStandardError = $true
|
||||
$startInfo.CreateNoWindow = $true
|
||||
|
||||
$process = [System.Diagnostics.Process]::Start($startInfo)
|
||||
Start-Sleep -Seconds 3
|
||||
|
||||
if (!$process.HasExited) {
|
||||
Write-Host " [OK] Server started (PID: $($process.Id))"
|
||||
} else {
|
||||
Write-Host " [ERROR] Server failed to start"
|
||||
}
|
||||
} -ErrorAction Stop
|
||||
} catch {
|
||||
Write-Host " [WARNING] Could not start via WinRM: $($_.Exception.Message)" -ForegroundColor Yellow
|
||||
Write-Host " [ACTION] Please start manually: cd C:\Shares\testdatadb && node server.js" -ForegroundColor Yellow
|
||||
}
|
||||
|
||||
# Step 6: Test connectivity
|
||||
Write-Host "`n[6/6] Testing server connectivity..." -ForegroundColor Green
|
||||
Start-Sleep -Seconds 2
|
||||
|
||||
$portTest = Test-NetConnection -ComputerName 192.168.0.6 -Port 3000 -WarningAction SilentlyContinue -InformationLevel Quiet
|
||||
if ($portTest) {
|
||||
Write-Host " [OK] Port 3000 is accessible" -ForegroundColor Green
|
||||
} else {
|
||||
Write-Host " [ERROR] Port 3000 is not accessible - server may not have started" -ForegroundColor Red
|
||||
}
|
||||
|
||||
# Cleanup
|
||||
Remove-PSDrive -Name AD2 -ErrorAction SilentlyContinue
|
||||
|
||||
Write-Host "`n========================================" -ForegroundColor Cyan
|
||||
Write-Host "Deployment Summary" -ForegroundColor Cyan
|
||||
Write-Host "========================================" -ForegroundColor Cyan
|
||||
Write-Host "[OK] Backup created" -ForegroundColor Green
|
||||
Write-Host "[OK] Optimized code deployed" -ForegroundColor Green
|
||||
Write-Host "`nOptimizations Applied:" -ForegroundColor Cyan
|
||||
Write-Host " - Connection timeout: 10 seconds" -ForegroundColor Cyan
|
||||
Write-Host " - WAL mode: Enabled (better concurrency)" -ForegroundColor Cyan
|
||||
Write-Host " - Cache size: 64MB" -ForegroundColor Cyan
|
||||
Write-Host " - Memory-mapped I/O: 256MB" -ForegroundColor Cyan
|
||||
Write-Host " - Synchronous mode: NORMAL (faster, safe)" -ForegroundColor Cyan
|
||||
Write-Host "`nWeb Interface: http://192.168.0.6:3000" -ForegroundColor Green
|
||||
Write-Host "`nNext Steps (Optional):" -ForegroundColor Yellow
|
||||
Write-Host " - Run VACUUM to optimize database" -ForegroundColor Yellow
|
||||
Write-Host " - Test queries via web interface" -ForegroundColor Yellow
|
||||
Write-Host "========================================`n" -ForegroundColor Cyan
|
||||
Reference in New Issue
Block a user