docs: Document Dataforth test database system and troubleshooting

Investigation and Documentation:
- Discovered and documented test database system on AD2 server
- Created comprehensive TEST_DATABASE_ARCHITECTURE.md with full system details
- Retrieved all key database files from AD2 (import.js, schema.sql, server configs)
- Documented data flow: DOS machines → NAS → AD2 → SQLite → Web interface
- Verified database health: 1,027,517 records, 1075 MB, dates back to 1990

Database System Architecture:
- SQLite database with Node.js/Express.js web server (port 3000)
- Automated import via Sync-FromNAS.ps1 (runs every 15 minutes)
- 8 log types supported: DSCLOG, 5BLOG, 7BLOG, 8BLOG, PWRLOG, SCTLOG, VASLOG, SHT
- FTS5 full-text search, comprehensive indexes for performance
- API endpoints: search, stats, export, datasheet generation

Troubleshooting Scripts Created:
- Database diagnostics: check-db-simple.ps1, test-db-directly.ps1
- Server status checks: check-node-running.ps1, check-db-server.ps1
- Performance analysis: check-db-performance.ps1, check-wal-files.ps1
- API testing: test-api-endpoint.ps1, test-query.js
- Import monitoring: check-new-records.ps1
- Database optimization attempts: api-js-optimized.js, api-js-fixed.js
- Deployment scripts: deploy-db-optimization.ps1, deploy-db-fix.ps1, restore-original.ps1

Key Findings:
- Database file healthy and queryable (verified with test-query.js)
- Node.js server not running (port 3000 closed) - root cause of web interface issues
- Database last updated 8 days ago (01/13/2026) - automated sync may be broken
- Attempted performance optimizations (WAL mode) incompatible with readonly connections
- Original api.js restored from backup after optimization conflicts

Retrieved Documentation:
- QUICKSTART-retrieved.md: Quick start guide for database server
- SESSION_NOTES-retrieved.md: Complete session notes from database creation
- Sync-FromNAS-retrieved.ps1: Full sync script with database import logic
- import-js-retrieved.js: Node.js import script (12,774 bytes)
- schema-retrieved.sql: SQLite schema with FTS5 triggers
- server-js-retrieved.js: Express.js server configuration
- api-js-retrieved.js: API routes and endpoints
- package-retrieved.json: Node.js dependencies

Action Items Identified:
1. Start Node.js server on AD2 to restore web interface functionality
2. Investigate why automated sync hasn't updated database in 8 days
3. Check Windows Task Scheduler for Sync-FromNAS.ps1 scheduled task
4. Run manual import to catch up on 8 days of test data if needed

Technical Details:
- Database path: C:\Shares\testdatadb\database\testdata.db
- Web interface: http://192.168.0.6:3000 (when running)
- Database size: 1075.14 MB (1,127,362,560 bytes)
- Total records: 1,027,517 (slight variance from original 1,030,940)
- Pass rate: 99.82% (1,029,046 passed, 1,888 failed)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

test-db-directly.ps1

@@ -0,0 +1,72 @@
+# Test database directly to see if it's corrupted or locked
+$password = ConvertTo-SecureString 'Paper123!@#' -AsPlainText -Force
+$cred = New-Object System.Management.Automation.PSCredential('INTRANET\sysadmin', $password)
+
+Write-Host "[OK] Mounting AD2 C$ share..." -ForegroundColor Green
+New-PSDrive -Name AD2 -PSProvider FileSystem -Root "\\192.168.0.6\C$" -Credential $cred -ErrorAction Stop | Out-Null
+
+$dbPath = "AD2:\Shares\testdatadb\database\testdata.db"
+
+Write-Host "[OK] Testing database file access..." -ForegroundColor Green
+
+# Test 1: File exists and readable
+if (Test-Path $dbPath) {
+    Write-Host "  [OK] Database file exists" -ForegroundColor Green
+
+    $dbFile = Get-Item $dbPath
+    Write-Host "  [OK] Size: $([math]::Round($dbFile.Length/1MB,2)) MB" -ForegroundColor Cyan
+    Write-Host "  [OK] Modified: $($dbFile.LastWriteTime)" -ForegroundColor Cyan
+} else {
+    Write-Host "  [ERROR] Database file not found!" -ForegroundColor Red
+    Remove-PSDrive -Name AD2
+    exit 1
+}
+
+# Test 2: Check file lock
+Write-Host "`n[OK] Checking if file is locked..." -ForegroundColor Green
+try {
+    $stream = [System.IO.File]::Open($dbFile.FullName, 'Open', 'Read', 'ReadWrite')
+    $stream.Close()
+    Write-Host "  [OK] File is not locked" -ForegroundColor Green
+} catch {
+    Write-Host "  [WARNING] File may be locked by another process" -ForegroundColor Yellow
+    Write-Host "  Error: $($_.Exception.Message)" -ForegroundColor Yellow
+}
+
+# Test 3: Check file header (SQLite magic bytes)
+Write-Host "`n[OK] Checking SQLite file header..." -ForegroundColor Green
+try {
+    $bytes = [System.IO.File]::ReadAllBytes($dbFile.FullName) | Select-Object -First 16
+    $header = [System.Text.Encoding]::ASCII.GetString($bytes)
+
+    if ($header.StartsWith("SQLite format 3")) {
+        Write-Host "  [OK] Valid SQLite database header detected" -ForegroundColor Green
+    } else {
+        Write-Host "  [ERROR] Invalid SQLite header! Database may be corrupted" -ForegroundColor Red
+        Write-Host "  Header bytes: $($bytes -join ' ')" -ForegroundColor Yellow
+    }
+} catch {
+    Write-Host "  [ERROR] Cannot read database file: $($_.Exception.Message)" -ForegroundColor Red
+}
+
+# Test 4: Check permissions
+Write-Host "`n[OK] Checking file permissions..." -ForegroundColor Green
+$acl = Get-Acl $dbPath
+Write-Host "  Owner: $($acl.Owner)" -ForegroundColor Cyan
+Write-Host "  Access Rules:" -ForegroundColor Cyan
+$acl.Access | ForEach-Object {
+    Write-Host "    $($_.IdentityReference): $($_.FileSystemRights) ($($_.AccessControlType))" -ForegroundColor Gray
+}
+
+# Test 5: Check if there are journal files
+Write-Host "`n[OK] Checking for journal files..." -ForegroundColor Green
+$journalFile = Get-Item "$($dbFile.DirectoryName)\testdata.db-journal" -ErrorAction SilentlyContinue
+if ($journalFile) {
+    Write-Host "  [FOUND] Journal file exists: $([math]::Round($journalFile.Length/1KB,2)) KB" -ForegroundColor Yellow
+    Write-Host "  [WARNING] This may indicate incomplete transaction or crash" -ForegroundColor Yellow
+} else {
+    Write-Host "  [OK] No journal file (clean state)" -ForegroundColor Green
+}
+
+Remove-PSDrive -Name AD2 -ErrorAction SilentlyContinue
+Write-Host "`n[OK] Done" -ForegroundColor Green