# TestDataDB Investigation - Missing Recent Records
**Date:** 2026-03-11
**Status:** BLOCKED - VPN down, need parser source code from AD2

## Problem
The test database on AD2 has newest `test_date` of 2026-01-19 despite daily tests being run every weekday. After a full re-import (1,028,275 -> 1,632,793 records), the max date did not change.

## Key Evidence
1. DAT files with TODAY's timestamps (2026-03-11 13:29-13:30) exist on AD2 at `C:\Shares\test\TS-1L\LOGS\5BLOG\`
2. These files were processed by the full import (604,518 new records added)
3. But `MAX(test_date)` is still 2026-01-19

## Sample DAT File Content (33-06D.DAT from 5BLOG)
```
"SCM5B33-06D  "
9.528068E-02,.9528068,.9696456,.1683873,"PASS"
.3142081,3.142081,3.155015,.1293349,"PASS"
.5374944,5.374944,5.380653,5.708694E-02,"PASS"
.7651215,7.651215,7.651233,1.811981E-04,"PASS"
.997809,9.97809,9.967015,-.1107502,"PASS"
"PASS 92.941","","PASS .16838733","PASS 2.045663E-023","PASS-2.213427E-023"
"","PASS .29938753","","PASS .79032473","PASS .05982453"
"","PASS 117.35150","","PASS-.1008325","PASS 91.444891"
"PASS 1.024068E-023",""
```

**No date anywhere in the content or filename.**

## Root Cause Hypothesis
The `multiline.js` parser at `C:\Shares\testdatadb\parsers\multiline.js` determines `test_date` from some source. Possibilities:
1. **File modification time (mtime)** - most likely since there's no date in content
2. **A date field elsewhere in larger files** - maybe we only saw a partial file
3. **A hardcoded date or fallback** - parser might have a bug

If parser uses mtime, the question is whether mtime is preserved when:
- DOS machines XCOPY files to NAS
- Sync-FromNAS.ps1 SCPs files from NAS to AD2
- SCP with -O flag may or may not preserve timestamps

## HISTLOGS vs Station Files
HISTLOGS files at `C:\Shares\test\Ate\HISTLOGS\` are the authoritative consolidated source. These may have a DIFFERENT format than the per-station DAT files. The initial import (1,030,940 records) came mostly from HISTLOGS (576K) and Recovery-TEST (454K), with only 59 from live station data.

The 604K new records from the re-import might all be from HISTLOGS/Recovery with dates up to Jan 19, while the per-station files might be producing 0 records or records with the same old dates.

## Next Steps (when VPN reconnects)
1. **READ THE PARSER:** `ssh sysadmin@AD2 "type C:\Shares\testdatadb\parsers\multiline.js"`
2. **Check a specific record:** Query DB for records from `33-06D.DAT` source file to see what test_date was assigned
3. **Check import logs:** `ssh sysadmin@AD2 "type C:\Shares\test\scripts\sync-from-nas.log"` for any import errors
4. **Verify HISTLOGS content:** Check if HISTLOGS files have different format than station files

## Deployed Fixes This Session
- Sync-FromNAS.ps1: Get-NASFileList fix (stdout deadlock), 8.3 filename filtering, SCP path escaping
- import.js: Changed INSERT OR IGNORE to INSERT OR REPLACE
- Both deployed to AD2 at C:\Shares\test\scripts\ and C:\Shares\testdatadb\database\
- Commit dd4086d (local only, not pushed - Gitea unreachable)

## Session Context
- Worked on from Windows machine (ACG-M-L5090)
- VPN went down during investigation
- Previous session summary in conversation compaction
- User said "continue to work this problem - we need to find those records"