claudetools/projects/wrightstown-smarthome/docs/CTONW_V1.2_CHANGELOG.md

# CTONW.BAT v1.2 - Changelog

**Date:** 2026-01-19
**Version:** 1.2
**Previous Version:** 1.1
**Status:** Deployed to AD2

---

## Critical Change: Test Data Routing

### Problem Identified

The Sync-FromNAS.ps1 script on AD2 expects test data in **LOGS folders** for database import:
- Expected path: `TS-*/LOGS/8BLOG/*.DAT`, `TS-*/LOGS/DSCLOG/*.DAT`, etc.
- CTONW v1.1 uploaded to: `TS-*/ProdSW/8BDATA/*.DAT`, `TS-*/ProdSW/DSCDATA/*.DAT`

**Result:** Test data was not being imported into the database because it was in the wrong location.

### Solution: Separate Data Workflows

v1.2 separates two distinct workflows:

#### 1. Software Distribution (ProdSW) - Bidirectional
- **Purpose:** Software updates and configuration files
- **Direction:** AD2 → NAS → DOS machines (via NWTOC) AND DOS machines → NAS → AD2 (via CTONW)
- **File Types:** .BAT, .EXE, .CFG, .TXT (non-test-data)
- **Upload Target:** `T:\TS-4R\ProdSW\`
- **Download Source:** `T:\COMMON\ProdSW\` and `T:\TS-4R\ProdSW\`

#### 2. Test Data Logging (LOGS) - Unidirectional Upload Only
- **Purpose:** Test results for database import and analysis
- **Direction:** DOS machines → NAS → AD2 database (via Sync-FromNAS.ps1 PULL)
- **File Types:** .DAT files (test data)
- **Upload Target:** `T:\TS-4R\LOGS\8BLOG\`, `T:\TS-4R\LOGS\DSCLOG\`, etc.
- **Download Source:** None (test data is never downloaded back to DOS machines)

---

## Changes in v1.2

### New Variables
- Added `LOGSDIR` variable (line 83): `SET LOGSDIR=T:\%MACHINE%\LOGS`

### Updated Banner Display (Lines 130-141)
Shows both target directories for machine-specific uploads:
```
Targets: T:\TS-4R\ProdSW (programs)
         T:\TS-4R\LOGS (test data)
```

### New Directory Creation (Lines 174-177)
Creates LOGS directory structure:
```batch
IF "%TARGET%"=="MACHINE" IF NOT EXIST %LOGSDIR%\NUL MD %LOGSDIR%
```

### Progress Indicator Changed
- Was: [1/2] and [2/2]
- Now: [1/3], [2/3], and [3/3]

### Step 8: Programs Upload (Lines 202-222)
**Changed from v1.1:**
- v1.1: `XCOPY C:\ATE\*.* %TARGETDIR%\ /S /Y /Q` (all files)
- v1.2: Explicit file type filters:
  ```batch
  XCOPY C:\ATE\*.EXE %TARGETDIR%\ /S /Y /Q
  XCOPY C:\ATE\*.BAT %TARGETDIR%\ /S /Y /Q
  XCOPY C:\ATE\*.CFG %TARGETDIR%\ /S /Y /Q
  XCOPY C:\ATE\*.TXT %TARGETDIR%\ /S /Y /Q
  ```
- **Result:** Excludes .DAT files from ProdSW upload

### Step 9: Test Data Upload (Lines 234-272) - NEW
**Completely new in v1.2:**
```batch
ECHO [3/3] Uploading test data to LOGS...

REM Create log subdirectories
IF NOT EXIST %LOGSDIR%\8BLOG\NUL MD %LOGSDIR%\8BLOG
IF NOT EXIST %LOGSDIR%\DSCLOG\NUL MD %LOGSDIR%\DSCLOG
IF NOT EXIST %LOGSDIR%\HVLOG\NUL MD %LOGSDIR%\HVLOG
IF NOT EXIST %LOGSDIR%\PWRLOG\NUL MD %LOGSDIR%\PWRLOG
IF NOT EXIST %LOGSDIR%\RMSLOG\NUL MD %LOGSDIR%\RMSLOG
IF NOT EXIST %LOGSDIR%\7BLOG\NUL MD %LOGSDIR%\7BLOG

REM Upload test data files to appropriate log folders
IF EXIST C:\ATE\8BDATA\NUL XCOPY C:\ATE\8BDATA\*.DAT %LOGSDIR%\8BLOG\ /Y /Q
IF EXIST C:\ATE\DSCDATA\NUL XCOPY C:\ATE\DSCDATA\*.DAT %LOGSDIR%\DSCLOG\ /Y /Q
IF EXIST C:\ATE\HVDATA\NUL XCOPY C:\ATE\HVDATA\*.DAT %LOGSDIR%\HVLOG\ /Y /Q
IF EXIST C:\ATE\PWRDATA\NUL XCOPY C:\ATE\PWRDATA\*.DAT %LOGSDIR%\PWRLOG\ /Y /Q
IF EXIST C:\ATE\RMSDATA\NUL XCOPY C:\ATE\RMSDATA\*.DAT %LOGSDIR%\RMSLOG\ /Y /Q
IF EXIST C:\ATE\7BDATA\NUL XCOPY C:\ATE\7BDATA\*.DAT %LOGSDIR%\7BLOG\ /Y /Q
```

### Subdirectory Mapping

| Local Directory | Network Target | Purpose |
|----------------|----------------|---------|
| C:\ATE\8BDATA\ | T:\TS-4R\LOGS\8BLOG\ | 8-channel test data |
| C:\ATE\DSCDATA\ | T:\TS-4R\LOGS\DSCLOG\ | DSC test data |
| C:\ATE\HVDATA\ | T:\TS-4R\LOGS\HVLOG\ | High voltage test data |
| C:\ATE\PWRDATA\ | T:\TS-4R\LOGS\PWRLOG\ | Power test data |
| C:\ATE\RMSDATA\ | T:\TS-4R\LOGS\RMSLOG\ | RMS test data |
| C:\ATE\7BDATA\ | T:\TS-4R\LOGS\7BLOG\ | 7-channel test data |

### Updated Completion Message (Lines 282-299)
Now shows both targets for machine-specific uploads:
```
Files uploaded to:
  T:\TS-4R\ProdSW (software/config)
  T:\TS-4R\LOGS (test data for database import)
```

### New Error Handler (Lines 319-331)
Added `LOGS_DIR_ERROR` label for LOGS directory creation failures.

### Updated Cleanup (Lines 360-364)
Added `LOGSDIR` variable cleanup:
```batch
SET TARGET=
SET TARGETDIR=
SET LOGSDIR=
```

---

## Expected Behavior Changes

### Before v1.2 (BROKEN)
```
DOS Machine: CTONW
↓
NAS: T:\TS-4R\ProdSW\8BDATA\*.DAT
↓ (Sync-FromNAS.ps1 looks in LOGS, not ProdSW)
[ERROR] Test data NOT imported to database
```

### After v1.2 (FIXED)
```
DOS Machine: CTONW
↓
NAS: T:\TS-4R\LOGS\8BLOG\*.DAT
↓ (Sync-FromNAS.ps1 finds files in LOGS)
[OK] Test data imported to AD2 database
```

---

## Backward Compatibility

### Impact on Existing DOS Machines

**Before deployment of v1.2:**
- DOS machines running CTONW v1.1 upload test data to ProdSW
- Test data NOT imported to database (broken workflow)

**After deployment of v1.2:**
- DOS machines download CTONW v1.2 via NWTOC
- Running CTONW v1.2 uploads test data to LOGS
- Test data correctly imported to database (fixed workflow)

### Migration Path

1. **Deploy v1.2 to AD2** [OK] COMPLETE
2. **Sync to NAS** (automatic, within 15 minutes)
3. **DOS machines run NWTOC** (downloads v1.2)
4. **DOS machines run CTONW** (uploads to correct LOGS location)
5. **Sync-FromNAS.ps1 imports data** (automatic, every 15 minutes)

### Data in Wrong Location

If test data exists in old location (`ProdSW/8BDATA/`), it will NOT be automatically migrated. Options:

1. **Manual cleanup:** Delete old DAT files from ProdSW after confirming they're in LOGS
2. **Let it age out:** Old data in ProdSW won't cause issues, just won't be imported
3. **One-time migration script:** Could create script to move DAT files from ProdSW to LOGS (not required)

---

## Testing Recommendations

### Test on TS-4R (Pilot Machine)

1. **Deploy v1.2:**
   - Run DEPLOY.BAT if not already deployed
   - Or run NWTOC to download v1.2

2. **Test CTONW Upload:**
   ```batch
   REM Create test data
   ECHO Test data > C:\ATE\8BDATA\TEST.DAT

   REM Run CTONW
   CTONW

   REM Verify upload
   DIR T:\TS-4R\LOGS\8BLOG\TEST.DAT
   ```

3. **Verify Database Import:**
   - Wait 15 minutes for sync
   - Check AD2 database for imported test data
   - Verify DAT file removed from NAS after import

4. **Test Programs Upload:**
   ```batch
   REM Create test program
   COPY C:\DOS\EDIT.COM C:\ATE\TESTPROG.EXE

   REM Run CTONW
   CTONW

   REM Verify upload
   DIR T:\TS-4R\ProdSW\TESTPROG.EXE
   ```

---

## Sync Script Compatibility

### Sync-FromNAS.ps1 PULL Operation (Lines 138-192)

**Searches for:**
```powershell
$findCommand = "find $NAS_DATA_PATH/TS-*/LOGS -name '*.DAT' -type f -mmin -$MaxAgeMinutes"
```

**Pattern match:**
```powershell
if ($remoteFile -match "/data/test/(TS-[^/]+)/LOGS/([^/]+)/(.+\.DAT)$") {
    $station = $Matches[1]    # TS-4R
    $logType = $Matches[2]    # 8BLOG
    $fileName = $Matches[3]   # TEST.DAT
}
```

**CTONW v1.2 uploads to:**
- `T:\TS-4R\LOGS\8BLOG\TEST.DAT` (NAS path: `/data/test/TS-4R/LOGS/8BLOG/TEST.DAT`)

[OK] **Compatible** - Paths match exactly

### Sync-FromNAS.ps1 PUSH Operation (Lines 244-360)

**Handles subdirectories:**
```powershell
$prodSwFiles = Get-ChildItem -Path $prodSwPath -File -Recurse
$relativePath = $file.FullName.Substring($prodSwPath.Length + 1).Replace('\', '/')
```

[OK] **Compatible** - Programs in ProdSW subdirectories sync correctly

---

## File Size Impact

**v1.1:** 293 lines
**v1.2:** 365 lines
**Change:** +72 lines (+24.6%)

**Additions:**
- 1 new variable (LOGSDIR)
- 1 new step (test data upload)
- 6 subdirectory creations
- 6 conditional XCOPY commands
- 1 new error handler
- Updated messages and banners

---

## Production Readiness

**Status:** [OK] READY FOR PRODUCTION

**Deployment Status:**
- [OK] Deployed to AD2 (both COMMON and _COMMON)
- ⏳ Waiting for sync to NAS (within 15 minutes)
- ⏳ Pending DOS machine NWTOC downloads

**Next Steps:**
1. Wait for AD2 → NAS sync (automatic)
2. Run NWTOC on TS-4R to download v1.2
3. Test CTONW upload to verify LOGS routing
4. Monitor database for imported test data
5. Deploy to remaining ~29 DOS machines

---

**Version:** 1.2
**Deployed:** 2026-01-19
**Author:** Claude Code
**Tested:** Pending pilot deployment on TS-4R