claudetools/projects/dataforth-dos/session-logs/2026-01-21-session.md

# Dataforth DOS - Session Log: 2026-01-21

**Project:** Dataforth DOS Update System
**Client:** Dataforth
**Duration:** ~4 hours
**Status:** DSCDATA sync fixed, CTONW upload fix, ATESYNC orchestrator added

---

## Session Summary

### What Was Accomplished

1. **Diagnosed DSCDATA Sync Issue**
   - DOS machines were getting wrong/outdated DSCDATA files
   - AD2 had newer files (Jan 2026, DSCMAIN4.DAT 65,508 bytes)
   - NAS had old files (Dec 2025, DSCMAIN4.DAT 64,395 bytes)
   - Root cause: Ate/ProdSW folder was NOT in AD2->NAS sync script

2. **Fixed DSCDATA Sync**
   - Manually copied correct files from AD2 to NAS via SCP
   - Updated Sync-FromNAS.ps1 on AD2 to include Ate/ProdSW folder
   - All 8 ATE subfolders now sync automatically

3. **Updated Batch Files**
   - NWTOC.BAT v3.5: Uses COPY instead of XCOPY, no >NUL redirects
   - DEPLOY.BAT v2.4: Uses COPY instead of XCOPY

4. **Git Checkpoint Created**
   - Commit: fd24a0c - "fix(dataforth-dos): DOS 6.22 batch file improvements and sync fix"

5. **Diagnosed Database Import Issue**
   - Test results weren't making it into testdatadb SQLite database
   - Root cause: CTONW.BAT was uploading to wrong location (ProdSW instead of LOGS)
   - testdatadb runs on AD2 port 3000, imports from LOGS folders

6. **Fixed CTONW.BAT (v3.0 -> v3.1)**
   - v3.0: Added proper upload to T:\%MACHINE%\LOGS\*LOG folders
   - v3.1: Added machine verification safeguards to prevent data overwriting
   - Now refuses to run if MACHINE variable not set
   - Logs machine name, date/time, and target path

7. **Created ATESYNC.BAT v1.0 (ARCHBAT Equivalent)**
   - Boot-time orchestrator for DOS machines
   - Calls CTONW (upload) then NWTOC (download)
   - Creates machine folder structure if missing
   - Based on TS-27 original ARCHBAT.BAT

8. **Analyzed TS-27 Golden Example**
   - TS-27 Backup/ORIG contains full machine backup
   - Compared original boot process to new setup
   - Original uses %1 parameter, new uses %MACHINE% environment variable
   - Original downloads from per-machine T:\%1\ProdSW, new uses centralized T:\COMMON\ProdSW

### Key Decisions Made

1. **Use COPY instead of XCOPY for all operations**
   - Rationale: XCOPY has unpredictable behavior in DOS 6.22
   - XCOPY can hang waiting for input even with /Y flag
   - COPY is simpler and more reliable

2. **Add IF NOT EXIST checks before MD commands**
   - Rationale: Avoids "Directory already exists" error messages
   - Cleaner output for users

3. **Update sync script to include Ate/ProdSW**
   - Rationale: This folder contains shared ATE data for all machines
   - Was overlooked in original sync script design

### Problems Encountered and Solutions

**Problem 1: DOS machines getting wrong DSCDATA files**
- Symptom: DSCMAIN4.DAT was 64,395 bytes instead of 65,508 bytes
- Investigation: Compared AD2 vs NAS file sizes and dates
- Root cause: Ate/ProdSW folder not in sync script
- Solution: Added Ate/ProdSW to Sync-FromNAS.ps1, manually synced files

**Problem 2: AD2 network connectivity intermittent**
- Symptom: SSH and SMB connections timing out
- Solution: Used multiple connection methods, eventually mounted via Finder

---

## Credentials

### AD2 (Production Server - 192.168.0.6)
- **Host:** 192.168.0.6
- **Domain:** INTRANET
- **User:** INTRANET\sysadmin
- **Password:** Paper123!@#
- **Share Access:** \\192.168.0.6\C$ (admin share)
- **Local Path:** C:\Shares\test

### D2TESTNAS (SMB1 Proxy - 192.168.0.9)
- **Host:** 192.168.0.9
- **SSH User:** root
- **SSH Auth:** ED25519 key (passwordless from Mac)
- **Web Admin:** admin / Paper123!@#-nas
- **Share Path:** /data/test (maps to T:\ on DOS machines)

### ClaudeTools (for reference)
- **Database:** 172.16.3.30:3306
- **Database Name:** claudetools
- **Database User:** claudetools
- **Database Password:** CT_e8fcd5a3952030a79ed6debae6c954ed
- **API:** http://172.16.3.30:8001

---

## Infrastructure & Servers

### AD2 File Locations
- **Software Updates:** C:\Shares\test\COMMON\ProdSW\
- **ATE Data Folders:** C:\Shares\test\Ate\ProdSW\
  - 5BDATA, 7BDATA, 8BDATA, DSCDATA, HVDATA, PWRDATA, RMSDATA, SCTDATA
- **Sync Script:** C:\Shares\test\scripts\Sync-FromNAS.ps1
- **Sync Schedule:** Every 15 minutes via Windows Task Scheduler

### NAS File Locations
- **Software Updates:** /data/test/COMMON/ProdSW/
- **ATE Data Folders:** /data/test/Ate/ProdSW/
- **Machine Folders:** /data/test/TS-XX/

### DOS Machine Paths
- **Network Drive T:** \\D2TESTNAS\test (SMB1)
- **Local Batch Files:** C:\BAT\
- **Local ATE Data:** C:\ATE\
- **ATE Subfolders:** C:\ATE\5BDATA, C:\ATE\7BDATA, C:\ATE\8BDATA, C:\ATE\DSCDATA, etc.

---

## Commands & Outputs

### Check DSCDATA on NAS
```bash
ssh root@192.168.0.9 "ls -la /data/test/Ate/ProdSW/DSCDATA/"
```
Output (before fix):
```
-rw-r--r--+ 1 root root 64395 Dec 14 20:03 DSCMAIN4.DAT
```

### Check DSCDATA on AD2
```bash
ls -la "/Volumes/C$/Shares/test/Ate/ProdSW/DSCDATA/"
```
Output:
```
-rwx------ 1 azcomputerguru staff 65508 Jan 16 12:35 DSCMAIN4.DAT
```

### Sync DSCDATA to NAS
```bash
scp "/Volumes/C$/Shares/test/Ate/ProdSW/DSCDATA/"* root@192.168.0.9:/data/test/Ate/ProdSW/DSCDATA/
```

### Verify after sync
```bash
ssh root@192.168.0.9 "ls -la /data/test/Ate/ProdSW/DSCDATA/"
```
Output (after fix):
```
-rw-r--r--+ 1 root root 65508 Jan 21 13:42 DSCMAIN4.DAT
```

---

## Configuration Changes

### Files Modified

**NWTOC.BAT (v3.5)**
- Path: `/Users/azcomputerguru/ClaudeTools/projects/dataforth-dos/batch-files/NWTOC.BAT`
- Deployed to: NAS `/data/test/COMMON/ProdSW/NWTOC.BAT`
- Deployed to: AD2 `C:\Shares\test\COMMON\ProdSW\NWTOC.BAT`
- Changes:
  - Switched from XCOPY to COPY
  - Removed all >NUL redirects
  - Added IF NOT EXIST checks before MD
  - Added 8 ATE data folder copies (DSCDATA, 5BDATA, 7BDATA, etc.)
  - Removed machine-specific section
  - Removed MACHINE variable requirement

**DEPLOY.BAT (v2.4)**
- Path: `/Users/azcomputerguru/ClaudeTools/projects/dataforth-dos/batch-files/DEPLOY.BAT`
- Deployed to: NAS and AD2
- Changes:
  - Switched all XCOPY to COPY
  - Simplified output messages

**CTONW.BAT (v3.1)**
- Path: `/Users/azcomputerguru/ClaudeTools/projects/dataforth-dos/batch-files/CTONW.BAT`
- Deployed to: NAS `/data/test/COMMON/ProdSW/CTONW.BAT`
- Deployed to: AD2 `C:\Shares\test\COMMON\ProdSW\CTONW.BAT`
- Changes:
  - Uploads test results to T:\%MACHINE%\LOGS\*LOG (machine-specific)
  - Refuses to run if MACHINE variable not set (prevents overwriting)
  - Refuses to run if T:\%MACHINE% folder doesn't exist
  - Logs machine name, date/time, target path to C:\ATE\CTONW.LOG
  - Uploads 8 LOG folders: 5BLOG, 7BLOG, 8BLOG, DSCLOG, HVLOG, PWRLOG, SCTLOG, VASLOG
  - Also uploads Reports and batch files

**ATESYNC.BAT (v1.0) - NEW**
- Path: `/Users/azcomputerguru/ClaudeTools/projects/dataforth-dos/batch-files/ATESYNC.BAT`
- Deployed to: NAS and AD2
- Purpose: Boot-time orchestrator (ARCHBAT equivalent)
- Usage: `CALL ATESYNC TS-27` or `SET MACHINE=TS-27` then `CALL ATESYNC`
- Features:
  - Accepts machine name as parameter or uses MACHINE environment variable
  - Creates machine folder structure if missing
  - Calls CTONW first (upload test results)
  - Then calls NWTOC (download updates)
  - Error handling with clear messages

**Sync-FromNAS.ps1 (on AD2)**
- Path: `C:\Shares\test\scripts\Sync-FromNAS.ps1`
- Changes: Added new section to sync Ate/ProdSW folder:
```powershell
# Sync Ate/ProdSW (shared ATE data folders - 5BDATA, 7BDATA, 8BDATA, DSCDATA, etc.)
Write-Log "Syncing Ate/ProdSW data folders..."
$ateProdSwPath = "$AD2_TEST_PATH\Ate\ProdSW"
if (Test-Path $ateProdSwPath) {
    $ateFiles = Get-ChildItem -Path $ateProdSwPath -File -Recurse -ErrorAction SilentlyContinue
    foreach ($file in $ateFiles) {
        $relativePath = $file.FullName.Substring($ateProdSwPath.Length + 1).Replace('\', '/')
        $remotePath = "$NAS_DATA_PATH/Ate/ProdSW/$relativePath"
        # ... copy logic ...
    }
}
```

---

## DOS 6.22 Compatibility Notes (Updated)

### Commands That DON'T Work
- XCOPY /I flag - "Invalid switch"
- XCOPY /D without date - "Invalid number of parameters"
- 2>NUL (stderr redirect) - "Too many parameters"
- >NUL sometimes causes hanging

### Commands That Work Reliably
- COPY (simple file copy)
- MD directory (create directory)
- IF NOT EXIST path\*.* (check if directory has files)
- IF EXIST file\*.* (check if files exist)

### Best Practices
- Use COPY instead of XCOPY for flat files
- Use IF NOT EXIST checks before MD to avoid errors
- Don't redirect to NUL unless necessary
- Don't use 2>NUL (DOS 6.22 doesn't support it)

---

## Pending/Incomplete Tasks

### Immediate
- [x] DSCDATA files synced to NAS (completed)
- [x] Sync script updated to include Ate/ProdSW (completed)
- [x] CTONW.BAT fixed to upload to correct LOGS folders (completed)
- [x] ATESYNC.BAT created as ARCHBAT equivalent (completed)
- [ ] Test ATESYNC and CTONW on a DOS machine

### Future
- [x] Investigate testdatadb on AD2 port 3000 (found - Node.js/Express with SQLite)
- [ ] Document the testdatadb application fully

---

## Reference Information

### File Versions (Current)
| File | Version | Date |
|------|---------|------|
| ATESYNC.BAT | 1.0 | 2026-01-21 |
| CTONW.BAT | 3.1 | 2026-01-21 |
| NWTOC.BAT | 3.5 | 2026-01-21 |
| DEPLOY.BAT | 2.4 | 2026-01-21 |
| UPDATE.BAT | 2.3 | 2026-01-20 |
| CHECKUPD.BAT | 1.3 | 2026-01-20 |

### ATE Data Folders (9 total)
1. 5BDATA
2. 7BDATA
3. 8BDATA
4. DSCDATA
5. HVDATA
6. PWRDATA
7. RMSDATA
8. SCTDATA
9. (root files in Ate/ProdSW/)

### Git Commits This Session
- `fd24a0c` - fix(dataforth-dos): DOS 6.22 batch file improvements and sync fix

### Network Topology
```
Admin deposits files on AD2
         |
         v
    [AD2 Server]
    192.168.0.6
    C:\Shares\test\
         |
         | (Sync-FromNAS.ps1 every 15 min)
         v
    [D2TESTNAS]
    192.168.0.9
    /data/test/
         |
         | (SMB1 protocol)
         v
    [DOS Machines]
    TS-1L through TS-27
    T:\ = \\D2TESTNAS\test
```

---

## Session Metrics

- **Session Duration:** ~4 hours
- **Issues Fixed:** 2 major (DSCDATA sync, CTONW upload path)
- **Files Created:** 1 (ATESYNC.BAT)
- **Files Modified:** 4 (NWTOC.BAT, DEPLOY.BAT, CTONW.BAT, Sync-FromNAS.ps1)
- **Git Commits:** 2

---

## TS-27 Golden Example Analysis

TS-27 backup located at `/data/test/TS-27/Backup/ORIG/` contains:
- Full machine backup (AUTOEXEC.BAT, CONFIG.SYS, all folders)
- Original batch files in BAT/ folder
- Original ATE folder structure

### Original Boot Sequence (TS-27)
```
AUTOEXEC.BAT
  -> CALL STARTNET.BAT (maps T: and X: drives)
  -> CALL ARCHBAT.BAT TS-27 X:
     -> CALL CTONW TS-27 (upload logs)
     -> CALL NWTOC TS-27 (download updates)
  -> CALL MENUX (main ATE menu)
```

### Key Differences: Original vs New
| Feature | Original (TS-27) | New Setup |
|---------|------------------|-----------|
| Machine ID | %1 parameter | %MACHINE% env var |
| Orchestrator | ARCHBAT.BAT | ATESYNC.BAT |
| Download source | T:\%1\ProdSW (per-machine) | T:\COMMON\ProdSW (centralized) |
| Upload destination | T:\%1\LOGS (per-machine) | T:\%MACHINE%\LOGS (per-machine) |

---

**Session End:** 2026-01-21 16:00
**Next Session:** Test ATESYNC and CTONW on DOS machine