azcomputerguru/claudetools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
000ee3da5c593c35b10a64f597b26e8feed97202
claudetools/projects/dataforth-dos/session-logs/2026-03-12-session.md
Mike Swanson 000ee3da5c Session log: D2TESTNAS VM build, NAS migration, rsync sync fix 
Built Debian 13 VM replacement for aging ReadyNAS, deployed rsync-based
sync script to AD2, transferred data, completed IP cutover to 192.168.0.9.
Includes setup scripts, sync fixes, and comprehensive session logs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 06:08:32 -07:00

10 KiB
Raw Blame History

Session Log: 2026-03-12 - D2TESTNAS VM Build, NAS Migration, Rsync Sync Fix

Session Summary

Major infrastructure session: replaced broken SCP-based sync with rsync, built a new Debian 13 VM to replace the aging ReadyNAS, transferred data, and performed IP cutover. Also investigated BTRFS snapshots on old NAS and began DOS machine testing against new Linux-based NAS.

Key Accomplishments

  1. Fixed Sync-FromNAS.ps1 on AD2 - Replaced broken SCP with rsync daemon protocol, added guards for stray files (TS-21, TS-3R/HVLOG), added log file write retry for AV locking
  2. Disabled old SCP scheduled tasks on AD2 - Killed Sync-FromNAS and BulkSync-Catchup tasks
  3. Built D2TESTNAS replacement VM on DF-HYPERV-B (Debian 13, Samba SMB1, rsync daemon, BTRFS 512GB data disk)
  4. Transferred data from old NAS - test/ data (~24GB+), datasheets, home, 82 snapshots (partial ~43GB logical)
  5. IP cutover completed - New VM now at 192.168.0.9, old NAS on DHCP at 192.168.0.117
  6. WINS/NetBIOS conflict resolved - Killed nmbd on old NAS, removed auto-restart cron, blocked ports 137/138 via iptables

Key Decisions

  • Chose Hyper-V VM on DF-HYPERV-B over repurposing physical server DF-SVR-D2-SYNC
  • Used BTRFS for data disk with subvolumes for test and datasheets
  • Single rsync stream to avoid overloading old NAS (ARM processor)
  • BTRFS snapshots from old NAS are being flattened (CoW -> full copies) which makes them much larger than ReadyNAS UI reported

Problems Encountered and Solutions

  • TS-21 stray file: 1,129-byte DAT file from 2012 existed instead of directory. Renamed, added script guard.
  • TS-3R/LOGS/HVLOG stray file: 56-byte file from 2013. Same fix.
  • Log file locking: AV locking sync-from-nas.log. Added 3-retry with 100ms delay.
  • AD2 high latency: AV causing 685-1056ms ping. Recommended exclusions.
  • NAS freezing under SSH load: Power cycled, limited to single rsync stream.
  • nmbd auto-restart on old NAS: Cron */5 * * * * pgrep -x nmbd || /usr/sbin/nmbd -D. Removed cron, blocked ports via iptables.
  • nmcli config didn't save first attempt: SSH dropped before apply. Re-ran successfully.
  • DOS Error 53 (network path not found): Old NAS still broadcasting D2TESTNAS name. Fixed by killing nmbd and blocking NetBIOS ports.

Credentials

New D2TESTNAS VM (Debian 13)

  • IP: 192.168.0.9 (static via NetworkManager)
  • SSH: root / Paper123!@# (also localadmin / Paper123!@#)
  • SSH Key: ed25519 generated on VM, public key installed on old NAS
  • Key fingerprint: SHA256:S2Eom4RwHS/8YMu+ePnOmDOJxGhIkxJQ2ocR3WsH24o root@D2TESTNAS

Rsync Daemon (new VM)

  • Port: 873
  • Module: test = /data/test
  • User: rsync
  • Password: IQ203s32119
  • Config: /etc/rsyncd.conf
  • Secrets: /etc/rsyncd.secrets

Samba (new VM)

  • Shares: test (/data/test), datasheets (/data/datasheets), snapshots (/data/test/.snapshots)
  • Protocol: SMB1 (CORE) through SMB3
  • Auth: Guest OK on all shares
  • Workgroup: D2TESTING
  • NetBIOS name: D2TESTNAS
  • WINS support: yes

Old NAS (ReadyNAS)

  • Current IP: 192.168.0.117 (DHCP, was 192.168.0.9)
  • MAC: 28:C6:8E:34:4B:5E
  • SSH: root (key-based auth from new VM)
  • Status: nmbd killed, cron cleared, NetBIOS ports blocked via iptables. Samba stopped. SSH still works for rsync transfers.

AD2 (Windows Server)

  • IP: 192.168.0.6
  • Sync script: C:\Scripts\Sync-FromNAS-rsync.ps1 (deployed, dry-run validated)
  • Test data path: C:\Shares\test\
  • cwRsync: Installed via Chocolatey

DF-SVR-D2-SYNC (unused physical server)

  • IP: 192.168.0.93
  • Creds: sysadmin / Paper123!@#
  • HP ProLiant ML350 G6, 64GB RAM, Server 2019
  • SMB share: NAS-BACKUP (was used temporarily for CIFS backup attempt)
  • SMB1 enabled on this server

UDM Network

  • WINS server: 192.168.0.9 (configured in UDM DHCP option 44)

Infrastructure

New D2TESTNAS VM Configuration

  • Host: DF-HYPERV-B (dedicated Hyper-V host)
  • OS: Debian 13 (Trixie)
  • Network: eth0, static 192.168.0.9/24, gateway 192.168.0.1
  • Disks:
    • /dev/sda: OS disk
    • /dev/sdb: 512GB BTRFS data disk mounted at /data
  • BTRFS subvolumes: test, datasheets (under /data)
  • Services: smbd, nmbd, rsync (daemon), sshd, cron
  • Snapshot cron: 
    0 * * * * /usr/local/bin/btrfs-snapshot.sh test 48
0 * * * * /usr/local/bin/btrfs-snapshot.sh datasheets 48
0 0 * * * /usr/local/bin/btrfs-snapshot.sh test 30
0 0 * * 0 /usr/local/bin/btrfs-snapshot.sh test 12

Key Config Files on New VM

  • /etc/samba/smb.conf - Samba config (SMB1/CORE, DOS charset CP437, WINS)
  • /etc/rsyncd.conf - rsync daemon (module "test")
  • /etc/rsyncd.secrets - rsync auth (rsync:IQ203s32119)
  • /usr/local/bin/btrfs-snapshot.sh - BTRFS snapshot script

Data Transfer Status (as of ~18:30)

  • test/ data (excl snapshots): ~24 GB transferred, rsync still running (single stream from .117)
  • test/ snapshots: ~43 GB logical transferred (82 snapshots), transfer was stopped to reduce NAS load - needs restart
  • datasheets/ + snapshots: Complete (2.3 MB + 82 snapshot dirs)
  • home/: Complete (612 KB)
  • Disk usage: ~26 GB actual on BTRFS (CoW dedup), 486 GB free
  • Note: ReadyNAS UI reported 5.26GB data + 16.28GB snapshots, but actual rsync transfer is MUCH larger due to BTRFS CoW flattening

Files Created/Modified

New Files

  • D:\ClaudeTools\projects\dataforth-dos\sync-fixes\Sync-FromNAS-rsync.ps1 - Complete rsync-based replacement sync script (deployed to AD2)
  • D:\ClaudeTools\projects\dataforth-dos\d2testnas-vm\setup-d2testnas.sh - 522-line post-install setup script
  • D:\ClaudeTools\projects\dataforth-dos\d2testnas-vm\README.md - Hyper-V creation commands, Debian install notes, cutover checklist

Script Fixes Applied (Sync-FromNAS-rsync.ps1)

  1. Directory-only filter for NAS station enumeration (line ~125)
  2. Station path guard - detects stray files where directories expected
  3. Log type directory guard - renames stray files in LOGS subdirs
  4. Write-Log retry - 3 attempts with 100ms delay for AV file locking

Deployed to New VM (via SSH)

  • /etc/samba/smb.conf (full Samba config)
  • /etc/rsyncd.conf + /etc/rsyncd.secrets
  • /usr/local/bin/btrfs-snapshot.sh + cron entries
  • SSH key pair generated, public key added to old NAS

Pending/Incomplete Tasks

Immediate (resume next session)

  1. Monitor test/ data rsync - Single stream running from old NAS (.117) to new VM (.9). Check with: 
    ssh root@192.168.0.9 "ps aux | grep 'rsync -av' | grep -v grep; du -sh /data/test/ --exclude=.snapshots"
  2. Restart snapshot transfer after data transfer completes: 
    ssh root@192.168.0.9 "nohup bash -c 'rsync -av root@192.168.0.117:/data/test/.snapshots/ /data/test/.snapshots/ 2>&1 | tail -5' &"
  3. Test DOS machine connectivity - Error 53 was resolved (old NAS NetBIOS killed). Need to reboot DOS machine and test:
    • NET USE T: \\D2TESTNAS\TEST
    • Run CTONW.BAT (copy logs to NAS)
    • Run NWTOC.BAT (download updates from NAS)
    • Verify files appear in /data/test/TS-XX/LOGS/ on new VM

After Data Transfer Complete

  1. Verify data integrity - Compare file counts/sizes between old and new NAS
  2. Power off old NAS once all data confirmed transferred
  3. Set up scheduled task on AD2 - Create 15-minute scheduled task for Sync-FromNAS-rsync.ps1
  4. Run real (non-dry) sync on AD2 - Execute Sync-FromNAS-rsync.ps1 without -DryRun flag
  5. AV exclusions on AD2 - Add exclusions for C:\Shares\test\ and rsync.exe

Nice to Have

  1. Copy NAS config backup to new VM (already backed up to DF-SVR-D2-SYNC)
  2. Datto Workplace SmartBadge research - Researched that SmartBadge add-in for Excel doesn't exist; Workplace integrates via sync client and web, not Excel plugin

DOS Machine Data Flow

DOS 6.22 (C:\ATE\) --COPY--> T:\MACHINE\LOGS\ (NAS via SMB1)
                                    |
                                    v (rsync daemon, port 873)
                              AD2 C:\Shares\test\
                                    |
                                    v (future: database ingestion)
                              MariaDB @ 172.16.3.30

Batch Files (DOS -> NAS)

  • CTONW.BAT v3.2 - Uses COPY (not XCOPY) to upload log files from C:\ATE\ to T:\MACHINE\LOGS\
  • NWTOC.BAT v3.5 - Uses COPY to download updates from T:\COMMON\ProdSW\ to C:\BAT\ and C:\ATE\
  • UPDATE.BAT v2.1 - Uses XCOPY for full machine backup (had /D flag fix for DOS 6.22)

Log Types

5BLOG, 7BLOG, 8BLOG, DSCLOG, SCTLOG, VASLOG, PWRLOG, HVLOG

Active Stations

TS-3L (most recent activity), TS-4R, TS-3R, TS-11L, TS-GURU, plus many others

Reference

Key Commands

# SSH to new D2TESTNAS
ssh root@192.168.0.9

# SSH to old NAS (DHCP)
ssh root@192.168.0.117

# Check rsync transfers on new VM
ssh root@192.168.0.9 "ps aux | grep rsync | grep -v grep"

# Test Samba from Windows
net view \\192.168.0.9
smbclient -L //192.168.0.9 -N

# Test rsync daemon
rsync rsync://rsync@192.168.0.9/test/

# Restart services on new VM
ssh root@192.168.0.9 "systemctl restart smbd nmbd rsync"

# BTRFS snapshot status
ssh root@192.168.0.9 "ls /data/test/.snapshots/"

Old NAS Lockdown Commands (already applied)

# Block NetBIOS (prevents name conflict)
ssh root@192.168.0.117 "iptables -A INPUT -p udp --dport 137 -j DROP; iptables -A INPUT -p udp --dport 138 -j DROP; iptables -A OUTPUT -p udp --sport 137 -j DROP; iptables -A OUTPUT -p udp --sport 138 -j DROP"

# Remove auto-restart cron
ssh root@192.168.0.117 "crontab -r"

Session Timeline

  • Started: ~14:00 (context recovery from previous session)
  • Rsync script fixes and deployment to AD2
  • Disabled old SCP scheduled tasks
  • Investigated BTRFS snapshots (81 found)
  • Built D2TESTNAS VM on DF-HYPERV-B (Debian 13)
  • Configured all services (Samba, rsync, BTRFS, SSH)
  • Started data transfer from old NAS
  • Killed snapshot transfer to reduce NAS load (single stream)
  • IP cutover: new VM .185 -> .9, old NAS .9 -> DHCP .117
  • Resolved WINS conflict (killed old NAS nmbd, removed cron, blocked ports)
  • DOS machine testing started - Error 53 resolved
  • Data transfer ongoing (~24GB+ transferred, snapshots pending restart)
  • Session saved: ~18:45
Reference in New Issue View Git Blame Copy Permalink