253 lines
5.7 KiB
Markdown
253 lines
5.7 KiB
Markdown
# Pavon Archive Cleanup Guide
|
|
|
|
**Server:** 172.16.1.33 (Pavon Unraid)
|
|
**Script Location:** `/root/pavon_cleanup.sh`
|
|
**Expected Recovery:** 25.2TB
|
|
**Date Created:** April 12, 2026
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
This script safely deletes camera footage older than 3 years (before April 2023) from Pavon's archive server.
|
|
|
|
**What will be deleted:**
|
|
- Dec 2022: 2.1TB (14,776 files)
|
|
- Jan 2023: 7.0TB (62,048 files)
|
|
- Feb 2023: 8.9TB (46,014 files)
|
|
- Mar 2023: 7.2TB (61,282 files)
|
|
- **Total: 25.2TB (184,120 files)**
|
|
|
|
**What will be kept:**
|
|
- May 2023 - Oct 2023: 35.1TB
|
|
- All data within 3-year retention policy
|
|
|
|
---
|
|
|
|
## Usage
|
|
|
|
### Step 1: Dry-Run (Preview Only - RECOMMENDED FIRST)
|
|
|
|
```bash
|
|
# SSH to Pavon server
|
|
ssh root@172.16.1.33
|
|
|
|
# Run preview (no files deleted)
|
|
/root/pavon_cleanup.sh
|
|
```
|
|
|
|
**What it does:**
|
|
- Shows exactly what will be deleted
|
|
- Calculates space recovery
|
|
- Lists sample files from each period
|
|
- **NO FILES ARE DELETED**
|
|
|
|
### Step 2: Review the Preview
|
|
|
|
Check the output carefully:
|
|
- Verify date ranges are correct (Dec 2022 - Mar 2023)
|
|
- Confirm file counts match audit (184,120 files)
|
|
- Review sample file paths
|
|
|
|
### Step 3: Execute Actual Deletion
|
|
|
|
**Option A: Interactive execution**
|
|
```bash
|
|
# Edit script to disable dry-run
|
|
nano /root/pavon_cleanup.sh
|
|
# Change: DRY_RUN=1 to DRY_RUN=0
|
|
# Save and exit (Ctrl+X, Y, Enter)
|
|
|
|
# Run deletion
|
|
/root/pavon_cleanup.sh
|
|
```
|
|
|
|
**Option B: One-time execution (no script edit)**
|
|
```bash
|
|
# Run with dry-run disabled
|
|
DRY_RUN=0 /root/pavon_cleanup.sh
|
|
```
|
|
|
|
**Confirmation Required:**
|
|
- Script will ask you to type `DELETE` to confirm
|
|
- This prevents accidental execution
|
|
- **Files are permanently deleted** (no recycle bin on Linux)
|
|
|
|
---
|
|
|
|
## Phased Deletion (Alternative Approach)
|
|
|
|
If you want to delete one month at a time:
|
|
|
|
### Delete Dec 2022 Only (2.1TB)
|
|
```bash
|
|
# Edit script and change PERIODS array to:
|
|
PERIODS=(
|
|
"202212:Dec 2022"
|
|
)
|
|
```
|
|
|
|
### Delete Jan 2023 Only (7.0TB)
|
|
```bash
|
|
PERIODS=(
|
|
"202301:Jan 2023"
|
|
)
|
|
```
|
|
|
|
### Delete Feb 2023 Only (8.9TB)
|
|
```bash
|
|
PERIODS=(
|
|
"202302:Feb 2023"
|
|
)
|
|
```
|
|
|
|
### Delete Mar 2023 Only (7.2TB)
|
|
```bash
|
|
PERIODS=(
|
|
"202303:Mar 2023"
|
|
)
|
|
```
|
|
|
|
---
|
|
|
|
## Monitoring Progress
|
|
|
|
The script provides:
|
|
- **Real-time output**: Shows each file being deleted
|
|
- **Progress indicators**: Updates every 1000 files
|
|
- **Detailed logging**: All actions logged to `/root/cleanup_logs/`
|
|
|
|
**To monitor:**
|
|
```bash
|
|
# Watch log file in real-time (in another SSH session)
|
|
tail -f /root/cleanup_logs/cleanup_*.log
|
|
|
|
# Check current disk usage
|
|
df -h /mnt/user
|
|
|
|
# Count remaining files
|
|
find /mnt/user/Storage/cam* -name "Event2022*.avi" | wc -l
|
|
```
|
|
|
|
---
|
|
|
|
## Expected Timeline
|
|
|
|
**Deletion speed:** ~500-1000 files/minute (depends on disk I/O)
|
|
|
|
| Period | Files | Est. Time |
|
|
|--------|-------|-----------|
|
|
| Dec 2022 | 14,776 | 15-30 min |
|
|
| Jan 2023 | 62,048 | 1-2 hours |
|
|
| Feb 2023 | 46,014 | 45-90 min |
|
|
| Mar 2023 | 61,282 | 1-2 hours |
|
|
| **Total** | **184,120** | **3-5 hours** |
|
|
|
|
---
|
|
|
|
## Safety Features
|
|
|
|
1. **Dry-run default:** Script runs in preview mode unless explicitly changed
|
|
2. **Confirmation required:** Must type `DELETE` to proceed
|
|
3. **Detailed logging:** All actions logged to `/root/cleanup_logs/`
|
|
4. **Pattern-based deletion:** Only deletes files matching `Event2022*.avi` and `Event2023[01-03]*.avi`
|
|
5. **No recursive wildcards:** Won't accidentally delete wrong directories
|
|
|
|
---
|
|
|
|
## Verification After Deletion
|
|
|
|
```bash
|
|
# Check new disk usage
|
|
df -h /mnt/user
|
|
|
|
# Verify old files are gone
|
|
find /mnt/user/Storage/cam* -name "Event2022*.avi" | wc -l # Should be 0
|
|
find /mnt/user/Storage/cam* -name "Event202301*.avi" | wc -l # Should be 0
|
|
find /mnt/user/Storage/cam* -name "Event202302*.avi" | wc -l # Should be 0
|
|
find /mnt/user/Storage/cam* -name "Event202303*.avi" | wc -l # Should be 0
|
|
|
|
# Verify remaining files intact
|
|
find /mnt/user/Storage/cam* -name "Event202305*.avi" | wc -l # Should have files
|
|
find /mnt/user/Storage/cam* -name "Event202306*.avi" | wc -l # Should have files
|
|
|
|
# Check logs
|
|
ls -lh /root/cleanup_logs/
|
|
```
|
|
|
|
---
|
|
|
|
## Rollback Plan
|
|
|
|
**Important:** Once deleted, files cannot be recovered unless you have backups.
|
|
|
|
**Before deletion:**
|
|
- If unsure, create backup: `rsync -av /mnt/user/Storage /mnt/user/Backups/pavon_archive_backup/`
|
|
- Takes ~6-8 hours to backup 60TB, requires 60TB free space
|
|
|
|
**No backups exist on Jupiter for this data** (confirmed during audit).
|
|
|
|
---
|
|
|
|
## Post-Cleanup Actions
|
|
|
|
After successful deletion:
|
|
|
|
1. **Verify space recovery:**
|
|
```bash
|
|
df -h /mnt/user
|
|
# Should show ~84TB free (was 59TB)
|
|
```
|
|
|
|
2. **Set up automated retention (optional):**
|
|
- Create monthly cron job
|
|
- Auto-delete data >3 years old
|
|
- Email notifications
|
|
|
|
3. **Document in credentials.md:**
|
|
- Update Pavon server notes
|
|
- Record cleanup date
|
|
- Note new available capacity
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Script hangs or runs slowly
|
|
- Normal for large deletions (184K files)
|
|
- Check progress: `tail -f /root/cleanup_logs/cleanup_*.log`
|
|
- Monitor disk I/O: `iotop` (if installed)
|
|
|
|
### "Permission denied" errors
|
|
- Run as root: `sudo /root/pavon_cleanup.sh`
|
|
- Check file ownership: `ls -l /mnt/user/Storage/cam*/`
|
|
|
|
### Want to cancel during execution
|
|
- Press `Ctrl+C` to stop
|
|
- Files deleted so far are gone
|
|
- Remaining files are safe
|
|
- Can resume by running script again (only deletes what remains)
|
|
|
|
### Disk space not showing as free
|
|
- Unraid may need array refresh
|
|
- Wait 5-10 minutes for system to update
|
|
- Run: `du -sh /mnt/user/Storage` to verify actual usage
|
|
|
|
---
|
|
|
|
## Support
|
|
|
|
**Script location on local machine:**
|
|
`/Users/azcomputerguru/ClaudeTools/temp/pavon_cleanup.sh`
|
|
|
|
**Logs location:**
|
|
`/root/cleanup_logs/` on Pavon server
|
|
|
|
**Contact:** Check session logs for questions
|
|
|
|
---
|
|
|
|
**Created:** April 12, 2026
|
|
**Audit performed:** April 12, 2026
|
|
**Last updated:** April 12, 2026
|