Session log: Pavon archive cleanup (25TB freed) and OwnCloud integration (35TB external storage)
This commit is contained in:
384
clients/pavon/infrastructure-analysis.md
Normal file
384
clients/pavon/infrastructure-analysis.md
Normal file
@@ -0,0 +1,384 @@
|
||||
# Pavon & Jupiter Infrastructure Analysis
|
||||
|
||||
**Date:** April 12, 2026
|
||||
**Audit Performed By:** Claude (AZ Computer Guru)
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
**Recommendation:** Keep Pavon server as dedicated infrastructure + archive tier
|
||||
|
||||
**Key Findings:**
|
||||
- Pavon has 40% MORE capacity than Jupiter (121TB vs 97TB)
|
||||
- Can reclaim 25.2TB from Pavon by deleting data >3 years old
|
||||
- After cleanup: Pavon will have 84TB free (69% available)
|
||||
- Jupiter is 57% full with limited growth room
|
||||
- SeaFile (11TB) appears to be legacy/duplicate storage
|
||||
|
||||
---
|
||||
|
||||
## Current Infrastructure
|
||||
|
||||
### Jupiter (Primary Infrastructure - 172.16.3.20)
|
||||
|
||||
**Capacity:** 97TB total
|
||||
**Used:** 55TB (57%)
|
||||
**Free:** 42TB (43%)
|
||||
**Array:** 12 active disks (mixed: 16TB, 12TB, 10TB, 6TB drives)
|
||||
|
||||
**Storage Breakdown:**
|
||||
```
|
||||
Plex/ 23TB (Media server - largest consumer)
|
||||
SeaFile/ 11TB (Legacy cloud storage?)
|
||||
OwnCloud/ 9.5TB (Current cloud storage)
|
||||
Backups/ 8.3TB (System backups)
|
||||
Tools/ 3.0TB (Software/utilities)
|
||||
domains/ 704GB (VMs)
|
||||
system/ 346GB (Unraid system)
|
||||
BT/ 280GB (BitTorrent)
|
||||
appdata/ 107GB (Docker app data)
|
||||
isos/ 18GB (ISO images)
|
||||
Users/ 5.7GB (User home directories)
|
||||
```
|
||||
|
||||
**Growth Concerns:**
|
||||
- Only 42TB free space remaining
|
||||
- Plex growing (media library)
|
||||
- OwnCloud growing (client data)
|
||||
- Limited room for new services/clients
|
||||
|
||||
---
|
||||
|
||||
### Pavon (Archive Server - 172.16.1.33)
|
||||
|
||||
**Capacity:** 121TB total
|
||||
**Used:** 62TB (51%)
|
||||
**Free:** 59TB (49%)
|
||||
**Array:** 12 active disks (11x ST 12TB + 1x ST 16TB parity)
|
||||
|
||||
**Current Storage:**
|
||||
```
|
||||
Storage/ 60TB (Camera archive: Dec 2022 - Oct 2023)
|
||||
├── Deletable 25.2TB (Dec 2022 - Mar 2023, >3 years old)
|
||||
└── Keep 35TB (May - Oct 2023, within retention)
|
||||
system/ 21GB (Unraid system files)
|
||||
```
|
||||
|
||||
**After 3-Year Cleanup:**
|
||||
```
|
||||
Storage/ 35TB (Camera archive retained)
|
||||
Free Space/ 84TB (69% available capacity!)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Comparative Analysis
|
||||
|
||||
| Metric | Jupiter | Pavon | Winner |
|
||||
|--------|---------|-------|--------|
|
||||
| Total Capacity | 97TB | 121TB | **Pavon +24%** |
|
||||
| Free Space (current) | 42TB | 59TB | **Pavon +40%** |
|
||||
| Free Space (after cleanup) | 42TB | 84TB | **Pavon +100%** |
|
||||
| Utilization | 57% | 51% (29% after cleanup) | **Pavon** |
|
||||
| Growth Capacity | Limited | Excellent | **Pavon** |
|
||||
| Service Load | High (Plex, OwnCloud, VMs, Docker) | None (archive only) | **Pavon** |
|
||||
|
||||
---
|
||||
|
||||
## Strategic Recommendations
|
||||
|
||||
### Option 1: Tiered Storage Architecture ⭐ RECOMMENDED
|
||||
|
||||
**Configuration:**
|
||||
```
|
||||
Jupiter (Hot Tier - 172.16.3.20)
|
||||
├── Active services (Plex, OwnCloud, Docker)
|
||||
├── Recent data (last 6-12 months)
|
||||
├── Fast access storage
|
||||
└── Current: 55TB used, 42TB free
|
||||
|
||||
Pavon (Cold/Archive Tier - 172.16.1.33)
|
||||
├── Camera footage archive (35TB)
|
||||
├── Backup target for Jupiter (planned: 18TB)
|
||||
├── DR replica of critical data
|
||||
├── Other client archives
|
||||
└── After cleanup: 37TB used, 84TB free
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- ✅ Best use of available capacity (84TB on Pavon vs 42TB on Jupiter)
|
||||
- ✅ Physical isolation (backups on separate hardware)
|
||||
- ✅ Disaster recovery capability
|
||||
- ✅ Supports MSP business growth
|
||||
- ✅ Automated tiering (move old data Jupiter → Pavon)
|
||||
- ✅ Can relocate Pavon offsite for geographic redundancy
|
||||
|
||||
**Implementation:**
|
||||
1. Clean up Pavon (delete 25.2TB of old data)
|
||||
2. Set up rsync backup: Jupiter critical data → Pavon
|
||||
3. Configure OwnCloud external storage: Pavon archive mounted in OwnCloud
|
||||
4. Automate archival: Jupiter data >6 months → Pavon
|
||||
5. Set retention policy: Auto-delete data >3 years from Pavon
|
||||
|
||||
**Cost:** ~$200/year power for Pavon server
|
||||
**ROI:** 84TB of backup/archive capacity, DR protection, client growth room
|
||||
|
||||
---
|
||||
|
||||
### Option 2: Consolidate to Jupiter (NOT RECOMMENDED)
|
||||
|
||||
**Problems:**
|
||||
- ❌ Jupiter only has 42TB free, Pavon has 35TB to migrate
|
||||
- ❌ Would use most of Jupiter's remaining capacity
|
||||
- ❌ No room for backups or growth
|
||||
- ❌ Single point of failure (all data on one server)
|
||||
- ❌ Need to delete SeaFile (11TB) or Backups (8.3TB) first
|
||||
- ❌ Massive data migration (days of transfer time)
|
||||
|
||||
**Only viable if:**
|
||||
- Delete SeaFile (appears to be duplicate/legacy)
|
||||
- Significantly reduce Plex library
|
||||
- Don't plan to add more clients/services
|
||||
|
||||
---
|
||||
|
||||
### Option 3: Hybrid (Start Small, Expand Later)
|
||||
|
||||
**Phase 1: Cleanup + Testing**
|
||||
1. Delete 25.2TB from Pavon (enforce 3-year retention)
|
||||
2. Mount Pavon Storage in OwnCloud as external storage
|
||||
3. Test performance and access patterns
|
||||
4. Evaluate for 30-60 days
|
||||
|
||||
**Phase 2: Expand Usage**
|
||||
Based on Phase 1 results:
|
||||
- Add Jupiter backup jobs → Pavon
|
||||
- Move old OwnCloud data → Pavon archive
|
||||
- Set up automated tiering
|
||||
|
||||
**Phase 3: Full Integration**
|
||||
- DR replica of critical infrastructure
|
||||
- Automated lifecycle management
|
||||
- Client archive storage offering
|
||||
|
||||
---
|
||||
|
||||
## Detailed Implementation Plan (Option 1 - Recommended)
|
||||
|
||||
### Phase 1: Cleanup Pavon (Week 1)
|
||||
```bash
|
||||
# 1. Run dry-run preview
|
||||
ssh root@172.16.1.33
|
||||
/root/pavon_cleanup.sh
|
||||
|
||||
# 2. Review preview output
|
||||
# Verify: 184,120 files, 25.2TB expected recovery
|
||||
|
||||
# 3. Execute deletion
|
||||
DRY_RUN=0 /root/pavon_cleanup.sh
|
||||
# Type: DELETE (when prompted)
|
||||
# Wait: 3-5 hours for completion
|
||||
|
||||
# 4. Verify results
|
||||
df -h /mnt/user
|
||||
# Expected: 84TB free (was 59TB)
|
||||
```
|
||||
|
||||
### Phase 2: Jupiter Backup Setup (Week 1-2)
|
||||
```bash
|
||||
# Create backup share on Pavon
|
||||
mkdir -p /mnt/user/jupiter_backups
|
||||
|
||||
# Test rsync from Jupiter → Pavon
|
||||
rsync -av --dry-run /mnt/user/appdata/ \
|
||||
root@172.16.1.33:/mnt/user/jupiter_backups/appdata/
|
||||
|
||||
# Schedule nightly backups (Jupiter cron)
|
||||
0 2 * * * rsync -av --delete /mnt/user/appdata/ \
|
||||
root@172.16.1.33:/mnt/user/jupiter_backups/appdata/
|
||||
```
|
||||
|
||||
**Backup Priority:**
|
||||
1. appdata/ (107GB - Docker configs)
|
||||
2. domains/ (704GB - VMs)
|
||||
3. Critical OwnCloud user data (subset of 9.5TB)
|
||||
4. System configs
|
||||
|
||||
**Expected Backup Size:** ~18TB (appdata + domains + critical data)
|
||||
**Remaining Pavon Space:** 66TB available
|
||||
|
||||
### Phase 3: OwnCloud External Storage (Week 2)
|
||||
```bash
|
||||
# On OwnCloud VM (172.16.3.22)
|
||||
# Mount Pavon Storage share as external storage
|
||||
|
||||
# 1. Install SMB/CIFS external storage app (if needed)
|
||||
sudo -u apache php /var/www/html/owncloud/occ app:enable files_external
|
||||
|
||||
# 2. Create mount for Pavon user
|
||||
sudo -u apache php /var/www/html/owncloud/occ files_external:create \
|
||||
"Camera Archives" smb password::password \
|
||||
--user pavon \
|
||||
-c host=172.16.1.33 \
|
||||
-c share=Storage \
|
||||
-c user=pavon \
|
||||
-c password=<pavon_smb_password>
|
||||
|
||||
# 3. Test access via OwnCloud web interface
|
||||
```
|
||||
|
||||
**Result:** Pavon can access 35TB of camera archives via:
|
||||
- OwnCloud web interface
|
||||
- OwnCloud desktop client
|
||||
- OwnCloud mobile apps
|
||||
|
||||
### Phase 4: Automated Archival (Week 3-4)
|
||||
```bash
|
||||
# Create archival script on Jupiter
|
||||
# Move OwnCloud data >6 months old → Pavon
|
||||
|
||||
# Example: Archive old camera footage
|
||||
find /mnt/user/OwnCloud/pavon/cameras -type f -mtime +180 \
|
||||
-exec rsync -av --remove-source-files {} \
|
||||
root@172.16.1.33:/mnt/user/Storage/archive/ \;
|
||||
```
|
||||
|
||||
### Phase 5: Retention Policy Automation (Week 4)
|
||||
```bash
|
||||
# On Pavon: Monthly cron to delete data >3 years old
|
||||
# /etc/cron.monthly/cleanup_old_archives
|
||||
|
||||
#!/bin/bash
|
||||
# Delete camera footage older than 3 years
|
||||
find /mnt/user/Storage/cam* -type f -mtime +1095 -delete
|
||||
find /mnt/user/Storage -type d -empty -delete
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Cost/Benefit Analysis
|
||||
|
||||
### Keeping Pavon Server
|
||||
|
||||
**Costs:**
|
||||
- Power: ~$200/year (100-150W @ $0.15/kWh)
|
||||
- Maintenance: Minimal (Unraid auto-updates)
|
||||
- Monitoring: 15 min/month
|
||||
|
||||
**Benefits:**
|
||||
- 84TB available capacity (worth ~$2,500 in new drives)
|
||||
- DR/backup capability (priceless for MSP)
|
||||
- Physical isolation (compliance/security)
|
||||
- Supports business growth (new clients/services)
|
||||
- Geographic redundancy option (can relocate)
|
||||
|
||||
**ROI:** 12-18 months (compared to buying new drives for Jupiter)
|
||||
|
||||
### Retiring Pavon Server
|
||||
|
||||
**Savings:**
|
||||
- Power: ~$200/year
|
||||
- Rackspace: 1U (if in datacenter)
|
||||
|
||||
**Losses:**
|
||||
- 84TB capacity (need to buy drives: ~$2,500)
|
||||
- DR capability (need backup solution: ~$500/year)
|
||||
- Growth capacity for MSP business
|
||||
- Hardware available for other projects
|
||||
|
||||
---
|
||||
|
||||
## Action Items
|
||||
|
||||
**Immediate (This Week):**
|
||||
- [DONE] Audit Pavon storage
|
||||
- [DONE] Create cleanup script
|
||||
- [ ] Review cleanup preview
|
||||
- [ ] Execute cleanup (user approval)
|
||||
- [ ] Verify 84TB free space
|
||||
|
||||
**Short-term (Next 2 Weeks):**
|
||||
- [ ] Set up Jupiter → Pavon backups
|
||||
- [ ] Mount Pavon in OwnCloud for Pavon user
|
||||
- [ ] Test backup/restore procedures
|
||||
- [ ] Document in credentials.md
|
||||
|
||||
**Medium-term (Next Month):**
|
||||
- [ ] Implement automated archival
|
||||
- [ ] Set up retention policy automation
|
||||
- [ ] Consider SeaFile migration/decommission (free 11TB on Jupiter)
|
||||
- [ ] Monitor backup success rates
|
||||
|
||||
**Long-term (Next Quarter):**
|
||||
- [ ] Evaluate geographic separation (move Pavon offsite?)
|
||||
- [ ] Add other client archives to Pavon
|
||||
- [ ] Implement monitoring/alerting
|
||||
- [ ] DR testing (restore from Pavon)
|
||||
|
||||
---
|
||||
|
||||
## Questions Answered
|
||||
|
||||
### "Should we migrate to TrueNAS Scale?"
|
||||
**Answer:** Not necessary for current needs. Evaluate if:
|
||||
- You add 3+ more servers
|
||||
- Need clustering/HA
|
||||
- Want enterprise features (SMB clustering, iSCSI ALUA)
|
||||
- Current: Unraid flexibility + tiered storage meets MSP needs
|
||||
|
||||
### "Can Pavon be an extension of Jupiter?"
|
||||
**Answer:** Not natively (Unraid doesn't cluster), but:
|
||||
- ✅ Can mount Pavon shares on Jupiter (Unassigned Devices)
|
||||
- ✅ Can use as backup target (rsync)
|
||||
- ✅ Can tier data (hot on Jupiter, cold on Pavon)
|
||||
- Better than extension: Proper tiered architecture
|
||||
|
||||
### "What about the camera data?"
|
||||
**Answer:** Keep as archive tier:
|
||||
- 35TB within retention policy (May-Oct 2023)
|
||||
- Mount in OwnCloud for web/mobile access
|
||||
- No active recording (data is historical only)
|
||||
- Delete when >3 years old (automated)
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
**Recommended Architecture:**
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ Jupiter (Hot/Production Tier) │
|
||||
│ - Plex, OwnCloud, VMs, Docker │
|
||||
│ - Recent data (< 6 months) │
|
||||
│ - 55TB used, 42TB free │
|
||||
└──────────────┬──────────────────────┘
|
||||
│
|
||||
│ rsync nightly backups
|
||||
│ archival (data >6mo)
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────┐
|
||||
│ Pavon (Cold/Archive/Backup Tier) │
|
||||
│ - Camera archives (35TB) │
|
||||
│ - Jupiter backups (18TB planned) │
|
||||
│ - Other client archives │
|
||||
│ - 37TB used, 84TB free (69%!) │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**This architecture:**
|
||||
- ✅ Maximizes available capacity (126TB total free space)
|
||||
- ✅ Provides disaster recovery (separate hardware)
|
||||
- ✅ Supports MSP growth (room for new clients)
|
||||
- ✅ Cost-effective (~$200/year vs $3,000 in new hardware)
|
||||
- ✅ Scalable (can add geographic redundancy)
|
||||
|
||||
**Next Step:** Execute Pavon cleanup to unlock 84TB capacity
|
||||
|
||||
---
|
||||
|
||||
**Created:** April 12, 2026
|
||||
**Last Updated:** April 12, 2026
|
||||
**Review Date:** July 12, 2026 (quarterly review)
|
||||
Reference in New Issue
Block a user