pavon: session log — OwnCloud VM cron stacking diagnosed and stabilized

Found 75-126 stale `occ system:cron` processes on 172.16.3.22 piling up
since 2026-04-27 due to bad oc_filecache LIKE query against pavon's 257K
camera files. Killed stale procs (load 80 -> 5), wrapped apache crontab
with `flock -n /tmp/oc-cron.lock` to prevent restacking. Per-user
versioning disable rejected by OwnCloud Community (`files_versions`
can't be enabled for groups); workaround `occ versions:cleanup pavon`
identified and deferred. Migration/retention cron deferred per user.
NVR architecture clarified: GeoVision NVRs sync via OC Desktop client
with virtual file placeholders; no direct SMB access to Jupiter.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-29 07:51:02 -07:00
parent a2f38c1038
commit f22d33f2ae
2 changed files with 361 additions and 8 deletions

View File

@@ -1,23 +1,39 @@
# Pavon — Project State
> Last updated: 2026-04-20
> Last updated: 2026-04-29
**Status:** COMPLETE
**Last Activity:** 2026-04-12
**Status:** ACTIVE — deferred follow-ups
**Last Activity:** 2026-04-29
Video archive management project. Cleaned 25 TB of old footage (184,124 files deleted), then integrated the remaining 35 TB with OwnCloud for organized archival access.
Video archive management with OwnCloud as source of truth (3-year retention). Original 25 TB cleanup completed 2026-04-12. New chapter opened 2026-04-28: OwnCloud VM cron stacking spiral diagnosed and stabilized, but root cause cleanup deferred per user.
## What Was Done
### 2026-04-12 (original project, COMPLETE)
- Identified and deleted 184,124 redundant/old files totaling 25 TB
- Infrastructure analysis of storage environment
- Remaining 35 TB integrated with OwnCloud via external storage setup
- Archive scan and cleanup completion documented
- Final setup summary written
### 2026-04-29 (cron stacking incident, STABLE / FOLLOW-UPS DEFERRED)
- Diagnosed that the OwnCloud VM (172.16.3.22) was running 75-126 stale `occ system:cron` processes since 2026-04-27, all spinning on a bad `oc_filecache` LIKE query against pavon's storage 78 (~237K camera files)
- Killed stale crons, load avg dropped 80 -> 5
- Wrapped the apache crontab line with `flock -n /tmp/oc-cron.lock` to prevent stacking — current production state
- Architecture clarified: GeoVision NVRs at Curves and Raiders sites use OwnCloud Desktop sync client with virtual file placeholders; NVRs have no direct SMB access to Jupiter/Saturn; pavon never touches OwnCloud directly
- Discovered `files_versions` cannot be group-restricted in OwnCloud Community; per-user disable not possible. Identified `occ versions:cleanup pavon` as the workaround. Deferred.
## Current Operational State
- OwnCloud VM stable. Cron protected by flock. No active firefighting needed.
- 30 GB of accumulated junk versions sitting in `/owncloud/pavon/files_versions/` waiting for `occ versions:cleanup pavon`
- A dangling `versioning_users` group exists on the OwnCloud VM (created during a failed group-restrict attempt; harmless)
- 1Password password for OwnCloud VM is stale (`Paper123!@#-unifi!`); SOPS has the working value (`r3tr0gradE99!!`) — needs reconciliation
## If Resuming
- Check `clients/pavon/owncloud-archive-setup.md` and `final-setup-summary.md` for the OwnCloud configuration
- Storage layout documented in `infrastructure-analysis.md`
- `cleanup-completion-report.md` confirms what was deleted and what was kept
- Session logs in `clients/pavon/session-logs/`
- **Most recent context:** `clients/pavon/session-logs/2026-04-29-session.md` (full diagnostic trail, command outputs, deferred-task checklist)
- **Original archive integration:** `owncloud-archive-setup.md`, `final-setup-summary.md`
- **Storage layout:** `infrastructure-analysis.md`
- **Pending work checklist:** see "Concrete next session checklist" in 2026-04-29 session log
- **Session logs index:** `clients/pavon/session-logs/`