Files

Howard Enos 0375a46a24 sync: auto-sync from HOWARD-HOME at 2026-06-18 18:26:19

Author: Howard Enos
Machine: HOWARD-HOME
Timestamp: 2026-06-18 18:26:19

2026-06-18 18:28:16 -07:00

10 KiB

Raw Blame History

Cascades — Network Optimization Master Plan (all devices, holistic)

Created: 2026-06-18 (Howard-Home / claude-main)
Status: PLAN — for execution tonight (floors 1–4) per Howard. Floors 5 & 6 (MemCare) EXCLUDED this round.
Goal: Fix the system, not one device at a time. Improve quality for every client (~587), not just the 31 voice devices, by sequencing AP + WLAN + QoS + firewall changes so we don't trade one problem for another.
Builds on: reports/2026-06-16-unifi-full-audit.md, reports/2026-06-16-2.4ghz-remediation-runbook.md (RF mechanics + gated apply commands), reports/2026-06-18-voice-quality-diagnostic.md, and the live 2026-06-18 fleet sample. All RF changes use the gated unifi-wifi scripts (per-zone, dry-run, rollback JSON).

1. Current state (what's actually true right now)

Layer	State	Verdict
2.4 GHz	OVER-THINNED. Overnight 6/17: 24 radios disabled + 42 set Low (~6 dBm). Interference dropped (cu_interf 64→32–48%) BUT retry rose 17→23.4%, satisfaction fell 39→30 (time-of-day-controlled). Edge clients now reach farther/weaker APs. Mesh + Floors 5/6 untouched (full 23 dBm).	Regressed — must correct power floor
5 GHz	80 MHz width on ~76/77 (too wide for the density). 55/77 on DFS (empirically clean — 0 radar). Channels biased to busy upper (149/157). AP 103 saturated: ch149, 75% airtime, ~25,900 retries, 12 clients (and Lauren's phone now locked there). Dining/Rec Room high retry (810/1083).	Constrain width + spread channels + relieve hotspots
6 GHz	75 radios live, ~1 client. Root cause: CSCNet not broadcasting 6 GHz (`wlan_bands=[2g,5g]`). Cleanest untapped capacity.	Open it — the relief valve
QoS	NONE. Voice now isolated on VLAN 30 but not prioritized — voice packets compete with data under load → jitter/breaks.	Add — guaranteed win, now possible
pfSense/WAN/DHCP/DNS	Healthy; ruled out as a WiFi factor (2026-06-16). Dual-WAN stable, DHCP 53% pool, unbound up.	Fine — add voice QoS shaping only
Switching / physical	~25 ports linked 100 M but gig-capable (caps some AP uplinks); 3 offline switches; AP 108 cable pending; p38 4% tx-drop.	Physical work — not tonight, but tracked

2. Root-cause model (why "some devices" are bad)

Three compounding RF causes, plus a missing QoS layer:

2.4 GHz contention — extreme neighbor density (ch6 ~33k BSSIDs). Any client that lands/sticks on 2.4 GHz suffers. Made worse by the over-thinning (weaker signal → more retransmits).
5 GHz over-width + hotspots — 80 MHz halves the usable channel count → co-channel overlap → retries; a few APs (103) are simply overloaded.
6 GHz unused — the clean band that should absorb modern clients is dark, so everything piles onto 5 GHz.
No voice prioritization — even with perfect RF, voice breaks under data bursts without QoS.

The trap we must avoid (the "whack-a-mole"): narrowing 5 GHz to 40 MHz without first opening 6 GHz pushes more clients onto fewer 5 GHz channels → congestion moves, not improves. And dropping 2.4 power further (it's already too low) starves edge clients. Sequence matters.

3. The holistic sequence (open relief valves BEFORE constraining)

Principle: (A) add capacity/priority that can't hurt → (B) fix the regression → (C) then constrain/optimize → (D) fine-tune → validate at every gate. Each step is reversible; gate on live metrics before the next.

PHASE 0 — Pre-flight + baseline (always)

VPN up; live-stats.sh cascades | head -3 (expect 77 APs).
Baseline (compare after, same time-of-day): live-stats.sh cascades > .claude/tmp/opt-pre.txt; radio-usage.sh cascades ng 77 > .claude/tmp/usage-pre.txt.
Pick a watch AP per floor (watch-ap.sh <ip>).

PHASE 1 — QoS for voice (do FIRST; orthogonal, lowest risk, guaranteed win)

Voice VLAN 30 is isolated → mark + prioritize it end-to-end so calls beat data under load.

UniFi (WLAN/switch): ensure WMM/QoS on; the AudioCodes/Poly tag voice DSCP — trust/honor it. On the USW, voice VLAN traffic should hit the high-priority queue.
pfSense: add a traffic-shaper/limiter or floating QoS rule that puts VOICE net (10.0.30.0/24) DSCP EF (46) / RTP UDP into a priority queue on the WAN(s). Low risk — additive, voice-only.
Validate: place test calls during a data-heavy moment; confirm no breakup. (No RF change here.)
Skill gap: the unifi-wifi skill has no QoS verb — this is a pfSense + UniFi config task; consider a small voice-qos helper later.

PHASE 2 — Open the relief valves (capacity + correct the regression)

2a. Enable 6 GHz on CSCNet + steering (creates the offload path BEFORE we narrow 5 GHz):

apply-wlan.sh cascades bands all --wlan CSCNet --apply     # -> [2g,5g,6g]
apply-wlan.sh cascades bsstm on --wlan CSCNet --apply       # 802.11v BSS-transition (assists up-band + roam)

Band-steering (no2ghz_oui) already ON. 6E/7 clients gravitate to clean 6 GHz, offloading 5 GHz. Validate: client mix shifts toward 6g; no SSID-visibility loss for legacy (2.4/5 stay on).

2b. Correct the 2.4 over-thinning — Low → MEDIUM on kept radios (restores edge signal; keeps cells smaller than full power). Per floor, dry-run then apply; regenerate the kept-radio list live:

for z in "Floor 1" "Floor 2" "Floor 3" "Floor 4"; do \
  apply-radio.sh cascades ng power medium --zone "$z" --apply; done   # ~12–15 dBm

Do NOT expand disables. If a specific area shows a dead zone/complaint, re-enable that one radio (ng enable --ap "<name>"). Gate: re-measure retry%/satisfaction same time-of-day vs opt-pre.txt — expect retry back down from ~23% and satisfaction recovering.

PHASE 3 — Constrain + optimize 5 GHz (now that 6 GHz absorbs load)

3a. Width 80 → 40 MHz (doubles non-overlapping channels → spatial reuse):

for z in "Floor 3" "Floor 1" "Floor 2" "Floor 4"; do \
  apply-radio.sh cascades na width 40 --zone "$z" --apply; done   # rollback: na width 80

3b. Channel plan (decision: INCLUDE clean-DFS for diversity vs non-DFS-only for resilience — DFS is empirically clean here, 0 radar):

SURVEY=.claude/tmp/cascades-survey.json; SURVEY_JSON=$SURVEY survey-collect.sh cascades
SURVEY_JSON=$SURVEY channel-plan.sh cascades na     # dry-run; review; apply per zone w/ validation

3c. Relieve AP 103 specifically (it now carries Lauren + 11 others on a 75%-busy ch149): move it off 149 to a clean channel from the plan, 40 MHz. Verify Lauren .202 retry drops after. Gate: 5 GHz retry down on the busy APs; AP 103 cu_total well under 50%; no client stranded.

PHASE 4 — Fine-tune (after 1–3 settle)

2.4 channel plan 1/6/11 (graph-color; co-channel pairs 92→35) + pin the 4 off-plan APs (128/108/108U7/salon) to 1/6/11.
2.4 min-RSSI ON for the 6 APs where it's OFF (615/608/505/517/622/salon) — note 505/517/615/608/622 are Floors 5/6 → DEFER with the rest of 5/6; do salon only this round.
Roaming for voice continuity: confirm 802.11k/v on CSCNet (r optional — test; some phones dislike 802.11r). Keeps calls alive when staff walk between APs.
min-RSSI tuning: only tighten where sticky-client far-AP behavior is proven; too aggressive blocks association.

PHASE 5 — Physical (separate visit, not tonight — but it caps results)

Re-terminate/replace the ~25 cables on ports stuck at 100 M (limits those APs' uplink throughput).
Chase the 3 offline switches (2nd Floor #2, 4th Floor #2, USW Pro Max 16); finish AP 108 cable run.
p38 (1st Floor USW) 4% tx-drop after the above.

4. Interdependency map (read before changing anything)

6 GHz BEFORE 5 GHz 40 MHz — else 5 GHz congestion just relocates. (Phase 2a before 3a.)
2.4 power MEDIUM not LOW — Low already over-thinned; going lower starves edge clients. (Phase 2b.)
AP-lock needs AP capacity — Lauren locked to 103 ⇒ 103 must be relieved (Phase 3c) or she trades mesh for congestion.
QoS is independent — do it first; it can't hurt RF and guarantees a voice win even before RF settles. (Phase 1.)
Disables + power-down compound — never do both aggressively in the same area; we already saw the satisfaction hit.
min-RSSI + power interact — raising min-RSSI while lowering power can orphan clients; tune one lever at a time.
Mesh-protected APs (2nd Floor Atrium, CC Bridge, salon, 206 U7 Pro, 108) — never disable; power changes only with watch.

5. Validation methodology (so we KNOW it worked)

Time-of-day controlled: compare to the same hour on a prior day, not just before/after minutes (load varies).
Metrics: 2.4 retry% + cu_interf (down), 5 GHz retry on busy APs (down), satisfaction (up, recovering from 30), client band split (5g→6g shift), per-voice-phone retry (Lauren, Dining, rm204/206/210).
Per-phase GATE: client counts steady, no AP offline, no new dead zone/complaint. Else rollback that step.
Rollback: every apply-radio/apply-wlan has a rollback JSON in .claude/tmp/; device-control poe-cycle for a hung AP (NOT force-provision). Power-up/width-80/re-enable are all safe reversals.

6. Out of scope tonight (explicit)

Floors 5 & 6 (MemCare) — all RF + the MemCare voice phones (.217/.218/.219/.220) DEFERRED per Howard.
Physical cabling / offline switches (Phase 5 — separate visit).
The 6 straggler phones — Howard re-keying separately; they'll benefit from the RF work regardless.

7. Open decisions for Howard

5 GHz channel plan: include clean-DFS (more diversity, tiny resilience risk) vs non-DFS-only (max resilience, more co-channel)? Recommendation: include clean-DFS.
QoS depth: UniFi WMM + DSCP-honor only, or also a pfSense WAN priority queue/limiter for RTP? Recommendation: both (additive).
802.11r on CSCNet: enable for seamless voice roaming, or k/v only (safer for mixed phones)? Recommendation: k/v now, test r on one phone first.
Tonight's stopping point: Phases 1–2 alone are a legitimate, lower-risk night; 3–4 can be a second night.

10 KiB Raw Blame History Unescape Escape