azcomputerguru/claudetools
1
0
Fork 0
Code Issues 58 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
48592bd16b12fb5b6753303fa071505def678cc9
claudetools/clients/cascades-tucson/session-logs/2026-06/2026-06-15-howard-cascades-wifi-rf-audit.md
Howard Enos 48592bd16b sync: auto-sync from HOWARD-HOME at 2026-06-16 07:26:57 
Author: Howard Enos
Machine: HOWARD-HOME
Timestamp: 2026-06-16 07:26:57
2026-06-16 07:27:06 -07:00

35 KiB
Raw Blame History

User

  • User: Howard Enos (howard)
  • Machine: Howard-Home
  • Role: tech

Session Summary

Resumed the Cascades wireless investigation that had stalled on 2026-05-16 (that session was read-only via the cloud API and blocked from per-AP RF data). The day's earlier syncs delivered the unblock: Mike (GURU-5070) vaulted infrastructure/uos-server-ssh-key + clients/cascades-tucson/unifi-ap-ssh and shipped a purpose-built unifi-wifi skill (audit/model-rank/optimize/apply/watch scripts + methodology references). Controller access via the vaulted key was verified, then the full live audit ran against the Cascades site (685f39068e65331c46ef6dd2) on the UOS controller (172.16.3.29).

The audit confirmed and quantified the 05-16 hypothesis with real controller data: 77 U7-Pro APs, all running 2.4GHz at auto (~full) power, 20MHz. 2.4 airtime (cu_total) is 74–94% busy on 75 radios with 61–81% of that being pure interference, serving ~1 client each, at TX-retry rates of 40–65% and single-digit AP satisfaction on the worst (209 sat=1/retr=65%, 139 sat=2/retr=49%, CC Bridge retr=48%). Neighbor-BSSID density is catastrophic on 2.4 (ch6=33,370, ch1=19,274, ch11=16,580). 5GHz is on 80MHz width on 76/77 APs (kills spatial reuse), biased to the busy upper channels (149/157). 6GHz is active on 75 radios but nearly empty of clients. 6 APs have 2.4 min-RSSI OFF (615, 608, 505, 517, 622, salon); 4 are off the 1/6/11 plan on auto (128, 108, 108U7-Pro, salon).

Diagnosis of "bad for SOME users": experience splits by band. Clients on 5/6GHz are fine; clients that land or stick on 2.4GHz (legacy phones, medical/IoT, poor band-steerers, or anything held by a min-RSSI-OFF AP from across the building) hit the saturated 2.4 radios with 40–65% retransmits and near-zero satisfaction.

Late in the session Howard raised the military-base / DFS factor — Cascades is in Tucson near Davis-Monthan AFB (+ TUS airport radar). This reverses the earlier "bias 5GHz toward DFS" recommendation: DFS channels (UNII-2/2e, ch52–144) will see frequent radar-detection events forcing channel-vacate + CAC silence, producing exactly the intermittent per-area dropouts reported. Revised plan uses non-DFS only (UNII-1 36–48 + UNII-3 149–161), which makes 40MHz width and 6GHz steering more important.

Key Decisions

  • Used the unifi-wifi skill end-to-end (audit-site, model-rank, optimize-radios, apply-radio dry-run) rather than ad-hoc Mongo queries — it encodes the multi-model methodology and coverage-safe model.
  • 5GHz channel plan revised to AVOID DFS due to proximity to Davis-Monthan AFB. Non-DFS UNII-1 + UNII-3 only; verify empirically against the controller's radar-detection event history before reconsidering DFS.
  • No changes applied — writes are intentionally gated until a read-WRITE controller admin is vaulted (infrastructure/uos-server-network-api-rw) and --apply is passed. Confirmed the apply path works via dry-run (Floor 3: 17 radios auto->low with rollback values captured).
  • Rollout will be per-zone (one floor at a time) with live before/after validation, never site-wide at once.

Problems Encountered

  • Controller SSH key not on Howard-Home and not in vault (earlier in day). Tested both vaulted keys (gururmm-server-physical, openclaw-fleet) against root@172.16.3.29 — both denied; OC-5070/OC-Mac unreachable over Tailscale, fleet key denied on OC-Beast. Resolved by coord-requesting Mike to vault the UOS key; he did (infrastructure/uos-server-ssh-key), picked up on sync.
  • Cloud Site Manager API insufficient (re-confirmed). Device objects carry no site field and no RF/channel/power/uplink data — only online/offline + firmware. Cannot drive RF tuning. The Mongo-via-SSH path (now available) is required.

Configuration Changes

  • None to Cascades infra. Read-only audit + dry-run only.
  • No repo files edited beyond this session log.

Credentials & Secrets

  • infrastructure/uos-server-ssh-key (vaulted by Mike this day) — root SSH key for the UOS controller 172.16.3.29; used by uos-mongo.sh / unifi-wifi scripts. This is the DATA plane (read), not an API write session.
  • clients/cascades-tucson/unifi-ap-ssh (vaulted by Mike this day) — device-auth cred for SSHing directly into Cascades APs (used by watch-ap.sh; needs site VPN for L3 reach to 192.168.2.x/3.x).
  • Needed, not yet created: infrastructure/uos-server-network-api-rw (read-write controller admin) to apply radio changes; infrastructure/uos-server-network-api (read-only admin) to wire live-stats Plane 2 validation.

Infrastructure & Servers

  • UOS controller: 172.16.3.29 (Rocky 9 VM "Unifi" on Jupiter 172.16.3.20); UniFi-OS HTTPS on 11443 (not 8443). Mongo ace on 127.0.0.1:27117 inside rootless podman uosserver. Cascades site_id 685f39068e65331c46ef6dd2.
  • Cascades wireless: 77 U7-Pro APs, ~550 clients. Firewall = pfSense 192.168.0.1 (site VPN endpoint; .ovpn comes from pfSense OpenVPN Client Export, NOT UniFi). APs on 192.168.2.x/3.x.
  • Location/RF: Tucson, near Davis-Monthan AFB + TUS radar → DFS unreliable.

Commands & Outputs

bash .claude/scripts/uos-mongo.sh --sites | grep -i casc        # 685f39068e65331c46ef6dd2  Cascades (access OK)
bash .claude/skills/unifi-wifi/scripts/audit-site.sh cascades    # config + neighbor-density + flags
bash .claude/skills/unifi-wifi/scripts/model-rank.sh cascades 7 ng
bash .claude/skills/unifi-wifi/scripts/optimize-radios.sh cascades 14 ng   # power-down 74, disable 0, keep 1
bash .claude/skills/unifi-wifi/scripts/apply-radio.sh cascades ng power low --zone "Floor 3"  # DRY-RUN: 17 radios auto->low

Key audit output: 2.4 cu_total 74–94% / interf 61–81% / ~1 client; retry 40–65%; ch6=33,370 neighbors; 5GHz 80MHz on 76/77; 6GHz active 75 but empty; min-RSSI OFF on 615/608/505/517/622/salon.

Pending / Incomplete Tasks

  • Vault read-WRITE controller admin infrastructure/uos-server-network-api-rw (blocks applying any radio change). Candidate: coord-request Mike.
  • Vault read-only infrastructure/uos-server-network-api to wire live-stats Plane 2 (before/after cu_total/satisfaction validation).
  • Apply Phase A (2.4 power-down to Low) per-zone with live validation, once RW cred exists.
  • 5GHz: 80->40MHz width; non-DFS channel plan (UNII-1 36–48 + UNII-3 149–161); 6GHz steering for capable clients.
  • Min data rates (kill 1–11Mbps, 2.4 floor 12/24Mbps); set 2.4 min-RSSI -75/-76 on the 6 OFF APs; pin 4 off-plan APs to 1/6/11.
  • Pull controller radar-detection event history to empirically confirm DFS unusability.
  • Secondary: fix the non-working ".ovpn / Download configuration" — likely pfSense OpenVPN Client Export (192.168.0.1), not UniFi. Needed for watch-ap.sh live validation.
  • AP 108 offline — KNOWN, needs a new cable run (per Howard); ignore for now. Also a stale duplicate controller object (108 vs 108U7 Pro) to clean up later.

Reference Information

  • Coord message to GURU-5070 re UOS key: id a4b385ad-4fbb-4097-a066-099622080055; backstop todo 3bd12a14-2b51-4c11-8f76-3f835b07e8dc (--user mike).
  • unifi-wifi skill: .claude/skills/unifi-wifi/ (methodology.md, data-access.md, interference-model.md).
  • Prior wireless log: clients/cascades-tucson/session-logs/2026-05-16-howard-wireless-diagnostic.md.
  • UOS system wiki: wiki/systems/uos-server.md.

Update: 20:40 PT — RW cred arrived, live (Plane 2) re-look, live-stats.sh accuracy fixes

Mike vaulted the RW controller admin (infrastructure/uos-server-network-api-rw) within ~30 min of the request, so the live Network API (Plane 2) became available. Re-audited Cascades live and, in doing so, found + fixed accuracy bugs in unifi-wifi/scripts/live-stats.sh that had skewed the earlier read.

live-stats.sh bugs fixed (held a coord lock; messaged Mike e8be889f)

  1. stat/device output hard-capped at head -60 -> only ~15 of 77 APs were shown (we'd been judging the whole site from a 15-AP sample). Removed the cap; verified 77/77 now.
  2. satisfaction was read per-radio (always -1 on this controller). DEVICE-level satisfaction IS populated -> switched to d['satisfaction'].
  3. tx_retries was the raw cumulative counter (scales with traffic, misleading). Switched to radio_table_stats.tx_retries_pct (a true rate).
  4. --clients was [:40] unsorted (hid 90% of 574 clients). Now sorts worst-by-satisfaction and prints signal/noise/retry%/satisfaction_reason.
  5. RF-neighbor table left as a documented TODO in the header (not a single API field; must be built from the rogue BSSID cross-ref vs each AP's vap_table). It's what unlocks confident radio DISABLES; until then power-down/channel/width are the safe levers.

Corrected diagnosis (the data fix changed a conclusion)

  • Accurate avg retry RATE: 2.4GHz 11.2% > 5GHz clear 9.0% ~= 5GHz DFS 8.4%. My mid-session claim that "5GHz/DFS is now the #1 problem" was an ARTIFACT of the raw counter + 15-AP sample and is WITHDRAWN. On the rate, DFS is NOT retrying worse than clear channels.
  • 2.4GHz is the primary pain band (highest retry; 27 of the 40 worst clients are on 2.4, retry 11-42%, mostly IoT/legacy: Ring cams, robotic cleaner, smart plugs, EPSON printer, Poly phone, handheld scanners, Watch). The original 2.4 power-down/prune plan stands as #1.
  • DFS = resilience risk, not throughput killer. 55/77 5GHz radios on DFS near Davis-Monthan = radar-vacate exposure (client drops), worth moving off DFS for stability, but not urgent for performance.
  • 6GHz still dead (1 client of 574) — top untapped clean/non-DFS capacity; steering remains a key opportunity.
  • AP-level satisfaction 95-100 across the fleet — network is healthy on average; pain is in the client tail = consistent with "bad for SOME users."

Cascades live snapshot (2026-06-15 ~20:30 PT)

  • 77/77 APs reporting; 574 wireless clients. Band split: na (5GHz) ~87 / ng (2.4) ~21 sampled per pull / 6e ~1.
  • 2.4 cu_total 69-94% live (saturation confirmed). 5GHz cu_total mostly <40%.
  • Worst clients: RingStickupCam (sat 60), Galaxy-A32-5G (60), DIRECTV (71), Samsung on ch1 (76, retry 42%).

Updated next steps

  • Build Floor-pilot DRY-RUN: 2.4 power-down to Low on one floor (e.g. Floor 4), validate live cu_total/retry% before+after via the fixed live-stats.sh. Get explicit go before --apply.
  • Implement the AP-to-AP RF-neighbor table (rogue BSSID x vap_table) to enable safe DISABLEs.
  • 6GHz steering plan; 5GHz 80->40MHz + non-DFS channel plan (resilience).
  • Coord msgs this update: RW-cred request 6b98282f (+todo cbb355ef); live-stats fix e8be889f.
  • pfSense .ovpn (Howard handling) — needed for per-AP watch-ap.sh live stream.

Update: 22:05 PT — site VPN up (split-tunnel), direct-AP cross-validation, watch-ap.sh fixed

Howard got the Cascades OpenVPN running. The pushed config is full-tunnel (pfSense redirect-gateway) which killed his local internet + Tailscale (and my tool calls, since they run on his machine). Fixed with a split-tunnel client profile: C:\Users\Howard\Documents\ComputerGuru Connect v2\Files\cascades-splittunnel.ovpn — copy of the working profile + pull-filter ignore "redirect-gateway", pull-filter ignore "dhcp-option DNS", route 192.168.0.0 255.255.252.0, and the VPN creds embedded inline (<auth-user-pass> = vault clients/cascades-tucson/pfsense-openvpn-howard, user Howard). After import+connect: internet + Tailscale + direct AP reach all work simultaneously.

Rescan + ground-truth verification (Howard: "test until sure there are no issues")

  • Controller rescan: 77/77 APs, config identical to prior pull — no drift, scripts stable.
  • Cross-validated controller data against AP ground truth (direct SSH to APs, mca-dump + iw survey) on TWO APs (132 @192.168.2.34, 335 @192.168.3.46): controller cu_total/clients match the radios' own numbers within polling jitter (AP132 2.4: controller 81% / mca 87% / iw survey 92% busy, -95dBm noise; AP335: 2.4 controller 81% vs mca 79%, 5GHz 10%=10% exact, clients 2=2 exact). Conclusion: the controller is accurate; the diagnosis stands on solid data.
  • New concrete fact: U7-Pro 2.4 radios report min_txpower 6 / max 23 dBm → "Low" = 6 dBm (aggressive cell shrink — good for the power-down plan).

watch-ap.sh fixed (2 bugs found by testing; validated live; Mike notified b0cf24a5)

  1. Hard sshpass dependency → failed on Windows (no sshpass). Added SSH_ASKPASS fallback (temp askpass helper + SSH_ASKPASS_REQUIRE=force); uses sshpass if present. Fallback needs ssh from PATH = MSYS ssh on Windows (system OpenSSH can't exec a shell askpass, CreateProcessW err 193); Linux/macOS system ssh is fine.
  2. Radio list included virtual APs (wifi0ap0..) + mld-wifi0 → duplicate lines AND sh: malformed ?: operator (hyphen in mld-wifi0 broke ${ACT_mld-wifi0:-0}). Filter now grep -E '^(wifi[0-9]+|ath[0-9]+)$'. Validated: clean 3-line/tick (wifi0 ch6 92%, wifi1 8%, wifi2 0%).
    • Minor (not fixed, interactive use is fine): script's grep|head block-buffers when piped non-interactively; would need stdbuf -oL to be pipeable.

Issues found this round (none in the data; environment/VPN only)

  • 192.168.0.0/24 home-LAN overlap: Howard's home LAN is 192.168.0.x, so Cascades devices on 192.168.0.x (pfSense 192.168.0.1, cascadesDS 192.168.0.120) are shadowed/unreachable over the VPN from Howard-Home. APs on 192.168.2.x/3.x are unaffected. Use Tailscale for controller; for pfSense/NAS over VPN would need a home renumber or a more specific route trick.
  • OpenVPN Connect DCO↔TAP failover intermittently drops the 192.168.2.x route mid-session (caused two transient watch-ap timeouts). Community OpenVPN GUI is the more stable client if it recurs.

State / next

  • live-stats.sh + watch-ap.sh both fixed + validated; controller data verified accurate.
  • NEXT (per Howard): one more clean retest pass, then build the AP-to-AP RF-neighbor table (now feasible via direct AP iw scan/scan_radio_table) to unlock safe radio DISABLEs; then the Floor-4 2.4 power-down pilot with live before/after validation.
  • Coord this update: watch-ap.sh fix b0cf24a5.

Update: 23:03 PT — RF-neighbor table DEFINITIVELY unobtainable; survey-dump is the usable substitute

Exhaustively investigated whether an AP-to-AP RF-neighbor table (which of OUR managed APs each AP hears + RSSI) is obtainable, to enable data-driven radio DISABLE decisions. Cross-checked against two rounds of UniFi's own AI guidance. Conclusion: NOT obtainable on this deployment by any safe/supported method. Stop chasing it.

Why it's impossible here (all paths tested)

  • rogue collection AND Network API stat/rogueap (7,354 records): both FILTER OUT our managed APs — only foreign BSSIDs (our OUI 0c:ea:14 = 2 hits, CSCNet = 0, is_ubnt = 2). By design.
  • AP iw dev wifiX scan: firmware-blocked — "Operation not permitted (-1)" even as uid=0(root), on all radios incl. an idle 6GHz radio. UniFi locks raw scanning on operating radios.
  • iw scan dump (cached): empty. device.scan_radio_table: empty []. No spectrum/neighbor collection exists. hostapd_cli show_neighbor (802.11k): FAIL on every VAP.
  • mca-cli-op: interactive CLI (bare invocation spews prompts — feed </dev/null). Subcommand info works (firmware 8.6.11.18870, U7-Pro) but exposes NO neighbor/RF data; wireless/radio/ status return nothing useful.
  • Channel AI / channelplan: stale (last run 2026-04-06); exposes only resulting channel lists, not the neighbor graph. UniFi consumes neighbor data internally but exposes it nowhere.
  • Roam graph (ace_stat.wifi_connectivity_event): too sparse (50 clients, 6 edges) — Cascades devices are largely stationary (senior living), so roams can't prove coverage redundancy.
  • A disruptive controller spectrum-scan would only return per-channel RF ENERGY, not a managed-AP BSSID list — so even disruptive testing won't yield the neighbor table. NOT worth running.

NET: automated radio DISABLES are not data-supportable at Cascades. optimize-radios.sh "disable=0" is correct. Any disables must be MANUAL (UniFi UI RF Environment view + on-site).

The usable WIN: non-disruptive per-AP per-channel RF environment via iw survey dump

iw dev wifiX survey dump returns, per AP, per-channel noise floor + airtime busy% across the WHOLE band (not just in-use) — already populated by background scanning, zero disruption. This is real measured RF data that supports a DATA-DRIVEN CHANNEL PLAN. Sample AP 132 (Rec Room):

  • 2.4 GHz: ch1 89%, ch6 95% (noise -65dBm), ch11 96% — every channel saturated.
  • 5 GHz: UNII-1/low (36-48) 1-20% = cleanest; ch157 31% = congested → confirms bias 5GHz LOW.
  • 6 GHz: ~0% across nearly all channels → wide-open lane, confirms 6GHz steering is the big win.

Decisions / state

  • Do NOT pursue the AP-neighbor table or disruptive spectrum-scan further (definitively closed).
  • Channel plan can be MEASURED from survey dump (non-disruptive) instead of inferred.
  • Two safe high-value moves remain: (1) 2.4 GHz power-down per-zone; (2) measured non-DFS channel plan + 6GHz steering.

Next steps

  • Build survey-collect.sh (unifi-wifi skill): sweep all 77 APs' iw survey dump (non-disruptive) → fleet-wide measured per-channel busy%/noise map → drive the channel plan.
  • Floor-4 2.4 GHz power-down pilot with live before/after (watch-ap.sh + live-stats.sh).
  • Record the neighbor-data dead-end + survey-dump method in skill docs (interference-model.md, watch-ap.sh/live-stats.sh headers) so it is never re-chased.
  • (Howard) has more information to add — pending.
  • Confirmed env: firmware 8.6.11.18870 on U7-Pro; AP device-auth SSH = uid 0 (root); split-tunnel VPN gives AP reach on 192.168.2.x/3.x (192.168.0.x shadowed by home LAN).

Update: 23:24 PT — BREAKTHROUGH: AP-to-AP SNR matrix IS obtainable; neighbor-collect.sh built; disables now data-supportable

REVERSES the earlier "AP-neighbor table unobtainable / disables not data-supportable" conclusion. The data DOES exist — not via any controller API/DB (all of which filter our managed APs) but on each AP in /proc, populated NON-DISRUPTIVELY by UniFi's background RRM scanning:

  • /proc/ui_neighbor/ess_ap_list — full list of managed APs this AP hears (serial/band/channel)
  • /proc/ui_neighbor/ssid/<n> — same neighbors WITH SNR (per scan-vap, per band) Discovered via strings $(which mcad) | grep neighbor → /proc/ui_neighbor/ess_ap_list + load_ssid_neighbor_cache. /proc/ui_neighbor exists on every UniFi AP → fleet-generic.

NEW: .claude/skills/unifi-wifi/scripts/neighbor-collect.sh (built + validated, Mike notified 3dbe2437)

Logs into controller for the mac/bssid->AP-name map, SSHes each online AP (sshpass OR SSH_ASKPASS fallback), reads /proc/ui_neighbor, maps neighbors to AP names, emits the AP-to-AP SNR adjacency matrix + a redundancy summary (APs heard by >=2 neighbors at SNR>=N = power-down/disable candidates). Usage: neighbor-collect.sh <site> [ap-ssh-vault-path] [snr_min]. Run FOREGROUND. Validated on Cascades: 74/74 APs reporting. Physically sensible (115<->116 SNR 63, 121<->221 SNR 65 = adjacent). Redundancy: 73/74 APs have >=2 strong (SNR>=20) 2.4GHz neighbors, 72/74 on 5GHz, 65/74 on 6GHz → nearly every AP is coverage-redundant on 2.4 → aggressive 2.4 pruning is data-supported, and individual radio DISABLES are now decidable from data.

Three bugs found+fixed building it (apply to any loop-over-AP script)

  1. Python wrote the AP-list temp file in Windows TEXT mode -> CRLF -> bash read left \r on the IP -> ssh "hostname contains invalid characters" on EVERY AP (the "reachable 0" symptom). Fix: open(...,newline='\n') + ip="${ip%$'\r'}" strip in the read loop.
  2. Must run FOREGROUND — a fully detached background process can't spawn the SSH_ASKPASS helper (both background runs got reachable 0; foreground after the CRLF fix got 74).
  3. ssh in a while read loop needs </dev/null or it consumes the loop's stdin.

Skill capability assessment (Howard: "must work for ALL clients / any unifi issue")

For WiFi/RF, the skill now gathers essentially everything obtainable, all site-parameterized (works for any of the 49 UOS sites): config (uos-mongo), live per-AP+per-client (live-stats), interference (audit-site + neighbor-collect), per-AP ground truth (watch-ap: mca-dump/iw survey), the AP-to-AP SNR matrix (neighbor-collect), and gated apply (apply-radio). Per-client gap: watch-ap/neighbor-collect default the AP device-auth cred to cascades — other clients need their own clients/<x>/unifi-ap-ssh vaulted (overridable via arg). Out of current scope (it's unifi-WIFI): switch/PoE, gateway/WAN/firewall, adoption — reachable via the same access layer (uos-mongo/controller API/device SSH) but not yet wrapped as scripts.

Pending follow-ups

  • Wire neighbor-collect redundancy output into optimize-radios.sh (data-backed disables).
  • Fold survey-dump (per-channel busy/noise) + dmesg DFS-radar recipes into the skill as reusable collectors (the two ad-hoc datasets from this session).
  • Per-client AP device-auth creds for other clients when extending beyond Cascades.
  • Floor-4 2.4 power-down pilot (still the next live change; nothing applied yet).
  • Coord this update: neighbor-collect.sh announce 3dbe2437.

Update: 23:50 PT — gap 1 + gap 2 closed: survey-collect + dfs-check built; SKILL marked wifi-working / switches-WIP

GAP 1 (fold ad-hoc recipes into reusable collectors) — DONE, both validated on Cascades:

  • survey-collect.sh <site> [ap-ssh-vault-path] — per-AP per-channel busy%/noise from iw survey dump -> in-use busy% + cleanest non-DFS channels per band. 72 APs reported. Result: 2.4 saturated 94-97% on every channel; 5GHz cleanest non-DFS = ch36/48/153/161; 6GHz ~0%.
  • dfs-check.sh <site> [ap-ssh-vault-path] — per-AP dmesg radar history, PRECISE patterns (excludes the "cached ifindex" false-positive). Result: 55/75 APs on DFS, ZERO real radar events fleet-wide -> DFS empirically low-risk at Cascades despite Davis-Monthan proximity (confirms the retry-rate finding; the DFS concern was theoretical).

GAP 2 (scope) — DONE via SKILL.md Status block: WiFi monitoring+tuning = WORKING for every UOS site we monitor (audit-site, live-stats, model-rank, optimize-radios, survey-collect, dfs-check, neighbor-collect, watch-ap, apply-radio — all site-parameterized). Switches/PoE, gateway/WAN/ firewall, adoption = WIP (access layer reaches them; need dedicated collectors). Per-client: vault each client's own clients/<x>/unifi-ap-ssh, pass as the script arg.

CROSS-PLATFORM LESSON (baked into both new scripts): pass temp paths to python via ARGV — MSYS translates POSIX->Windows for python.exe; $TMP embedded in a python -c string is NOT translated and fails ("No such file or directory") on Windows. Use python - "$TMP/x" <<'PY' ... sys.argv[1].

Coord: collectors+status announce c3ccaa07. Next: wire neighbor-collect redundancy into optimize-radios.sh; Floor-4 2.4 power-down pilot (still nothing applied to live radios).

Update: 00:05 PT (06-16) — wired neighbor SNR matrix into optimize-radios.sh (disables now data-backed)

Closed the last follow-up: the AP-to-AP SNR matrix now DRIVES optimize-radios.sh disable decisions.

  • neighbor-collect.sh: NBR_JSON=<path> env now also emits machine-readable adjacency {ap:{band:{nbr:snr}}} (74 APs, ~21KB).
  • optimize-radios.sh: NEIGHBOR_JSON=<path> env now uses the measured bidirectional SNR overlap (A hears B AND B hears A, SNR>=NBR_SNR_MIN default 20, any band) as the coverage-adjacency source, replacing the sparse roam graph. Roam graph remains the fallback when NEIGHBOR_JSON unset.

Workflow: NBR_JSON=.claude/tmp/cascades-nbr.json bash .../neighbor-collect.sh cascades NEIGHBOR_JSON=.claude/tmp/cascades-nbr.json bash .../optimize-radios.sh cascades 14 ng

VALIDATED on Cascades 2.4GHz: roam-graph -> POWER-DOWN 74 / DISABLE 0; neighbor-SNR -> POWER-DOWN 65 / DISABLE 9 / KEEP 1, est. 621% interference-airtime removed, each disable listing its covering neighbors (127->128; 229->128; 248->348; 330->128; 445->347/348/247; 428->128; 622->505/615/517; Kitchen->Memcare TV room; salon->128). The disable capability we concluded was impossible is now data-backed for any site with a neighbor-collect run.

IMPL NOTE (bug fixed): injecting the full matrix as a JS object literal CRASHED the old mongo shell (SpiderMonkey GC-during-compile on a ~21KB literal). Fix: precompute bidirectional strong NAME-pairs in python, inject a compact ';'-joined flat STRING, parse in JS. (Pattern for any large data -> mongo-shell injection: flat string, not object literal.)

Still gated: apply-radio.sh has NO disable action (power/channel/width only); disables remain a reviewed, per-zone, live-validated MANUAL step. Coord: wire announce 68cae757. Remaining open: Floor-4 2.4 power-down pilot (still nothing applied to live radios).

Update: 2026-06-16 00:03 PT — skill-build focus: apply-radio generalized + write path validated + ROADMAP

Per Howard: pause the Cascades production fix; build the unifi-wifi skill into a fully working tool for any client. Used Cascades 0-client radios as the safe sandbox (6 GHz is ~0-client fleetwide; no whole-AP had 0 clients).

ADDED references/ROADMAP.md — full A/B/C/D plan: (A) apply side, (B) multi-client enablement, (C) switches/gateway WIP, (D) robustness. Cross-platform gotchas captured for future scripts.

GENERALIZED apply-radio.sh from power-only to 4 radio_table actions (same gated REST mechanism, field-generic via a compact FIELDS JSON shared by the dry-run preview and the apply python; rollback auto-saved): power low|medium|high|auto| -> tx_power_mode (+tx_power) width 20|40|80|160 -> ht channel |auto -> channel minrssi off|on|- -> min_rssi_enabled (+min_rssi)

WRITE PATH VALIDATED end-to-end (first real --apply): on Floor 6 0-client 6e radios (608, 622) applied ht 160->80, controller confirmed ht=80, then reverted to 160 (state restored). Dry-run previews for width/minrssi verified correct (incl. 615 U6-Pro correctly has no 6e radio).

radio_table field names confirmed: channel, ht (width), tx_power_mode/tx_power, min_rssi + min_rssi_enabled. NO radio enable/disable field -> DISABLE not implemented (ROADMAP A; discover via UI-toggle + device-JSON diff). min-data-rate/band-steering are wlanconf-level -> separate path.

SKILL.md apply section updated with all 4 actions + the validated-write-path note. Coord: 6aac1298. Next per roadmap: A (disable mechanism, wlanconf knobs, channel-plan apply) then B (per-client creds/VPN).

Update: 2026-06-16 00:24 PT — radio DISABLE mechanism found; apply-radio disable/enable + --ap (validated)

Howard toggled 6GHz off on AP 622 in the UI; diffed the device-JSON before/after. Disable mechanism: radio_table[].tx_power_mode = "disabled" (radio stays in table, channel/ht untouched). Re-enable = tx_power_mode "auto". Same field as power. (Other diffs were timestamps/cfgversion + JSON int/str noise.)

apply-radio.sh now supports actions: power | width | channel | minrssi | disable | enable, plus a new --ap "" single-AP filter (correct scope for disables). Arg parsing reworked so disable/enable take no value. VALIDATED full enable->disable->enable cycle on 622's 0-client 6GHz via --apply; 622 restored to ON (auto). Workflow: optimize-radios (NEIGHBOR_JSON) disable list -> apply-radio <site> <band> disable --ap <name> --apply one AP at a time, watch-ap validated.

SKILL.md + ROADMAP.md updated (disable now done). apply-side now covers all radio_table knobs; only wlanconf-level (min-data-rate, band-steering) remains -> future apply-wlan.sh. Coord: 6aac1298-> this: msg sent.

Update: 2026-06-16 00:40 PT — multi-client enablement + WLAN-level apply (wlanconf); both validated

MULTI-CLIENT (B): NEW scripts/sites.sh lists all ~49 UOS sites (APs/SW/GW) + per-client AP-cred readiness — the entry point for other clients. Controller-side scripts validated on other clients (Glabman 6 APs, Sonoran Glass 5 APs) with zero setup. AP-side scripts now print the exact vault command when a client's clients//unifi-ap-ssh is missing (controller-side still works). SKILL.md Multi-client section added. (Per-client AP creds vaulted as clients come online.)

WLAN-LEVEL APPLY (A2): NEW scripts/apply-wlan.sh (wlanconf, gated like apply-radio, --wlan filter): minrate ng|na auto|off| (kill 1-11Mbps: minrate ng 12) steer on|off (5GHz roaming_assistant) GOTCHA found+fixed: manual min rate only applies when minrate_setting_preference=manual (else controller auto-manages + silently reverts — first PUT looked [ok] but value stayed 1000). Script now sets the preference; 'minrate ... auto' restores controller management. WRITE PATH VALIDATED on a 0-client WLAN (Green Valley Computer Club): minrate ng 12 -> persisted (pref=manual, 12000) -> restored to auto.

apply-radio (radio_table: power all-states/width/channel/minrssi/disable/enable + --ap/--zone) and apply-wlan (wlanconf: minrate/steer) now cover the WiFi apply surface, all gated + rollback + validated on 0-client sandboxes. ROADMAP updated. Coord: this msg. Remaining: classic band-steering field, per-client creds/VPN, switches/gateway collectors (C).

Update: 2026-06-16 00:58 PT — band-steering field FOUND + wired into apply-wlan (validated)

Modern UniFi (Network 8.x) has NO classic 'bandsteering_mode'. Located its replacements by inspecting wlanconf + the setting collection (no UI toggle needed this time): no2ghz_oui -> the band-steering toggle (keep 5GHz-capable OUIs off 2.4) wlan_bands (["2g","5g"]) -> per-WLAN band membership (set ["5g"]/["5g","6e"] to force clients up) bss_transition -> 802.11v BSS-TM roaming_assistant_na_* -> 5GHz sticky-client kicker fast_roaming_enabled -> 802.11r (left out — risky for IoT/medical)

Wired into apply-wlan.sh as actions: bandsteer (no2ghz_oui) | bands (wlan_bands) | steer (roaming_assistant) | bsstm (bss_transition), alongside minrate. Validated on the 0-client Green Valley WLAN: bandsteer on->verify->off, bsstm toggle, bands 5g dry-run; restored to original (bss_transition=true).

Also surfaced setting.radio_ai (Channel AI: channels_na includes DFS, ht_modes_na=20/40, enabled=false)

  • setting.connectivity — possible future apply targets (pin the channel plan).

WiFi APPLY SURFACE NOW COMPLETE: apply-radio (power all-states/width/channel/minrssi/disable/enable, --zone/--ap) + apply-wlan (minrate/bandsteer/bands/steer/bsstm, --wlan), all gated + rollback + validated on 0-client sandboxes. Coord: this msg. Remaining (ROADMAP C): switch/PoE + gateway collectors.

Update: 2026-06-16 01:09 PT — WiFi control surface complete: full wlanconf knobs + device-lock + client controls

(a) DEVICE LOCK / CONTROL:

  • apply-wlan aps <ap-names|all> -> broadcasting_aps (resolves AP NAMES->MACs; restricts SSID to APs = the "lock device to AP" lever, since UniFi has NO native per-client AP pin). Validated dry-run: aps 622 --wlan CSCNet -> broadcasting_aps=[0c:ea:14:3e:65:a2].
  • apply-wlan macfilter off|allow|deny -> mac_filter_*. Dry-run validated.
  • NEW client-control.sh: block|unblock|kick a MAC via cmd/stamgr (gated). Validated block+unblock on dummy 00:00:00:00:00:01 (rc:ok), cleaned up.

(b) REMAINING wlanconf TUNING KNOBS added to apply-wlan: wlan(enable/disable), dtim (dtim_mode=custom gate), mcast(mcastenhance), bcfilter, rrm(802.11k), ftroam(802.11r, warns), isolation(l2_isolation), hidessid. dtim apply-validated on 0-client Green Valley (mode default->custom, reverted). Others via the proven gated REST path.

apply-wlan restructured: SITE resolved before the action case (so aps can map names->macs). WiFi APPLY SURFACE COMPLETE: apply-radio (radio_table) + apply-wlan (wlanconf, 15 actions) + client-control (cmd/stamgr), all gated + rollback + validated on 0-client sandboxes / dummy MAC. SKILL.md + ROADMAP updated. Coord: this msg. NEXT: switches/PoE collector (ROADMAP C).

Update: 2026-06-16 01:13 PT — switches: switch-audit.sh shipped (first non-WiFi collector, ROADMAP C)

NEW scripts/switch-audit.sh [--all-ports] (controller-side, read-only, any site): per switch -> PoE budget used/total, ports up/total, FLAGS: UNDERSPEED (gig-capable port linked 10/100 = FastEthernet uplink problem), DROPS (rate-based tx_dropped/tx_packets>2%), ERRORS (>1000), POE-FAULT (poe_enable & !poe_good), POE-BUDGET (>85%), + OFFLINE switches. port_table fields: speed/speed_caps/poe_*/rx_tx_errors.

VALIDATED on Cascades (12 switches): found ~25 ports linked at 100M but gig-capable (systematic cabling/NIC issue = the long-suspected FastEthernet uplink problem, now fleet-visible), offline switches (2nd Floor #2, 4th Floor #2, USW Pro Max 16), PoE budgets healthy, one real 4.1% tx-drop port (p38 on 1st Floor USW). Refined DROPS to rate-based (cumulative counters were noisy).

SKILL coverage now: WiFi (monitor+tune+full apply incl. device lock/client controls) + switch/PoE audit, all controller-side multi-client + AP-side where cred vaulted. ROADMAP remaining: gateway/WAN/firewall + adoption collectors. Coord: this msg.

Update: 2026-06-16 01:20 PT — gateway/WAN + site-health collector (gw-audit.sh); ROADMAP C largely done

NEW scripts/gw-audit.sh (controller-side, any site, read-only) via stat/health + gateway device: WAN status/IP/uplink/duplex + gw CPU/mem/uptime; internet(www) status/latency/drops/last-speedtest; health rollup (APs/switches/gw adopted vs disconnected vs pending, client user/guest/iot, firmware- upgradable); flags WAN/internet not-ok, latency>80/drops>5, WAN down, disconnected/pending, gw cpu>85/ mem>90. Third-party-firewall sites (num_gw=0) handled. Fixed: gw_name fallback to device name; drops threshold >5 (was >0, too sensitive).

Validated: Sonoran Glass (USG 3P [UGW3]) -> all healthy (cpu2%/mem21%, latency 14-24ms); Cascades (pfSense) -> "no UniFi gateway" + correctly caught 2 disconnected APs + 3 disconnected switches.

SKILL now end-to-end for any client: WiFi (monitor + tune + full apply incl device-lock/client-control), switch/PoE audit, gateway/WAN/site-health audit, sites.sh discovery. ROADMAP remaining: deeper firewall/VPN policy + adoption REMEDIATION (adopt/restart), channel-plan apply, per-client AP creds/VPN. Coord: this msg.

Update: 2026-06-16 07:26 PT — adoption remediation (device-control.sh) + channel-plan apply (channel-plan.sh)

NEW device-control.sh adopt|restart|provision|locate|unlocate|upgrade [--apply] (cmd/devmgr, gated, controller-side) — remediates what gw-audit flags. Validated locate+unlocate on AP 622 (rc:ok).

NEW channel-plan.sh ng|na [--apply] (NEIGHBOR_JSON + SURVEY_JSON): ng: graph-color 1/6/11 via the neighbor matrix (strong neighbors differ). na: per-AP lowest-cost NON-DFS channel = measured busy% (survey) + neighbor-collision penalty. Reports co-channel pairs before/after; gated apply per-AP rest/device + rollback. DRY-RUN validated on Cascades: ng 92->35 co-channel pairs (51 APs change); na 20->0 + ALL moved off DFS onto non-DFS (radar-safe) (71 APs). Not applied (production paused); PUT mechanism proven. Added SURVEY_JSON= emit to survey-collect.sh ({ap:{band:{channel:busy%}}}).

SKILL now: WiFi (monitor+tune+full apply+device-lock+client/device control+channel-plan) + switch/PoE audit + gateway/WAN/site-health + multi-client. ROADMAP nearly clear (deeper firewall/VPN policy + per-client AP creds/VPN remain). Coord: this msg.

Reference in New Issue View Git Blame Copy Permalink