diff --git a/clients/cascades-tucson/session-logs/2026-05-16-howard-wireless-diagnostic.md b/clients/cascades-tucson/session-logs/2026-05-16-howard-wireless-diagnostic.md new file mode 100644 index 0000000..c942477 --- /dev/null +++ b/clients/cascades-tucson/session-logs/2026-05-16-howard-wireless-diagnostic.md @@ -0,0 +1,265 @@ +# Session Log — Cascades Wireless Network Diagnostic + +## User +- **User:** Howard Enos (howard) +- **Machine:** Howard-Home +- **Role:** tech +- **Session:** 2026-05-16 + +--- + +## Session Summary + +Investigated poor wireless performance at Cascades Tucson using the UniFi cloud API (api.ui.com) with a provided API key. Work began by enumerating all sites under the account to locate the Cascades site, then pulled device inventory and site-level statistics. The session was primarily diagnostic — no configuration changes were made per Howard's explicit instruction to remain read-only. + +Cascades is managed on a shared UniFi OS Server (UOS Server) hosted at 172.16.3.29 on the ACG internal network. The cloud API exposes basic device status and site-level aggregates, but does not expose per-AP RF statistics or per-client data. Howard supplemented the API data by pasting Channel AI output and client list data directly from the UniFi dashboard, which enabled deeper analysis. + +The primary finding was severe 2.4 GHz saturation across the entire building. With 64+ U7 Pro APs deployed in a dense multi-floor building, all three non-overlapping 2.4 GHz channels (1, 6, 11) are running at 60–92% busy and 72–93% interference. This is caused by too many APs transmitting on the same channels simultaneously — a fundamental co-channel interference problem. Several APs were also found offline, and two APs have Fast Ethernet uplinks instead of Gigabit, capping those rooms at 100 Mbps. + +The session concluded with a prioritized list of 2.4 GHz power reduction and disable recommendations, segmented by channel group. Work on implementing those changes was deferred to a follow-up session. + +--- + +## Key Decisions + +- Kept the session strictly read-only per Howard's instruction — no UniFi settings were changed. +- Used the cloud API (api.ui.com) as the primary data source. Attempted but failed to access the local controller API at 172.16.3.29 due to path/auth limitations; the cloud API key does not authenticate to the local Tomcat instance. +- Created session log as a separate file (not appended to today's caregiver AD session) due to unrelated subject matter. +- Deferred 2.4 GHz power changes to a follow-up session after documentation. + +--- + +## Problems Encountered + +- **Cloud API does not expose per-AP RF stats or per-client data.** Only site-level aggregates and basic device status available via api.ui.com. Resolved by having Howard pull Channel AI and client list from the dashboard and paste the data into the conversation. +- **Local controller at 172.16.3.29 not fully accessible.** Port 8080 (Tomcat/UniFi) returns HTTP 400 for API calls because it requires HTTPS and session auth — the cloud API key does not work here. Port 8443 responds but returns empty 404 for all paths (likely a reverse proxy without the correct routes). Port 443 is unreachable from Howard-Home even via Tailscale. +- **developer.ui.com docs are JavaScript-rendered.** WebFetch cannot render the page. Worked around by searching community docs and GitHub and by probing API endpoints directly via curl. +- **Building layout misread initially.** Assumed a vertical multi-story layout, then corrected to a linear `[5/6 wings | dining room | + main building]` layout. Also corrected assumption that rooms 342 and 343 are across the hall from each other — a room conversion to a double room changed the physical layout, so they are not directly opposite. + +--- + +## Configuration Changes + +None. Read-only diagnostic session. + +--- + +## Credentials & Secrets + +- **UniFi Cloud API Key:** `amY54KqX0i0OuGEYNykLdH9M1Kd4jhzt` + - Scope: all sites under ACG's ui.com account + - Access level: admin (read + write, though only read was used) + - Used with header: `X-API-Key: ` against `https://api.ui.com` + - Does NOT work for local controller auth at 172.16.3.29 + +--- + +## Infrastructure & Servers + +### Cascades UniFi Site +- **Site name (display):** Cascades +- **Site ID:** `685f39068e65331c46ef6dd2` +- **Site short name:** `va6iba3v` +- **Host ID:** `2d6b654d-9b79-4eaa-b2e1-52062a5690ef` +- **Host name:** UOS Server +- **UOS Server IP:** 172.16.3.29 +- **UOS Server ports:** 8080 (HTTP/Tomcat), 8443 (HTTPS/proxy — 404 all paths), 443 (unreachable) +- **API base:** `https://api.ui.com` (cloud, works with key above) + +### Network topology +- **Total devices:** 87 (75 wifi, 12 wired) +- **Active wifi clients:** 549 +- **Site-wide TX retry rate:** 11.2% (elevated — healthy is <5%) +- **Internet uplink:** no gateway device shown in site stats (no WAN magic / no ISP data) +- **Floor switches:** USW Pro 24 PoE per floor via SFP+ (10 GbE) — 2nd floor, 3rd floor, 4th floor confirmed online; Switch MemCare online +- **AP model:** Predominantly U7 Pro (64 units); 1 U6 Pro (room 615); 1 AC Mesh Pro (Patio/Kitchen); CC Bridge (U7 Pro, wireless mesh) + +### Building layout +``` +[5th floor - N ground wing] + | + [Dining Room]----[+ main building, floors 1-4] + | +[6th floor - S ground wing] +``` +- Floors 1–4: multi-story "+" shaped building, 4 wings per floor +- Floors 5–6: single-story ground-level wings, connected to main building via Dining Room +- Hallway convention: **odd rooms on one side, even rooms on the other** (applies to all wings) +- Room 342 and 343 are NOT directly across from each other — a room was converted to a double room, changing the local layout + +--- + +## AP Inventory by Floor + +### Floor 1 (13 APs) +102, 103, 109, 115, 116, 122, 127, 132 (Rec Room), 133, 139, 140, 145, 146 + +### Floor 2 (15 APs + 2nd Floor Atrium) +203, 204, 206, 209, 210, 217, 221, 222, 229, 236, 237, 240, 241, 247, 248, 2nd Floor Atrium + +### Floor 3 (14 APs + 3rd Floor Atrium) +303, 304, 309, 310, 317, 318, 323, 324, 327, 330, 336, 341, 345, 348, 3rd Floor Atrium + +### Floor 4 (12 APs + 4th Floor Atrium) +402, 403, 407, 415, 421, 427, 428, 433, 434, 441, 442, 445, 4th Floor Atrium + +### Floor 5 — North ground wing (2 APs) +505, 517 + +### Floor 6 — South ground wing (3 APs) +608, 615 (U6 Pro), 622 + +### Common/shared areas +Dining Room, Kitchen, CC Bridge (mesh), Memcare Nurse Station, memcare piano, Memcare TV room + +--- + +## Offline Devices + +| Device | Type | IP | Notes | +|--------|------|----|-------| +| AP 335 | U7 Pro | 192.168.2.206 | No clients, offline | +| AP 342 | U7 Pro | 192.168.2.98 | No clients, offline. Not directly across from room 343 (double-room conversion) | +| AP 406 | U7 Pro | 192.168.2.4 | No clients, offline | +| AP 450 | U7 Pro | 192.168.6.207 | No clients, offline | +| Switch 2nd Floor #2 | USW 24 PoE | 192.168.2.193 | Offline | +| Switch 4th Floor #2 | USW 24 PoE | 192.168.3.65 | Offline | +| USW Pro Max 16 PoE | Switch | 192.168.1.20 | Adopted 2026-04-22, current status unclear/offline | + +--- + +## 2.4 GHz Saturation Findings + +All three non-overlapping 2.4 GHz channels are critically congested across the entire building. + +| AP | Ch | Clients | RSSI | Busy% | Interf% | Data | +|----|----|---------|------|-------|---------|------| +| 3rd Floor Atrium | 1 | 2 | -54 dBm | 76.0% | 91.5% | 420 MB | +| 2nd Floor Atrium | 6 | 2 | -53 dBm | 79.9% | 92.9% | 115 MB | +| 217 | 6 | 2 | -46 dBm | 71.1% | 85.8% | 1.07 GB | +| 309 | 6 | 3 | -43 dBm | 68.2% | 83.1% | 396 MB | +| 140 | 6 | 1 | -43 dBm | 73.5% | 86.7% | 135 MB | +| 323 | 6 | 3 | -44 dBm | 68.9% | 83.6% | 1.53 GB | +| 133 | 6 | 3 | -60 dBm | 72.2% | 83.8% | 124 MB | +| 324 | 6 | 1 | -52 dBm | 63.0% | 80.4% | 18.3 GB | +| 145 | 6 | 1 | -46 dBm | 69.4% | 81.0% | 13.0 MB | +| 204 | 11 | 2 | -47 dBm | 67.4% | 80.1% | 40.6 MB | +| 415 | 11 | 2 | -48 dBm | 69.1% | 80.4% | 20.1 MB | +| 441 | 11 | 0 | -51 dBm | 70.5% | 84.0% | 1.36 MB | +| 608 | 11 | 2 | -68 dBm | 46.4% | 61.3% | 103 MB | +| Memcare NS | 11 | 4 | -52 dBm | 33.2% | 45.7% | 592 MB | +| 427 | 1 | 1 | -47 dBm | 66.8% | 83.3% | 192 MB | +| 247 | 1 | 1 | -60 dBm | 62.9% | 75.6% | 26.8 MB | +| 336 | 1 | 0 | -64 dBm | 58.4% | 72.1% | 25.4 MB | +| 139 | 1 | 0 | -72 dBm | 70.9% | 84.5% | 305 KB | +| 505 | 1 | 2 | -62 dBm | 32.2% | 49.6% | 7.54 GB | + +Note: data covers ~25 APs (Channel AI view, partial). Same pattern expected across all 65+ APs. + +### 5 GHz status (healthy by comparison) +Most APs: 4–17% channel utilization on 5 GHz. No critical congestion found. The site-wide 11.2% TX retry rate is primarily driven by 2.4 GHz. + +--- + +## Room 343 Specific Findings + +- **No dedicated AP for room 343** — room was never assigned its own AP +- **Nearest APs:** 341 (online, ch6 2.4 GHz / ch36 5 GHz) and 345 (online, ch11 2.4 GHz / ch64 5 GHz) +- **Device in room 343:** `BRW4845E652A420` (likely Brother printer/device) + - Connected to: AP 341 + - Standard: WiFi 4 (802.11n), 1x1 spatial stream + - Band: 2.4 GHz, channel 6, 20 MHz width + - IP: 10.3.43.14 + - Experience (UniFi): Excellent + - Actual throughput: 432 bps (effectively zero — 2.4 GHz saturation) + - Uptime: 21d 11h (stable connection, not a roaming/dropout issue) +- **Root cause:** Device is hardware-limited to 2.4 GHz and is trapped on channel 6 which runs at 83%+ interference. Cannot benefit from 5 GHz. Fix requires either wiring the device or accepting degraded wifi performance for 2.4 GHz-only legacy hardware. + +--- + +## Other Infrastructure Issues + +- **AP 236:** Fast Ethernet uplink (100 Mbps cap) — all clients capped regardless of wifi speed +- **AP 240:** Fast Ethernet uplink (100 Mbps cap) — same issue +- **CC Bridge:** Operating as wireless mesh (not wired uplink). Everything downstream of this AP shares its wireless backhaul bandwidth. +- **AP 121:** Shows in Channel AI with all channels set to Auto and 0 clients/no data — likely unadopted or orphaned. + +--- + +## 2.4 GHz Power Reduction Recommendations (Pending Implementation) + +### Disable 2.4 GHz radio entirely +| AP | Ch | Rationale | +|----|----|-----------| +| 139 | 1 | 0 clients, 305 KB data, -72 dBm (barely transmitting, still causing noise) | +| 336 | 1 | 0 clients, 25.4 MB, adjacent to 341 which handles that side | +| 441 | 11 | 0 clients, 1.36 MB, adding 84% interference to ch11 for nothing | + +### Reduce TX power to Low +| AP | Ch | Rationale | +|----|----|-----------| +| 140 | 6 | Tied for strongest on ch6 (-43 dBm), only 1 client / 135 MB | +| 145 | 6 | -46 dBm, 1 client, 13 MB — very low use | +| 427 | 1 | -47 dBm, 1 client, 192 MB — shrink footprint | + +### Reduce TX power to Medium +| AP | Ch | Rationale | +|----|----|-----------| +| 415 | 11 | -48 dBm, 2 clients, 20 MB | +| 204 | 11 | -47 dBm, 2 clients, 40 MB | + +### Do not touch (active 2.4 GHz use or isolated coverage) +324 (18.3 GB heavy user), 323 (1.53 GB / 3 clients), 217 (1.07 GB), 309 (3 clients, strongest on ch6 — reduce only after others are done), Memcare NS (4 clients, isolated wing, only 33% busy), 505 (isolated 5th floor wing), 608 (isolated 6th floor), 341 (serves BRW legacy device in room 343) + +### Next step after changes +Wait 24–48 hours, re-pull Channel AI. If ch6 is still above 50% busy, reduce 309 next. + +--- + +## Commands & Outputs + +```bash +# List all sites (API key auth) +curl -s -H "X-API-Key: amY54KqX0i0OuGEYNykLdH9M1Kd4jhzt" "https://api.ui.com/ea/sites" + +# Cascades site statistics +# siteId: 685f39068e65331c46ef6dd2 — returned in /ea/sites response + +# Get all devices (returns by host, not filtered by site despite siteId param) +curl -s -H "X-API-Key: amY54KqX0i0OuGEYNykLdH9M1Kd4jhzt" \ + "https://api.ui.com/v1/devices?siteId=685f39068e65331c46ef6dd2" +# Filter for UOS Server host: 2d6b654d-9b79-4eaa-b2e1-52062a5690ef + +# UOS Server local Tomcat (reachable via Tailscale) +curl -s "http://172.16.3.29:8080/status" +# Returns: {"meta":{"rc":"ok","uuid":"ed829b7c-8e32-5f8c-a2b1-a61a23712320"},"data":[]} +# Classic API paths return HTTP 400 without session auth +``` + +--- + +## Pending / Incomplete Tasks + +- [ ] Implement 2.4 GHz power changes (disable: 139, 336, 441 / low: 140, 145, 427 / medium: 415, 204) +- [ ] Apply same Channel AI review to remaining ~50 APs not covered in shared data +- [ ] Investigate offline APs 335, 342, 406, 450 — physical inspection or PoE switch port check +- [ ] Investigate offline Switch 2nd Floor #2 and Switch 4th Floor #2 +- [ ] Fix Fast Ethernet uplinks on APs 236 and 240 — check cable/switch port +- [ ] Resolve CC Bridge wireless mesh — determine if wired backhaul is possible +- [ ] Re-pull Channel AI 24–48 hours after 2.4 GHz changes to measure improvement +- [ ] Evaluate AP 121 — unadopted or orphaned device, investigate +- [ ] Consider minimum RSSI threshold on 2.4 GHz to force legacy device proximity +- [ ] Consider alternating 2.4 GHz disable pattern across room APs as a longer-term strategy +- [ ] Identify and possibly replace the BRW device in room 343 with a wired connection + +--- + +## Reference Information + +- **UniFi Cloud API base:** `https://api.ui.com` +- **Working endpoints:** `/ea/sites`, `/v1/sites`, `/v1/devices?siteId=` +- **Non-working via cloud:** `/ea/sites//devices`, `/proxy/network//...`, `/v1/clients` +- **Developer docs:** https://developer.ui.com/site-manager-api/list-devices (JS-rendered, not WebFetch accessible) +- **UniFi getting started:** https://developer.ui.com/site-manager/v1.0.0/gettingstarted +- **Prior Cascades session logs:** `clients/cascades-tucson/session-logs/` +- **Cascades CONTEXT.md:** `clients/cascades-tucson/CONTEXT.md`