**1. Updated ranked hypotheses** The new evidence (commands provably leave Rust, leave origin nginx .30, and are sent toward NPM .20) eliminates everything before NPM. The fact that ~40 other agents on the identical NPM path continue to receive commands normally is the strongest signal. - **NPM buffering/relay defect** (per-connection or per-stream): Still plausible but now narrowed. Global NPM breakage is refuted. A defect that only affects a subset of long-lived WS connections (e.g., specific idle-timeout / buffer / upgrade handling / connection-tracking state inside the nginx instance inside the Docker container, or only after certain connection age or header combinations from particular agents) remains possible. The 13x vs 8x marker count hint of retransmit/re-offer on the internal leg is consistent with the proxy sometimes being slow or lossy on the client leg for those particular sockets. - **NPM->WAN/UDR black-hole (H3)**: Remains high. Different affected sites (PST UDR + GTS-PEDRO-H) make a single UDR bug less likely, but per-agent-path problems are common: UniFi stateful inspection / NAT table quirks on inbound WS data frames (even when outbound heartbeats keep the flow alive), silent drops after the TCP connection is established, MTU/PMTUD issues on the return path for some ISPs, or middlebox behavior that treats small outbound heartbeats differently from inbound application data. Tiny 80 B frames failing is compatible if the drop is flow- or direction-based rather than size-based. - **Agent-side defect (H2)**: Now the strongest single hypothesis for the subset pattern. The WS connection is clearly alive from the agent's viewpoint (it emits heartbeats, does not hit the 90 s no-inbound reconnect timer). This is exactly what you expect if the agent process is still writing on the socket and the TCP stack is ACKing, but the receive side of the agent either (a) is not reading the socket for "command" frames, (b) has a dispatcher that drops or mishandles "type":"command" messages while still handling pings/heartbeats, or (c) is under local resource pressure / wedged thread only on those specific hosts. Same agent version + only a subset affected fits an agent-local trigger (particular host load, Windows firewall / security product interaction with the socket, specific network interface, etc.). The two PST servers being at the same site is consistent with them sharing the same local conditions. "Subset on the same NPM" therefore pushes weight toward either per-connection state inside NPM or (more likely) causes that are downstream of NPM and specific to the individual agent or its access path. It does not favor a broad NPM configuration or capacity problem. **2. Single most decisive next test** Targeted packet capture on the NPM box (.20) itself, focused on the established TCP 4-tuples for the affected agents (source ports on 98.190.129.150 etc.), while sending additional uniquely marked commands. Capture at the host level (or the relevant Docker bridge) so you see what actually leaves the NPM container toward the public internet. Also capture the corresponding inbound heartbeats/ACKs on the same flows. You already have the marking technique that makes correlation trivial. This is decisive; the three clean outcomes map directly to the three remaining locations: - No TCP data / TLS application_data packets emitted from .20 toward the agent IP shortly after the frame is seen arriving at NPM on the internal leg → NPM received the WS message but did not forward it on the client connection. → NPM buffering/relay defect (or its per-connection state for that socket). - Packets are emitted promptly from .20, normal TCP ACKs come back from the agent IP (often within one RTT), but the agent still never ACKs/executes the marked command → the bytes reached the agent's OS, were delivered to the TCP socket, and were ACKed. The defect is after TCP delivery inside the agent process. → Agent-side defect (H2). - Packets are emitted from .20 but you see repeated retransmissions (RTOs), growing send buffer, zero-window probes, or missing ACKs for NPM's data segments, while the agent continues to send heartbeats that NPM is ACKing → data is leaving the NPM host but is not reaching (or not being accepted by) the agent's TCP stack. → Black-hole or severe loss on the NPM→agent leg (H3 or UDR / path / host firewall). Secondary signals (ICMP unreachable, RSTs from intermediate hops, TCP timestamps, etc.) can further localize inside the path. Is the .20 capture decisive enough? Yes, for practical purposes. It directly answers "did NPM emit?" and "did the far end TCP-ACK the data?" without requiring any agent reconfiguration. It is low-risk, quick to set up, and can be done while the system is live. Better move? Only marginally: first do a quick non-capture check on .20 (NPM container logs + access/error logs for the exact vhost and the affected connection IDs around the mark timestamps; `ss -tpi` or `ss -tinfo` on the sockets to see send-q, last-ack, etc.). If logs are silent and sockets look healthy, proceed straight to the capture. Do not yet touch NPM advanced config or origin nginx unless the capture implicates the proxy. **3. Verdict on "remove NPM / NAT straight to .30"** The new evidence does make the bypass more justified as a *targeted follow-up*, because everything before NPM is now exonerated. If the capture shows NPM is not emitting the frames, the bypass becomes high-confidence: you would be removing the exact component that is failing to relay. However, the capture is still the cheaper, safer, higher-ROI step first. Reasons: - It isolates NPM itself with almost no blast radius and no production routing change. - If the capture instead shows NPM *is* emitting + retransmits or clean ACKs, the bypass is unlikely to help. The packets would still have to traverse the public internet from the same public IP (72.194.62.10) to the same agent WAN IPs; only the internal origin changes from .20 to .30. The black-hole or agent receive problem would remain. - Bypass requires edge firewall DNAT change + moving TLS termination to .30 (or reconfiguring the origin nginx) + potential SNAT/return-path adjustments. That carries risk to the ~40 working agents and is harder to roll back cleanly. Do the .20 capture (with the marked commands) first. Use its outcome to decide whether the bypass is worth the cost and risk, or whether you need to move to agent-side investigation (on-site, reinstall, debug logging in the agent, etc.) or path debugging at the affected sites' UDRs. The capture gives you the information needed to avoid an expensive test that may not be diagnostic.