azcomputerguru/claudetools
1
0
Fork 0
Code Issues 59 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
dafcec5bcec2c3329ff451fb761820249e3ddddc
claudetools/clients/darrell-delphen/session-logs/2026-06/2026-06-18-mike-email-link-sni-block.md
Mike Swanson 6f4cadb16f sync: auto-sync from GURU-5070 at 2026-06-18 12:49:38 
Author: Mike Swanson
Machine: GURU-5070
Timestamp: 2026-06-18 12:49:38
2026-06-18 12:51:08 -07:00

7.6 KiB
Raw Blame History

2026-06-18 — Darrell Delphen — Outlook email links failing (ISP SNI block)

User

  • User: Mike Swanson (mike)
  • Machine: GURU-5070
  • Role: admin

Session Summary

Darrell Delphen reported that links in Outlook email would not open on one workstation (DDDOffice072023), failing with "can't reach this page" / ERR_CONNECTION_ABORTED against a url.emailprotection.link URL, while links opened from Gmail worked fine. The failing host is Intermedia's Email Protection "Safe Link" rewriter — every link in Intermedia-protected mail is rewritten to https://url.emailprotection.link/..., so all Outlook-delivered links routed through it.

Diagnosis was done entirely over GuruRMM against agent 000ed57d-fd05-4001-871c-244f43155c16. DNS resolved correctly and TCP 443 connected, but the TLS handshake died with SChannel 0x80090326 (SEC_E_ILLEGAL_MESSAGE — "message received was unexpected or badly formatted"). The endpoint's TLS stack was clean: FIPS off, no SCHANNEL protocol/cipher overrides, no cipher-suite GPO, only Windows Defender (no third-party AV/proxy/VPN/LSP/WFP callout). The same node 199.193.205.140 handshook successfully with the real SNI from GURU-5070 on a different network, proving the origin was healthy and the interference was on the endpoint's path. A blast-radius sweep showed only url.emailprotection.link failed while google.com, microsoft.com, outlook.office365.com, cloudflare.com, badssl.com, and even login.serverdata.net (same Intermedia infra) all succeeded; MTU was fine. An SNI-varied handshake to the same IP isolated it conclusively: example.com and login.serverdata.net SNIs succeeded 12/12 while url.emailprotection.link failed 12/12, across interleaved source ports — deterministic, SNI-keyed, not flow-hash/LAG. Root cause: a network device on the path performing SNI-based content inspection that corrupted the handshake for that one hostname.

The gateway turned out to be an ISP-provided Extreme EXOS device the client had no login for. The fix path was therefore ISP escalation. An escalation packet was drafted, and Cloudflare WARP was installed on the workstation as an interim workaround — tunneling past the SNI block. With WARP connected, egress moved to Cloudflare (104.28.152.216) and the real-SNI handshake succeeded (TLS 1.2, HTTP 200). The ISP then disabled a "NetIQ" web/URL-filtering feature on the gateway, which cleared the block at the source. After WARP was disconnected the native ISP path was verified working (5/5 handshakes + HTTP 200, egress back to 167.89.210.225), so WARP was uninstalled and the machine returned to normal. Work was documented and billed on Syncro ticket #32437 — private technical note, customer-facing/emailed summary, and 1.0h remote labor ($150.00, invoice #1650728058).

Key Decisions

  • Diagnosed exclusively via repeated GuruRMM SslStream/Test-NetConnection probes rather than asking the client to run tools — faster and reproducible.
  • Used an SNI-varied handshake to the same fixed IP as the decisive test. It separated destination/server problems from path interference and proved the block was keyed on the SNI string.
  • Ran a 12x repeatability test per SNI to rule out a faulty LAG/ECMP member (intermittent, 5-tuple keyed) vs deliberate content matching (deterministic). Result was 0/12 vs 12/12 — deterministic.
  • Chose Cloudflare WARP as the interim workaround because the block is SNI-based; any tunnel that encrypts egress past the EXOS hides the SNI. WARP is the lightest deploy.
  • Installed/connected WARP in stages (install, then connect, then verify) so each step could be confirmed before flipping the tunnel, given the agent bounces on network-stack changes.
  • Emailed the customer a plain-language summary (do_not_email:false) and kept the technical detail in a hidden note.

Problems Encountered

  • Shell state not persisting between Bash calls — $TOKEN/$RMM from rmm-auth.sh were gone on the next call (first dispatch produced no output). Fixed by eval "$(rmm-auth.sh)" inside every Bash invocation.
  • PowerShell single-quotes collided with bash single-quoted SCRIPT='...' — embedded 'C:\Program Files\...' terminated the bash string (FilesCloudflareCloudflare: command not found). Fixed by defining the script via a quoted heredoc SCRIPT=$(cat <<'PS' ... PS).
  • WARP install/connect bounced the RMM agent — commands returned interrupted ("Agent restarted during execution") because installing/connecting WARP resets the network stack/WFP. The agent auto-reconnected (through WARP after connect); verified state with follow-up commands.
  • First WARP uninstall found nothing — the product registers as "Cloudflare One Client", not "Cloudflare WARP", so the DisplayName filter missed it. Found the GUID {9E49837E-2971-413F-9587-119FA819E572} and removed via msiexec /x.

Configuration Changes

  • Endpoint DDDOffice072023: Cloudflare WARP (Cloudflare One Client 2026.4.1390.0) installed, registered, connected, then later disconnected and fully uninstalled. Net change to the machine: none.
  • ISP gateway (not us): ISP disabled the "NetIQ" web/URL-filtering feature on the Extreme EXOS device.
  • No changes to the ClaudeTools repo code.

Credentials & Secrets

None discovered, created, or rotated this session.

Infrastructure & Servers

  • Endpoint: DDDOffice072023 — Windows, GuruRMM agent 000ed57d-fd05-4001-871c-244f43155c16 (v0.6.66), RMM client "AZ Computer Guru" / site "Discovery test site". LAN gateway 192.168.1.1, ISP egress 167.89.210.225, WARP egress (while active) 104.28.152.216.
  • ISP gateway: Extreme Networks EXOS device, ISP-provided/managed (client has no login). Was running a "NetIQ" URL-filtering feature doing SNI inspection.
  • Blocked destination: url.emailprotection.link → CNAME urlrs.gslb.serverdata.net → A 199.193.205.140 (Intermedia Email Protection link-rewriter). GSLB pool also advertises 199.193.200.65 / 64.78.20.65 / 162.244.196.65, all TCP-dead from any network (not an ISP block).
  • GuruRMM API: http://172.16.3.30:3001 (JWT via rmm-auth.sh).

Commands & Outputs

  • Decisive SNI test (same IP, varied SNI), via RMM PowerShell:
    • SNI url.emailprotection.link → 0x80090326 SEC_E_ILLEGAL_MESSAGE (0/12 ok)
    • SNI example.com / login.serverdata.net → TLS 1.2 AES256 (12/12 ok)
  • Off-network control (GURU-5070) to 199.193.205.140 with real SNI → OK TLS 1.2.
  • Post-ISP-fix native verify: real-SNI handshake 5/5 OK + Invoke-WebRequest HEAD → HTTP 200, egress 167.89.210.225.
  • WARP removal: msiexec /x {9E49837E-2971-413F-9587-119FA819E572} /qn /norestart → exit 0; warp-svc/warp-cli/install-dir/uninstall-entry all gone.

Pending / Incomplete Tasks

  • None functional. Block is resolved at the ISP. If the issue recurs, suspect the EXOS "NetIQ" feature being re-enabled — re-run the SNI-varied handshake test to confirm.
  • Optional: if WARP is ever rolled to more machines as a workaround, harden it (force auto-connect, lock client) and note egress moves to Cloudflare (breaks office-static-IP allowlisting).

Reference Information

  • Syncro ticket: #32437 (id 112766479) — https://computerguru.syncromsp.com/tickets/112766479
    • Private note id 419714810; public/emailed summary id 419714813
    • Line item id 42925426 (Labor - Remote Business 1.0h @ $150)
    • Invoice #1650728058, total $150.00; status Invoiced
  • Customer: Darrell Delphen (Syncro customer_id 35996725), no prepaid block.
  • SChannel error: 0x80090326 = SEC_E_ILLEGAL_MESSAGE (handshake message malformed/unexpected) — signature of in-path TLS/SNI tampering when paired with same-IP success off-network.
Reference in New Issue View Git Blame Copy Permalink