From f962cb87d028933e09e35fec71bef983ab02e0b6 Mon Sep 17 00:00:00 2001 From: Mike Swanson Date: Tue, 26 May 2026 07:05:10 -0700 Subject: [PATCH] sync: auto-sync from GURU-5070 at 2026-05-26 07:05:06 Author: Mike Swanson Machine: GURU-5070 Timestamp: 2026-05-26 07:05:06 --- session-logs/2026-05-26-session.md | 110 ++++++++++++++++++++++++ wiki/clients/internal-infrastructure.md | 6 +- wiki/clients/western-tire.md | 6 +- 3 files changed, 116 insertions(+), 6 deletions(-) diff --git a/session-logs/2026-05-26-session.md b/session-logs/2026-05-26-session.md index 23b7831..ffeb149 100644 --- a/session-logs/2026-05-26-session.md +++ b/session-logs/2026-05-26-session.md @@ -228,3 +228,113 @@ StandardErrorPath: /usr/local/var/log/gururmm-agent.log - Phase 3 (menu bar app): 4-5 days - Phase 4 (integration/testing): 2-3 days - Total: 10-13 days for complete system + +--- + +## Update: 06:20 MST — SSL cert triage (Mike Swanson / GURU-5070) + +### User +- **User:** Mike Swanson (mike) +- **Machine:** GURU-5070 (DESKTOP-0O8A1RL) +- **Role:** admin +- **Session span:** 2026-05-26 ~06:20 MST + +### Session Summary + +Session opened with `/context` to recover prior state, followed by `/sync` which pulled the MacBook session log (macOS agent enrollment fix). Two SSL cert items from the pending list were addressed. + +Neptune SSL cert (mail.acghosting.com, expiring 2026-05-31) was flagged as having auto-renewal configured by the user. The wiki was updated to remove the URGENT flag and note that auto-renewal is in place. + +Western Tire SSL was investigated in depth. The `*.westerntire.com` wildcard cert (Let's Encrypt, issued 2026-03-01) was expiring 2026-05-30 and had not renewed. Direct SSL inspection via `openssl s_client` confirmed the cert was the original March 1 issue — no renewal had occurred. SSH to IX (172.16.3.10) via paramiko revealed the root cause: all 9 westerntire.com domains were listed as user-excluded in cPanel's AutoSSL, so AutoSSL had been skipping the account entirely on every 3-hour run. No acme.sh installation and no cron job for cert renewal existed. + +Per the user's direction, only `mail.westerntire.com` was removed from the AutoSSL exclusion list (the 8 other domains — www, webmail, cpanel, autodiscover, webdisk, cpcontacts, cpcalendars, westerntire.com — remain excluded as they carry no live services). An AutoSSL run was triggered via `whmapi1 start_autossl_check_for_one_user`. AutoSSL completed within 30 seconds: HTTP DCV passed, Let's Encrypt issued a cert for `mail.westerntire.com` only, and it was installed immediately. Live verification via `openssl s_client` on port 993 confirmed the new cert (CN=mail.westerntire.com, R13, issued 2026-05-26, expires 2026-08-24). Wiki articles for western-tire and internal-infrastructure were updated to reflect the resolved state. + +### Key Decisions + +- **Only remove mail.westerntire.com from AutoSSL exclusions** — user specified mail sub only. The other 8 domains serve no live purpose (westerntire.com redirects to jackfurriers.com; cPanel/webmail/autodiscover subdomains unused by active users). Retaining their exclusion avoids AutoSSL issuing unnecessary certs for unused names. +- **Let AutoSSL issue per-domain cert, not wildcard** — AutoSSL (HTTP-01 challenge) cannot issue wildcards. Switching from the wildcard to a per-domain cert for mail.westerntire.com is functionally equivalent for IMAP/SMTP clients. +- **Neptune SSL flagged as no-action** — user confirmed auto-renewal is configured. Removed from urgent list without verifying the renewal client directly; if renewal fails by 2026-05-31 it will surface as a mail TLS error. + +### Problems Encountered + +- **`uapi remove_autossl_excluded_domains domain=...` failed** — cPanel UAPI expects the argument as `domains=` (plural), not `domain=`. Fixed by correcting the argument name; API returned status 1 on retry. +- **sshpass not available on Windows** — initial SSH attempt using `sshpass -p` failed with command not found. Switched to paramiko (Python) for all SSH operations to IX. + +### Configuration Changes + +**IX server (172.16.3.10) — cPanel westernt account:** +- Removed `mail.westerntire.com` from AutoSSL excluded domains list +- New cert issued and installed: CN=mail.westerntire.com, issued 2026-05-26, expires 2026-08-24 + +**Wiki (claudetools repo):** +- `wiki/clients/western-tire.md` — SSL section updated: wildcard replaced by per-domain AutoSSL cert for mail subdomain; P2 SSL item removed from open items; anti-pattern warning updated +- `wiki/clients/internal-infrastructure.md` — Neptune Let's Encrypt cert entry updated: removed URGENT flag, noted auto-renewal configured + +### Credentials & Secrets + +No new credentials. Used existing: +- IX root password: `infrastructure/ix-server.sops.yaml` → `credentials.password` + +### Infrastructure & Servers + +| Host | Detail | +|---|---| +| IX web server | 172.16.3.10 (internal) / 72.194.62.5 (external). cPanel account `westernt`. AutoSSL runs every 3 hours via Let's Encrypt provider. | +| mail.westerntire.com | Port 993 (IMAPS), port 587 (SMTP STARTTLS). New cert CN=mail.westerntire.com, expires 2026-08-24. | + +### Commands & Outputs + +```bash +# Verify existing cert before fix +echo | openssl s_client -connect mail.westerntire.com:993 -servername mail.westerntire.com 2>/dev/null \ + | openssl x509 -noout -subject -dates +# subject=CN=*.westerntire.com +# notBefore=Mar 1 10:05:18 2026 GMT notAfter=May 30 10:05:17 2026 GMT + +# Check AutoSSL log for westerntire (most recent run) +# -> "User-excluded domains: 9 (westerntire.com, www.westerntire.com, mail.westerntire.com, ...)" + +# Get current exclusion list +uapi --user=westernt SSL get_autossl_excluded_domains +# -> 9 domains listed including mail.westerntire.com + +# Remove mail.westerntire.com from exclusions +uapi --user=westernt SSL remove_autossl_excluded_domains domains=mail.westerntire.com +# -> status: 1 (success) + +# Trigger AutoSSL run +/usr/local/cpanel/bin/whmapi1 start_autossl_check_for_one_user username=westernt +# -> pid: 3715934, result: 1 + +# AutoSSL log after run +# [2026-05-26T14:02:21Z] "mail.westerntire.com" is managed. +# [2026-05-26T14:02:22Z] CA authorized: "mail.westerntire.com" +# [2026-05-26T14:02:23Z] Local HTTP DCV OK: mail.westerntire.com +# [2026-05-26T14:02:25Z] "Let's Encrypt™" HTTP DCV OK: mail.westerntire.com +# [2026-05-26T14:02:27Z] Installing "westerntire.com"'s new certificate ... + +# Verify new cert +echo | openssl s_client -connect mail.westerntire.com:993 -servername mail.westerntire.com 2>/dev/null \ + | openssl x509 -noout -subject -issuer -dates +# subject=CN=mail.westerntire.com +# issuer=C=US, O=Let's Encrypt, CN=R13 +# notBefore=May 26 13:03:56 2026 GMT notAfter=Aug 24 13:03:55 2026 GMT +``` + +### Pending / Incomplete Tasks + +- HIGH: `fix/audit-2-remediation` branch (gururmm) — awaiting Mike's merge + deploy +- HIGH: macOS PKG installer — plan at `projects/msp-tools/guru-rmm/agent/MACOS_INSTALLATION_PLAN.md` +- HIGH: Kittle WS2025 EVAL license activation; no backup, no firewall +- HIGH: Kittle-Design Ken inbox rule (potential active compromise) +- MEDIUM: TGC-SERVER Hyper-V disposition (MAS90 VM — customer decision needed) +- MEDIUM: Neptune SSL auto-renewal — verify client is healthy before 2026-05-31 (user says it's configured) +- LOW: Seed wiki/systems/neptune.md, wiki/systems/beast.md + +### Reference Information + +- Western Tire Syncro ticket: #32199 (ID: 109325058) — not yet billed +- IX AutoSSL log path: `/var/cpanel/logs/autossl//txt` +- IX AutoSSL schedule: every 3 hours (confirmed from log dir timestamps) +- cPanel UAPI: `uapi --user= SSL remove_autossl_excluded_domains domains=` (note: `domains=`, not `domain=`) +- New cert details: CN=mail.westerntire.com, Let's Encrypt R13, 2026-05-26 → 2026-08-24, auto-renews ~2026-07-25 diff --git a/wiki/clients/internal-infrastructure.md b/wiki/clients/internal-infrastructure.md index 1166640..222d593 100644 --- a/wiki/clients/internal-infrastructure.md +++ b/wiki/clients/internal-infrastructure.md @@ -115,7 +115,7 @@ Neptune is ACG's on-premises Exchange Server 2016, hosting mail for multiple cli - **AD Domain:** acg.local - **DNS Server (primary):** ACG-DC16 at 172.16.3.52 (also .50) - **Mailboxes:** 56 total (N-Hosting1 DB: 809 GB / 54 boxes; N-LargeBoxes DB: 313 GB / 2 boxes) -- **Let's Encrypt cert:** CN=mail.acghosting.com, expires 2026-05-31 [WARNING] — renewal needed +- **Let's Encrypt cert:** CN=mail.acghosting.com, expires 2026-05-31 — auto-renewal configured (win-acme or equivalent); no manual action needed unless renewal client reports failure - **Internal transport cert:** Thumbprint `E58BFCBAEFEFDCAED0BF9E894127A3DE64CE9C69`, expires 2026-07-22 [WARNING] - **Access:** Local PowerShell with Exchange Management Shell snapin (`Add-PSSnapin Microsoft.Exchange.Management.PowerShell.SnapIn`); must run as administrator.ACG on the box or via domain-admin WinRM - **Vault:** `infrastructure/neptune-exchange.sops.yaml` [unverified — check vault for current entry name] @@ -228,7 +228,7 @@ Neptune is Exchange 2016 running on an unsupported OS (WS2022 after the 2026-04- **Pending transport cert renewal:** Thumbprint `E58BFCBAEFEFDCAED0BF9E894127A3DE64CE9C69` expires 2026-07-22. -**Pending Neptune Let's Encrypt renewal:** CN=mail.acghosting.com cert expires 2026-05-31 — URGENT. +**Neptune Let's Encrypt renewal:** CN=mail.acghosting.com cert expires 2026-05-31 — auto-renewal configured; monitor for renewal client errors only. **Incomplete domain MX fixes from 2026-03-17** (still unresolved as of last session): - `airandspaceacademy.com`: DNS on GoDaddy still points MX to mail.acghosting.com (direct, no filter) — being rejected by the transport inbound restriction rule. Needs changing to Mailprotector inbound. @@ -259,7 +259,7 @@ The Claude Code hooks (user-prompt-submit, task-complete) spawn background `sync As of last session (2026-04-23): - **Neptune Exchange migration** — Build Exchange 2019 on fresh WS2022 VM. Runbook at `C:\NeptuneConfigExport-20260423\MIGRATION-RUNBOOK.md` on Neptune. Mike building the VM. Critical gate: **back up ACG-DC16 before running `/PrepareSchema`** (forest-permanent, no rollback). -- **Neptune Let's Encrypt cert** — expires 2026-05-31. Renewal critical. +- **Neptune Let's Encrypt cert** — expires 2026-05-31; auto-renewal configured. No manual action unless renewal client fails. - **Neptune internal transport cert** — expires 2026-07-22. - **DkimSigner re-enable / replace** — outbound mail currently unsigned. Evaluate whether Exchange DkimSigner is runtime-compatible post-KB5084071, or replace with alternative. - **MAIL server AD decommission** — once Exchange 2019 is live and mailboxes moved: `Remove-ADObject -Recursive` on the MAIL carcass. After that, remove hosts file entries for MAIL/mail.acg.local and DC-side DNS records (n-hosting1, n-largeboxes, mail can remain or be repurposed for the new server). diff --git a/wiki/clients/western-tire.md b/wiki/clients/western-tire.md index 29aceec..0a8e4c6 100644 --- a/wiki/clients/western-tire.md +++ b/wiki/clients/western-tire.md @@ -49,7 +49,7 @@ System/automated (not notified): `donotreply, storealert, integrilogic, receipts - **Home dir:** 62 GB - **Mailboxes:** 30 accounts under westerntire.com - **MySQL:** None (account does not use MySQL) -- **SSL:** Wildcard `*.westerntire.com` from Let's Encrypt, valid to 2026-05-30 (AutoSSL should renew) +- **SSL:** `mail.westerntire.com` managed by AutoSSL (renewed 2026-05-26, expires 2026-08-24). Wildcard `*.westerntire.com` expired 2026-05-30 — other subdomains (www, webmail, cpanel, autodiscover, webdisk, cpcontacts, cpcalendars, westerntire.com) remain AutoSSL-excluded; all excluded domains redirect or are unused. Only mail.westerntire.com matters for live service. ### Key file paths on IX @@ -109,7 +109,7 @@ System/automated (not notified): `donotreply, storealert, integrilogic, receipts |---|---|---| | P1 | Monitor for user mail client issues after email setup guide was sent (new IMAP/SMTP settings) | Mike | | P2 | Bill ticket #32199 when scope is confirmed | Mike | -| P2 | westerntire.com SSL cert (`*.westerntire.com`) expires 2026-05-30 — verify AutoSSL renewed | Mike | +| P2 | Bill ticket #32199 when scope is confirmed | Mike | | P3 | Update Syncro customer property "DNS Detail" field — currently says "Email is on Websvr" (now IX) | Mike | ### User mail client settings (from setup guide sent 2026-04-22) @@ -152,7 +152,7 @@ A duplicate ticket #32198 was inadvertently created and deleted. ## Anti-Patterns / Warnings - [WARNING] Plaintext SSH credentials for websvr and IX appeared in session log. Always retrieve from vault — never hardcode. -- [WARNING] SSL cert `*.westerntire.com` expires 2026-05-30 — check AutoSSL renewal immediately if it's past that date. +- SSL: only `mail.westerntire.com` is AutoSSL-managed (renewed 2026-05-26). Other subdomains are excluded; the wildcard expired 2026-05-30 but none of those domains carry live services. - Do NOT use `${sg{}{\\\.}{-}}` in exim.conf.local on WHM servers — WHM buildeximconf strips backslash levels and breaks the regex. Use `${tr{}{.}{-}}` instead. - Do NOT use tainted `$sender_address_domain` directly in file path lookups in exim 4.94+ — use `dsearch` (returns untainted value) for DKIM private key paths. - Do NOT look for westerntire.com mail on websvr — migration is complete; mail lives on IX.