sync: auto-sync from GURU-5070 at 2026-05-26 07:05:06

Author: Mike Swanson
Machine: GURU-5070
Timestamp: 2026-05-26 07:05:06

session-logs/2026-05-26-session.md

@@ -228,3 +228,113 @@ StandardErrorPath: /usr/local/var/log/gururmm-agent.log
 - Phase 3 (menu bar app): 4-5 days
 - Phase 4 (integration/testing): 2-3 days
 - Total: 10-13 days for complete system
+
+---
+
+## Update: 06:20 MST — SSL cert triage (Mike Swanson / GURU-5070)
+
+### User
+- **User:** Mike Swanson (mike)
+- **Machine:** GURU-5070 (DESKTOP-0O8A1RL)
+- **Role:** admin
+- **Session span:** 2026-05-26 ~06:20 MST
+
+### Session Summary
+
+Session opened with `/context` to recover prior state, followed by `/sync` which pulled the MacBook session log (macOS agent enrollment fix). Two SSL cert items from the pending list were addressed.
+
+Neptune SSL cert (mail.acghosting.com, expiring 2026-05-31) was flagged as having auto-renewal configured by the user. The wiki was updated to remove the URGENT flag and note that auto-renewal is in place.
+
+Western Tire SSL was investigated in depth. The `*.westerntire.com` wildcard cert (Let's Encrypt, issued 2026-03-01) was expiring 2026-05-30 and had not renewed. Direct SSL inspection via `openssl s_client` confirmed the cert was the original March 1 issue — no renewal had occurred. SSH to IX (172.16.3.10) via paramiko revealed the root cause: all 9 westerntire.com domains were listed as user-excluded in cPanel's AutoSSL, so AutoSSL had been skipping the account entirely on every 3-hour run. No acme.sh installation and no cron job for cert renewal existed.
+
+Per the user's direction, only `mail.westerntire.com` was removed from the AutoSSL exclusion list (the 8 other domains — www, webmail, cpanel, autodiscover, webdisk, cpcontacts, cpcalendars, westerntire.com — remain excluded as they carry no live services). An AutoSSL run was triggered via `whmapi1 start_autossl_check_for_one_user`. AutoSSL completed within 30 seconds: HTTP DCV passed, Let's Encrypt issued a cert for `mail.westerntire.com` only, and it was installed immediately. Live verification via `openssl s_client` on port 993 confirmed the new cert (CN=mail.westerntire.com, R13, issued 2026-05-26, expires 2026-08-24). Wiki articles for western-tire and internal-infrastructure were updated to reflect the resolved state.
+
+### Key Decisions
+
+- **Only remove mail.westerntire.com from AutoSSL exclusions** — user specified mail sub only. The other 8 domains serve no live purpose (westerntire.com redirects to jackfurriers.com; cPanel/webmail/autodiscover subdomains unused by active users). Retaining their exclusion avoids AutoSSL issuing unnecessary certs for unused names.
+- **Let AutoSSL issue per-domain cert, not wildcard** — AutoSSL (HTTP-01 challenge) cannot issue wildcards. Switching from the wildcard to a per-domain cert for mail.westerntire.com is functionally equivalent for IMAP/SMTP clients.
+- **Neptune SSL flagged as no-action** — user confirmed auto-renewal is configured. Removed from urgent list without verifying the renewal client directly; if renewal fails by 2026-05-31 it will surface as a mail TLS error.
+
+### Problems Encountered
+
+- **`uapi remove_autossl_excluded_domains domain=...` failed** — cPanel UAPI expects the argument as `domains=` (plural), not `domain=`. Fixed by correcting the argument name; API returned status 1 on retry.
+- **sshpass not available on Windows** — initial SSH attempt using `sshpass -p` failed with command not found. Switched to paramiko (Python) for all SSH operations to IX.
+
+### Configuration Changes
+
+**IX server (172.16.3.10) — cPanel westernt account:**
+- Removed `mail.westerntire.com` from AutoSSL excluded domains list
+- New cert issued and installed: CN=mail.westerntire.com, issued 2026-05-26, expires 2026-08-24
+
+**Wiki (claudetools repo):**
+- `wiki/clients/western-tire.md` — SSL section updated: wildcard replaced by per-domain AutoSSL cert for mail subdomain; P2 SSL item removed from open items; anti-pattern warning updated
+- `wiki/clients/internal-infrastructure.md` — Neptune Let's Encrypt cert entry updated: removed URGENT flag, noted auto-renewal configured
+
+### Credentials & Secrets
+
+No new credentials. Used existing:
+- IX root password: `infrastructure/ix-server.sops.yaml` → `credentials.password`
+
+### Infrastructure & Servers
+
+| Host | Detail |
+|---|---|
+| IX web server | 172.16.3.10 (internal) / 72.194.62.5 (external). cPanel account `westernt`. AutoSSL runs every 3 hours via Let's Encrypt provider. |
+| mail.westerntire.com | Port 993 (IMAPS), port 587 (SMTP STARTTLS). New cert CN=mail.westerntire.com, expires 2026-08-24. |
+
+### Commands & Outputs
+
+```bash
+# Verify existing cert before fix
+echo | openssl s_client -connect mail.westerntire.com:993 -servername mail.westerntire.com 2>/dev/null \
+  | openssl x509 -noout -subject -dates
+# subject=CN=*.westerntire.com
+# notBefore=Mar  1 10:05:18 2026 GMT  notAfter=May 30 10:05:17 2026 GMT
+
+# Check AutoSSL log for westerntire (most recent run)
+# -> "User-excluded domains: 9 (westerntire.com, www.westerntire.com, mail.westerntire.com, ...)"
+
+# Get current exclusion list
+uapi --user=westernt SSL get_autossl_excluded_domains
+# -> 9 domains listed including mail.westerntire.com
+
+# Remove mail.westerntire.com from exclusions
+uapi --user=westernt SSL remove_autossl_excluded_domains domains=mail.westerntire.com
+# -> status: 1 (success)
+
+# Trigger AutoSSL run
+/usr/local/cpanel/bin/whmapi1 start_autossl_check_for_one_user username=westernt
+# -> pid: 3715934, result: 1
+
+# AutoSSL log after run
+# [2026-05-26T14:02:21Z]  "mail.westerntire.com" is managed.
+# [2026-05-26T14:02:22Z]  CA authorized: "mail.westerntire.com"
+# [2026-05-26T14:02:23Z]  Local HTTP DCV OK: mail.westerntire.com
+# [2026-05-26T14:02:25Z]  "Let's Encrypt™" HTTP DCV OK: mail.westerntire.com
+# [2026-05-26T14:02:27Z]  Installing "westerntire.com"'s new certificate ...
+
+# Verify new cert
+echo | openssl s_client -connect mail.westerntire.com:993 -servername mail.westerntire.com 2>/dev/null \
+  | openssl x509 -noout -subject -issuer -dates
+# subject=CN=mail.westerntire.com
+# issuer=C=US, O=Let's Encrypt, CN=R13
+# notBefore=May 26 13:03:56 2026 GMT  notAfter=Aug 24 13:03:55 2026 GMT
+```
+
+### Pending / Incomplete Tasks
+
+- HIGH: `fix/audit-2-remediation` branch (gururmm) — awaiting Mike's merge + deploy
+- HIGH: macOS PKG installer — plan at `projects/msp-tools/guru-rmm/agent/MACOS_INSTALLATION_PLAN.md`
+- HIGH: Kittle WS2025 EVAL license activation; no backup, no firewall
+- HIGH: Kittle-Design Ken inbox rule (potential active compromise)
+- MEDIUM: TGC-SERVER Hyper-V disposition (MAS90 VM — customer decision needed)
+- MEDIUM: Neptune SSL auto-renewal — verify client is healthy before 2026-05-31 (user says it's configured)
+- LOW: Seed wiki/systems/neptune.md, wiki/systems/beast.md
+
+### Reference Information
+
+- Western Tire Syncro ticket: #32199 (ID: 109325058) — not yet billed
+- IX AutoSSL log path: `/var/cpanel/logs/autossl/<timestamp>/txt`
+- IX AutoSSL schedule: every 3 hours (confirmed from log dir timestamps)
+- cPanel UAPI: `uapi --user=<account> SSL remove_autossl_excluded_domains domains=<domain>` (note: `domains=`, not `domain=`)
+- New cert details: CN=mail.westerntire.com, Let's Encrypt R13, 2026-05-26 → 2026-08-24, auto-renews ~2026-07-25

wiki/clients/internal-infrastructure.md

@@ -115,7 +115,7 @@ Neptune is ACG's on-premises Exchange Server 2016, hosting mail for multiple cli
 - **AD Domain:** acg.local
 - **DNS Server (primary):** ACG-DC16 at 172.16.3.52 (also .50)
 - **Mailboxes:** 56 total (N-Hosting1 DB: 809 GB / 54 boxes; N-LargeBoxes DB: 313 GB / 2 boxes)
-- **Let's Encrypt cert:** CN=mail.acghosting.com, expires 2026-05-31 [WARNING] — renewal needed
+- **Let's Encrypt cert:** CN=mail.acghosting.com, expires 2026-05-31 — auto-renewal configured (win-acme or equivalent); no manual action needed unless renewal client reports failure
 - **Internal transport cert:** Thumbprint `E58BFCBAEFEFDCAED0BF9E894127A3DE64CE9C69`, expires 2026-07-22 [WARNING]
 - **Access:** Local PowerShell with Exchange Management Shell snapin (`Add-PSSnapin Microsoft.Exchange.Management.PowerShell.SnapIn`); must run as administrator.ACG on the box or via domain-admin WinRM
 - **Vault:** `infrastructure/neptune-exchange.sops.yaml` [unverified — check vault for current entry name]
@@ -228,7 +228,7 @@ Neptune is Exchange 2016 running on an unsupported OS (WS2022 after the 2026-04-
 
 **Pending transport cert renewal:** Thumbprint `E58BFCBAEFEFDCAED0BF9E894127A3DE64CE9C69` expires 2026-07-22.
 
-**Pending Neptune Let's Encrypt renewal:** CN=mail.acghosting.com cert expires 2026-05-31 — URGENT.
+**Neptune Let's Encrypt renewal:** CN=mail.acghosting.com cert expires 2026-05-31 — auto-renewal configured; monitor for renewal client errors only.
 
 **Incomplete domain MX fixes from 2026-03-17** (still unresolved as of last session):
 - `airandspaceacademy.com`: DNS on GoDaddy still points MX to mail.acghosting.com (direct, no filter) — being rejected by the transport inbound restriction rule. Needs changing to Mailprotector inbound.
@@ -259,7 +259,7 @@ The Claude Code hooks (user-prompt-submit, task-complete) spawn background `sync
 As of last session (2026-04-23):
 
 - **Neptune Exchange migration** — Build Exchange 2019 on fresh WS2022 VM. Runbook at `C:\NeptuneConfigExport-20260423\MIGRATION-RUNBOOK.md` on Neptune. Mike building the VM. Critical gate: **back up ACG-DC16 before running `/PrepareSchema`** (forest-permanent, no rollback).
-- **Neptune Let's Encrypt cert** — expires 2026-05-31. Renewal critical.
+- **Neptune Let's Encrypt cert** — expires 2026-05-31; auto-renewal configured. No manual action unless renewal client fails.
 - **Neptune internal transport cert** — expires 2026-07-22.
 - **DkimSigner re-enable / replace** — outbound mail currently unsigned. Evaluate whether Exchange DkimSigner is runtime-compatible post-KB5084071, or replace with alternative.
 - **MAIL server AD decommission** — once Exchange 2019 is live and mailboxes moved: `Remove-ADObject -Recursive` on the MAIL carcass. After that, remove hosts file entries for MAIL/mail.acg.local and DC-side DNS records (n-hosting1, n-largeboxes, mail can remain or be repurposed for the new server).

wiki/clients/western-tire.md

@@ -49,7 +49,7 @@ System/automated (not notified): `donotreply, storealert, integrilogic, receipts
 - **Home dir:** 62 GB
 - **Mailboxes:** 30 accounts under westerntire.com
 - **MySQL:** None (account does not use MySQL)
-- **SSL:** Wildcard `*.westerntire.com` from Let's Encrypt, valid to 2026-05-30 (AutoSSL should renew)
+- **SSL:** `mail.westerntire.com` managed by AutoSSL (renewed 2026-05-26, expires 2026-08-24). Wildcard `*.westerntire.com` expired 2026-05-30 — other subdomains (www, webmail, cpanel, autodiscover, webdisk, cpcontacts, cpcalendars, westerntire.com) remain AutoSSL-excluded; all excluded domains redirect or are unused. Only mail.westerntire.com matters for live service.
 
 ### Key file paths on IX
 
@@ -109,7 +109,7 @@ System/automated (not notified): `donotreply, storealert, integrilogic, receipts
 |---|---|---|
 | P1 | Monitor for user mail client issues after email setup guide was sent (new IMAP/SMTP settings) | Mike |
 | P2 | Bill ticket #32199 when scope is confirmed | Mike |
-| P2 | westerntire.com SSL cert (`*.westerntire.com`) expires 2026-05-30 — verify AutoSSL renewed | Mike |
+| P2 | Bill ticket #32199 when scope is confirmed | Mike |
 | P3 | Update Syncro customer property "DNS Detail" field — currently says "Email is on Websvr" (now IX) | Mike |
 
 ### User mail client settings (from setup guide sent 2026-04-22)
@@ -152,7 +152,7 @@ A duplicate ticket #32198 was inadvertently created and deleted.
 ## Anti-Patterns / Warnings
 
 - [WARNING] Plaintext SSH credentials for websvr and IX appeared in session log. Always retrieve from vault — never hardcode.
-- [WARNING] SSL cert `*.westerntire.com` expires 2026-05-30 — check AutoSSL renewal immediately if it's past that date.
+- SSL: only `mail.westerntire.com` is AutoSSL-managed (renewed 2026-05-26). Other subdomains are excluded; the wildcard expired 2026-05-30 but none of those domains carry live services.
 - Do NOT use `${sg{}{\\\.}{-}}` in exim.conf.local on WHM servers — WHM buildeximconf strips backslash levels and breaks the regex. Use `${tr{}{.}{-}}` instead.
 - Do NOT use tainted `$sender_address_domain` directly in file path lookups in exim 4.94+ — use `dsearch` (returns untainted value) for DKIM private key paths.
 - Do NOT look for westerntire.com mail on websvr — migration is complete; mail lives on IX.