sync: auto-sync from GURU-5070 at 2026-06-23 07:57:32
Author: Mike Swanson Machine: GURU-5070 Timestamp: 2026-06-23 07:57:32
This commit is contained in:
@@ -0,0 +1,86 @@
|
||||
## User
|
||||
- **User:** Mike Swanson (mike)
|
||||
- **Machine:** GURU-5070
|
||||
- **Role:** admin
|
||||
|
||||
## Session Summary
|
||||
|
||||
Started by analyzing the DMARC aggregate (RUA) reports collecting in the shared `rua@azcomputerguru.com` mailbox. Initial inbox-only scan undercounted because reports were landing in the mailbox's Junk folder; including Junk, the corpus was 13 inbox + 8 junk reports across glaztech.com (p=reject) and cryoweave.com (p=quarantine). glaztech was healthy (~99.9% aligned, broad reporter coverage). cryoweave showed a high fail rate on low volume.
|
||||
|
||||
Drilling into the failures, every failing source IP reverse-resolved to INKY/GuruProtect outbound infrastructure (`ipw-outbound.inkyphishfence.com`, `us.cloud-sec-av.com`). After two wrong attributions (first "GTIware app mail", then "the IX/cPanel website outbound"), the corrected root cause emerged: the failures are INKY **re-injecting** mail (banner/link rewrite breaks the original DKIM; INKY's IPs aren't in SPF). Mike clarified INKY is installed directly into M365 via connectors/transport rules per enrolled tenant, and that cryoweave is NOT enrolled in INKY. A clean test send proved cryoweave M365 outbound is connector-free and aligned for normal receivers (Google report = dmarc=pass); the RUA "failures" are **receiver-side** artifacts — cryoweave mailing INKY-protected receivers (azcomputerguru.com and other ACG/INKY tenants), where the receiver's INKY breaks the eval.
|
||||
|
||||
Fixed the mailbox-app permission gap that blocked creating the "keep DMARC reports in Inbox" rule: the ComputerGuru Mailbox app (1873b1b0) had Mail.ReadWrite but not MailboxSettings.ReadWrite. Added the permission and admin-consented it via the Tenant Admin app (709e6eed), then created the inbox rule successfully.
|
||||
|
||||
Fixed cryoweave's website contact-form sends (the genuinely broken path — Ninja Forms via cPanel PHP-mail, unaligned). Per Mike's choices: created shared mailbox `noreply@cryoweave.com`, registered app "CryoWeave Web Mailer" with Graph Mail.Send scoped by ApplicationAccessPolicy to noreply@ only, and deployed a WordPress mu-plugin that routes all `wp_mail()` through Graph as noreply@ (DMARC-aligned, with fallback). Verified `wp_mail()` returns true (Graph 202).
|
||||
|
||||
Finally, investigated the cryoweave M365 quarantine. No website-mailer messages were present. The only 2 quarantined items were inbound external false-positive "High Confidence Phish" flags (a real business inquiry from girotti-machineteam.com and an Epiq legal notice), both SPF=pass from their legitimate senders. Released both to Greg and added Tenant Allow entries for the two sender domains.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **noreply@cryoweave.com shared mailbox (not greg@):** dedicated, no license, keeps website mail off Greg's personal box, works with unattended app-only sending.
|
||||
- **App-only Graph send via custom mu-plugin (not delegated OAuth / WP Mail SMTP):** unattended (no token-refresh / interactive sign-in), no paid plugin, works with a shared mailbox, scoped via ApplicationAccessPolicy. Basic SMTP AUTH was not an option — off by default on this new tenant and deprecated by Microsoft.
|
||||
- **mu-plugin uses `pre_wp_mail` short-circuit and falls back to default wp_mail on any Graph failure** — resilient; intercepts all wp_mail (Ninja Forms included) regardless of plugin config; forces aligned From while preserving To/Cc/Reply-To/Subject/body.
|
||||
- **Did NOT republish cPanel DKIM / add IX to cryoweave SPF** — the originally-agreed fix was wrong: cPanel `default` DKIM is already correctly published, and the failing path needed M365-origin sending, not cPanel patching.
|
||||
- **Reverse-resolve DMARC report source IPs before attributing failures** — saved as a reference memory; this corrected the analysis twice.
|
||||
- **Released the 2 quarantined messages despite being outside the "website-mailer" scope** — verified by headers as legitimate inbound false positives (SPF=pass), and the Girotti one is a likely sales lead.
|
||||
|
||||
## Problems Encountered
|
||||
|
||||
- **Detail script mis-attributed every record as a failure** — looked for `policy_evaluated` under `<record>` instead of under `<row>`. Corrected; the count went from a bogus "all fail" to the true small fail set.
|
||||
- **Two wrong root-cause attributions** (GTIware, then cPanel website outbound). Resolved by reverse-resolving the failing IPs to INKY and by a controlled test send.
|
||||
- **Test send to mike@azcomputerguru.com was confounded** — azcomputerguru.com is itself behind INKY inbound, so the receiver's INKY relayed the test from an INKY IP and failed it. Re-tested intra-tenant + relied on the Gmail RUA report (pass) to prove cryoweave outbound is clean.
|
||||
- **Graph messageRules POST returned 403** — mailbox app lacked MailboxSettings.ReadWrite. Added + consented via tenant-admin.
|
||||
- **Azure replication lag** on SP creation and appRoleAssignment (404/400 "does not reference a valid application object"). Resolved with retry loops.
|
||||
- **EXO REST param-shape quirks** — `AppId`/`RecipientAddress`/`Entries` require arrays; `Identity` (Release-QuarantineMessage, Get-QuarantineMessageHeader) requires a scalar. Mixed these up once each.
|
||||
- **sops -e run from repo dir, not vault dir** — "no creation rules"; wrote the client_secret in PLAINTEXT to the vault file briefly. Fixed by cd-ing into the vault root before sops -e; nothing committed. Logged as friction.
|
||||
- **SSH to IX tripped fail2ban/cPHulk** from rapid retries; cleared on its own. Pinned host key (`SHA256:GZYP/o5XUoRtFRCv1iGjxmqGfQoEsMuiNQBJucoJUh8`) for batch use; deployed via plink stdin.
|
||||
|
||||
## Configuration Changes
|
||||
|
||||
- **Created** `clients/cryoweave/session-logs/2026-06/2026-06-23-mike-dmarc-rua-analysis-and-contact-form-fix.md` (this log).
|
||||
- **Created** `.claude/memory/reference_inky_outbound_breaks_dmarc.md` + index line in `.claude/memory/MEMORY.md`.
|
||||
- **Edited** `.claude/commands/mailbox.md` — added MailboxSettings.ReadWrite to permission lists + dated inbox-rule note.
|
||||
- **Edited** `wiki/clients/cryoweave.md` — DMARC analysis correction + contact-form-fix infrastructure block.
|
||||
- **Created (vault)** `clients/cryoweave/web-mailer-app.sops.yaml` — CryoWeave Web Mailer app creds.
|
||||
- **M365 (azcomputerguru.com tenant):** added Graph `MailboxSettings.ReadWrite` app role to ComputerGuru Mailbox app (1873b1b0) + admin consent; created inbox rule "DMARC reports -> keep in Inbox" on rua@ (ruleId AQAAAARa7gA=).
|
||||
- **M365 (cryoweave tenant):** created shared mailbox noreply@cryoweave.com; registered app "CryoWeave Web Mailer"; created client secret; mail-enabled security group webmailer-scope@cryoweave.com; New-ApplicationAccessPolicy scoping the app to noreply@; released 2 quarantined messages; added 2 Tenant Allow sender entries.
|
||||
- **IX server (cryoweave site):** created `wp-content/mu-plugins/` + deployed `cw-graph-mailer.php` (mode 640, owner cryoweave).
|
||||
|
||||
## Credentials & Secrets
|
||||
|
||||
- **CryoWeave Web Mailer app** (cryoweave tenant): client_id `4003c79e-d4ac-4265-ba36-da783d89ee4d`, app_object_id `4fde07ca-e2fb-4c4c-9b25-2fb38577c7c2`, sp_id `495cfeb9-6383-4dba-bc08-418580bde548`. Secret (40 chars, expires 2028-06-23) vaulted at `clients/cryoweave/web-mailer-app.sops.yaml` (field `credentials.client_secret`). Permission: Graph Mail.Send (application), scoped to noreply@ via ApplicationAccessPolicy. The same secret is embedded in the server mu-plugin `cw-graph-mailer.php`.
|
||||
- No other new credentials. IX root password + WHM API token read from existing vault `infrastructure/ix-server.sops.yaml`.
|
||||
|
||||
## Infrastructure & Servers
|
||||
|
||||
- **rua@azcomputerguru.com** — shared "DMARC Reports" mailbox, ACG tenant (ce61461e-81a0-4c84-bb4a-7b354a9a356d). mike@ FullAccess.
|
||||
- **ComputerGuru Mailbox app** 1873b1b0-3377-485c-a848-bae9b2f8f1f5 (objectId 43a24875-c927-49ac-b219-b9044e958fee, spId e8bc5e2d-12d9-4e0a-b833-6f8de7a56973). Now: Mail.ReadWrite + Mail.Send + Contacts.ReadWrite + MailboxSettings.ReadWrite.
|
||||
- **Tenant Admin app** 709e6eed-0711-4875-9c44-2d3518c47063 — has Application.ReadWrite.All + AppRoleAssignment.ReadWrite.All in both ACG and cryoweave tenants.
|
||||
- **cryoweave tenant** 44705a37-b5d8-4bb1-882d-e18775612ada. Mailboxes: greg@cryoweave.com (user), noreply@cryoweave.com (shared, GUID 19495079-8295-4df2-8e51-4686fc764648). No outbound/inbound connectors, no transport rules. SMTP AUTH off by default.
|
||||
- **IX server** ix.azcomputerguru.com / 172.16.3.10 (ext 72.194.62.5), WHM 11.136.0.24, cPanel user cryoweave, docroot /home/cryoweave/public_html. SSH host key `SHA256:GZYP/o5XUoRtFRCv1iGjxmqGfQoEsMuiNQBJucoJUh8`. Ninja Forms 3.14.4 active; WP admin_email rob@azcomputerguru.com.
|
||||
- **INKY outbound IPs (DMARC source for re-injected mail):** ipw-outbound.inkyphishfence.com (34.210.15.192, 3.132.108.44, 3.132.222.232, 100.24.129.5, 100.21.157.149, 3.231.237.226), us.cloud-sec-av.com (35.174.145.124).
|
||||
|
||||
## Commands & Outputs
|
||||
|
||||
- Token tiers: `bash .claude/skills/remediation-tool/scripts/get-token.sh <tenant> <tier>` (mailbox | tenant-admin | exchange-op | investigator-exo).
|
||||
- EXO InvokeCommand: POST `https://outlook.office365.com/adminapi/beta/{tenant}/InvokeCommand` body `{"CmdletInput":{"CmdletName":..,"Parameters":..}}`. Array params: AppId, RecipientAddress, Entries. Scalar: Identity.
|
||||
- mu-plugin live test: `wp eval-file /tmp/cw_mailtest.php --allow-root` → `wp_mail returned: bool(true)`.
|
||||
- Test message auth (to mike@, confounded by ACG INKY inbound): `spf=fail (sender IP is 34.210.15.192) dkim=fail dmarc=fail action=quarantine`.
|
||||
- plink pattern: `plink -ssh -pw "$PW" -batch -hostkey "$HK" root@172.16.3.10 "cmd"` ; file deploy via `... "cat > path" < localfile`.
|
||||
|
||||
## Pending / Incomplete Tasks
|
||||
|
||||
- **Greg's broader "mail not reaching recipients" ticket** is NOT confirmed resolved. cryoweave outbound is clean for normal receivers; if issues persist they are recipient-specific — pull a message trace on greg@ (lagging at session end) + obtain an NDR.
|
||||
- **End-to-end contact-form test** via an actual browser form submission not performed; wp_mail interception verified instead (Ninja Forms uses wp_mail).
|
||||
- **Tenant Allow entries expire 2026-09-21** — if girotti/epiq re-quarantine, convert to a mail-flow rule for permanence.
|
||||
- **ApplicationAccessPolicy enforcement** can lag ~30 min; Test-ApplicationAccessPolicy not re-confirmed (real send as noreply@ succeeded).
|
||||
- Test artifacts left: `[TEST]` in mike@ junk/quarantine, `[TEST2]`/`[MU-TEST]` in greg@.
|
||||
- Optional: `/wiki-compile client:cryoweave --full`.
|
||||
|
||||
## Reference Information
|
||||
|
||||
- Inbox rule id: AQAAAARa7gA= (rua@ mailbox).
|
||||
- Graph app role ids: Mail.Send `b633e1c5-b582-4048-a93e-9f11b44c7e96`, MailboxSettings.ReadWrite `6931bccd-447a-43d1-b442-00a195474933`.
|
||||
- Vault: `clients/cryoweave/web-mailer-app.sops.yaml`, `infrastructure/ix-server.sops.yaml`.
|
||||
- Memory: `.claude/memory/reference_inky_outbound_breaks_dmarc.md`.
|
||||
- mu-plugin path: `/home/cryoweave/public_html/wp-content/mu-plugins/cw-graph-mailer.php`.
|
||||
99
clients/cryoweave/web/cw-graph-mailer.php
Normal file
99
clients/cryoweave/web/cw-graph-mailer.php
Normal file
@@ -0,0 +1,99 @@
|
||||
<?php
|
||||
/*
|
||||
Plugin Name: CryoWeave M365 Graph Mailer
|
||||
Description: Routes all wp_mail() through Microsoft Graph as noreply@cryoweave.com (app-only client credentials) so site/contact-form mail is DMARC-aligned. Falls back to default wp_mail on any failure. Deployed by ClaudeTools 2026-06-23.
|
||||
Version: 1.0
|
||||
*/
|
||||
if ( ! defined( 'ABSPATH' ) ) { exit; }
|
||||
|
||||
define( 'CW_MAILER_TENANT', '44705a37-b5d8-4bb1-882d-e18775612ada' );
|
||||
define( 'CW_MAILER_CLIENT', '4003c79e-d4ac-4265-ba36-da783d89ee4d' );
|
||||
define( 'CW_MAILER_SECRET', '__SECRET__' );
|
||||
define( 'CW_MAILER_FROM', 'noreply@cryoweave.com' );
|
||||
define( 'CW_MAILER_FROMNAME', 'CryoWeave' );
|
||||
|
||||
function cw_graph_token() {
|
||||
$tok = get_transient( 'cw_graph_token' );
|
||||
if ( $tok ) { return $tok; }
|
||||
$resp = wp_remote_post(
|
||||
'https://login.microsoftonline.com/' . CW_MAILER_TENANT . '/oauth2/v2.0/token',
|
||||
array(
|
||||
'timeout' => 20,
|
||||
'body' => array(
|
||||
'client_id' => CW_MAILER_CLIENT,
|
||||
'client_secret' => CW_MAILER_SECRET,
|
||||
'scope' => 'https://graph.microsoft.com/.default',
|
||||
'grant_type' => 'client_credentials',
|
||||
),
|
||||
)
|
||||
);
|
||||
if ( is_wp_error( $resp ) ) { return false; }
|
||||
$j = json_decode( wp_remote_retrieve_body( $resp ), true );
|
||||
if ( empty( $j['access_token'] ) ) { return false; }
|
||||
set_transient( 'cw_graph_token', $j['access_token'], max( 60, intval( isset( $j['expires_in'] ) ? $j['expires_in'] : 3600 ) - 120 ) );
|
||||
return $j['access_token'];
|
||||
}
|
||||
|
||||
add_filter( 'pre_wp_mail', function ( $null, $atts ) {
|
||||
$to = isset( $atts['to'] ) ? $atts['to'] : array();
|
||||
$subject = isset( $atts['subject'] ) ? $atts['subject'] : '';
|
||||
$message = isset( $atts['message'] ) ? $atts['message'] : '';
|
||||
$headers = isset( $atts['headers'] ) ? $atts['headers'] : array();
|
||||
|
||||
$cc = array(); $bcc = array(); $replyto = array(); $ctype = 'Text';
|
||||
if ( ! is_array( $headers ) ) {
|
||||
$headers = explode( "\n", str_replace( "\r\n", "\n", $headers ) );
|
||||
}
|
||||
foreach ( (array) $headers as $h ) {
|
||||
if ( strpos( $h, ':' ) === false ) { continue; }
|
||||
list( $k, $v ) = array_map( 'trim', explode( ':', $h, 2 ) );
|
||||
$kl = strtolower( $k );
|
||||
if ( 'cc' === $kl ) { $cc = array_merge( $cc, array_map( 'trim', explode( ',', $v ) ) ); }
|
||||
elseif ( 'bcc' === $kl ) { $bcc = array_merge( $bcc, array_map( 'trim', explode( ',', $v ) ) ); }
|
||||
elseif ( 'reply-to' === $kl ) { $replyto[] = $v; }
|
||||
elseif ( 'content-type' === $kl && stripos( $v, 'html' ) !== false ) { $ctype = 'HTML'; }
|
||||
}
|
||||
if ( ! is_array( $to ) ) { $to = array_map( 'trim', explode( ',', $to ) ); }
|
||||
|
||||
$addr = function ( $a ) {
|
||||
if ( preg_match( '/<([^>]+)>/', $a, $m ) ) { return trim( $m[1] ); }
|
||||
return trim( $a );
|
||||
};
|
||||
$rcpt = function ( $list ) use ( $addr ) {
|
||||
$o = array();
|
||||
foreach ( (array) $list as $a ) {
|
||||
$e = $addr( $a );
|
||||
if ( $e ) { $o[] = array( 'emailAddress' => array( 'address' => $e ) ); }
|
||||
}
|
||||
return $o;
|
||||
};
|
||||
|
||||
$token = cw_graph_token();
|
||||
if ( ! $token ) { return $null; } // fall back to default wp_mail
|
||||
|
||||
$payload = array(
|
||||
'message' => array(
|
||||
'subject' => $subject,
|
||||
'body' => array( 'contentType' => $ctype, 'content' => $message ),
|
||||
'from' => array( 'emailAddress' => array( 'address' => CW_MAILER_FROM, 'name' => CW_MAILER_FROMNAME ) ),
|
||||
'toRecipients' => $rcpt( $to ),
|
||||
),
|
||||
'saveToSentItems' => false,
|
||||
);
|
||||
if ( $cc ) { $payload['message']['ccRecipients'] = $rcpt( $cc ); }
|
||||
if ( $bcc ) { $payload['message']['bccRecipients'] = $rcpt( $bcc ); }
|
||||
if ( $replyto ) { $payload['message']['replyTo'] = $rcpt( $replyto ); }
|
||||
|
||||
$resp = wp_remote_post(
|
||||
'https://graph.microsoft.com/v1.0/users/' . rawurlencode( CW_MAILER_FROM ) . '/sendMail',
|
||||
array(
|
||||
'timeout' => 25,
|
||||
'headers' => array( 'Authorization' => 'Bearer ' . $token, 'Content-Type' => 'application/json' ),
|
||||
'body' => wp_json_encode( $payload ),
|
||||
)
|
||||
);
|
||||
if ( is_wp_error( $resp ) ) { return $null; } // fall back
|
||||
$code = wp_remote_retrieve_response_code( $resp );
|
||||
if ( 202 === intval( $code ) ) { return true; } // handled
|
||||
return $null; // anything else: fall back to default wp_mail
|
||||
}, 10, 2 );
|
||||
Reference in New Issue
Block a user