262 lines
17 KiB
Markdown
262 lines
17 KiB
Markdown
# Session Log — Kittle BEC Incident Remediation
|
||
|
||
## User
|
||
- **User:** Mike Swanson (mike)
|
||
- **Machine:** GURU-BEAST-ROG
|
||
- **Role:** admin
|
||
|
||
---
|
||
|
||
## Session Summary
|
||
|
||
Full BEC (Business Email Compromise) investigation and remediation on kittlearizona.com tenant (ID: `3d073ebe-806a-4a5e-9035-3c7c4a264fc0`). Triggered by Howard receiving a phishing email that appeared to originate from Ken@kittlearizona.com. Mike had already manually blocked Ken's sign-in and set a temp password by the time investigation began.
|
||
|
||
Investigation used the ComputerGuru MSP app suite (Security Investigator for Graph reads, Exchange Operator for EXO reads/writes, Tenant Admin for directory and policy actions). Initial consent was obtained via admin consent URLs at the start of the session; additional consent URLs were provided and accepted mid-session for remaining app permissions.
|
||
|
||
The attacker accessed Ken's account via OWA starting at 13:24 UTC using IP 64.44.131.168 (Chicago, Nexeon Technologies — a VPN/hosting provider). They harvested contacts via python-httpx/0.28.1 from Azure IP 40.126.41.96 during 18:36–18:53 UTC, then sent 1,000 phishing emails in 5 batches from 45.134.224.220 (Kansas City, PacketHub S.A.) between 21:14–21:26 UTC. The phishing subject was "Ken Schagel shared a file with you" — a fake OneDrive file-share lure. 747 delivered, 227 failed/bounced, 740 victim notifications dispatched via EWS SOAP from admin@kittlearizona.com.
|
||
|
||
Remediation actions completed: Ken's sessions revoked and admin roles stripped (all 10); Wrex sessions revoked and password reset; 5 malicious inbox rules deleted across 3 mailboxes (Ken x2, Alexis x1, Accounting x2 — Accounting rules were actively suppressing ALL incoming mail at time of discovery, immediately restoring mail flow); Alexis PERFECTDATA and Alignable OAuth grants revoked; Lori's 10 admin roles (including Global Administrator) stripped and re-assigned User Administrator only (confirmed via directoryAudits that Lori's GA was pre-existing, not attacker-planted); victim notification sent to 740 filtered addresses; Syncro ticket #32393 updated with public comment containing temp passwords; breach report written to `clients/kittle/reports/2026-06-08-breach-check.md`; SharePoint confirmed clean; Security Defaults confirmed enabled; Ken's MFA verified clean (single iPhone 12 Pro Max, no attacker-registered devices).
|
||
|
||
---
|
||
|
||
## Key Decisions
|
||
|
||
- **EXO hidden rule sweep used Exchange Operator (not Security Investigator):** `Get-InboxRule -IncludeHidden:true` requires `Exchange.ManageAsApp` permission, which only the Exchange Operator app holds. Security Investigator has `full_access_as_app` (EWS), not the Exchange admin cmdlets tier. Switched on first 401.
|
||
|
||
- **Phishing send via EWS SOAP (not Graph SendMail):** 740-recipient victim notification sent using EWS SOAP from admin@kittlearizona.com. Graph SendMail would have required a delegated token or app-only send permission not available on this tenant. EWS worked with Exchange Operator EXO token. HTML body wrapped in `<![CDATA[...]]>` to prevent schema validation errors on `<br>` tags.
|
||
|
||
- **Lori's 10 admin roles stripped even though origin unknown:** At time of stripping, it was unknown whether roles were attacker-planted or pre-existing. Decision to strip all and re-assign legitimate scope afterward. Confirmed post-session via directoryAudits that roles were pre-existing (no "Add member to role" events in last 30 days from any non-ACG initiator). Re-assigned User Administrator as legitimate scope.
|
||
|
||
- **Victim notification filtered 7 automated/non-human addresses:** vzwpix, mms.att.net, Microsoft internal, streaming service addresses removed from the 747 delivered list before sending notifications. 740 addresses used.
|
||
|
||
- **Did not revoke Ken's MFA devices:** Ken's MFA verified clean — single iPhone 12 Pro Max, no attacker-registered devices. Advised Ken to verify at mysignins.microsoft.com himself; no forced MFA reset needed.
|
||
|
||
- **Used Search-UnifiedAuditLog (EXO REST) instead of Graph signIns:** Tenant is on O365 E3, which lacks Entra P1. Graph `/auditLogs/signIns` returns `Authentication_RequestFromNonPremiumTenantOrB2CTenant` 403. UAL via EXO REST is available without P1 and provided the full attack timeline.
|
||
|
||
- **Message trace used Get-MessageTraceV2:** `Get-MessageTrace` deprecated September 2025. Switched to `Get-MessageTraceV2` which returned full 1,000-record phishing campaign data.
|
||
|
||
- **PERFECTDATA OAuth revoked immediately, Alignable deferred then revoked:** PERFECTDATA was clearly malicious (Mail.ReadWrite, Files.ReadWrite scopes, unknown publisher). Alignable was initially deferred pending Alexis's input — later revoked at Mike's direction (offline_access + Contacts.Read, unverified publisher).
|
||
|
||
---
|
||
|
||
## Problems Encountered
|
||
|
||
- **EXO /adminapi/beta 401 with Security Investigator token:** `full_access_as_app` is an EWS permission, not the Exchange admin cmdlets tier. Fixed by switching to Exchange Operator token which holds `Exchange.ManageAsApp`. HTTP 200 after switch.
|
||
|
||
- **Remove-InboxRule JSON escaping error (400 "unrecognized escape sequence \4"):** The rule Identity string contains backslashes (e.g. `Ken\4160878082195980289`). Passing via shell variable without escaping caused malformed JSON. Fixed using `jq -n --arg identity "$IDENTITY"` to properly escape.
|
||
|
||
- **Remove-InboxRule "can't use Identity and Mailbox together":** Identity already contains mailbox prefix, so separate Mailbox parameter caused conflict. Fixed by removing Mailbox parameter from the payload.
|
||
|
||
- **EWS SOAP body schema error on HTML:** `<br>` tags in email body treated as XML elements, breaking SOAP schema validation. Fixed by wrapping body in `<![CDATA[...]]>`.
|
||
|
||
- **Sign-in logs 403 from all token tiers:** Tenant Admin lacked `AuditLog.Read.All` in manifest. Security Investigator has `AuditLog.Read.All` in roles but tenant returns `Authentication_RequestFromNonPremiumTenantOrB2CTenant`. O365 E3 does not include Entra P1 — sign-in log API requires P1. No fix available. Worked around via Search-UnifiedAuditLog (EXO REST) which does not require P1.
|
||
|
||
- **Get-MessageTrace deprecated:** Returned deprecation warning. Switched to Get-MessageTraceV2. Also removed PageSize parameter (invalid for V2).
|
||
|
||
- **directoryAudits `ne` operator not supported:** `activityDisplayName ne 'Remove member from role'` returned UnknownError. Workaround: fetch all RoleManagement events and filter via `jq` client-side.
|
||
|
||
- **UAL search returned 0 results for Lori role events:** EXO REST `Search-UnifiedAuditLog` with `Operations:Add member to role` returned empty for Lori. Resolved via Graph directoryAudits API (which did work with Security Investigator token after consent) — confirmed no role assignment events for Lori in 30 days, meaning roles were pre-existing.
|
||
|
||
---
|
||
|
||
## Configuration Changes
|
||
|
||
- **Created:** `vault/clients/kittle/m365-ken-schagel-incident.sops.yaml` — SOPS-encrypted vault entry for Ken's temp password and incident notes
|
||
- **Created:** `clients/kittle/reports/2026-06-08-breach-check.md` — Full breach incident report (attack timeline, attacker IPs, all remediation actions, open items)
|
||
- **Created:** `clients/kittle/session-logs/2026-06/2026-06-08-mike-bec-incident-remediation.md` — this file
|
||
|
||
---
|
||
|
||
## Credentials & Secrets
|
||
|
||
**Ken Schagel temp password:** `B/947405806521av`
|
||
- Vaulted at: `vault/clients/kittle/m365-ken-schagel-incident.sops.yaml`
|
||
- Communicated to Mike via Syncro ticket #32393 (public comment with temp passwords for Ken + Wrex)
|
||
- Ken's account re-enabled by Mike mid-session
|
||
|
||
**Wrex password reset:** `Kittle@1426Wrx!47E742`
|
||
- Communicated via Syncro ticket #32393
|
||
|
||
---
|
||
|
||
## Infrastructure & Servers
|
||
|
||
| Item | Value |
|
||
|------|-------|
|
||
| Tenant | kittlearizona.com |
|
||
| Tenant ID | `3d073ebe-806a-4a5e-9035-3c7c4a264fc0` |
|
||
| ACG MSP tenant ID | `ce61461e-81a0-4c84-bb4a-7b354a9a356d` |
|
||
| Syncro ticket | #32393 |
|
||
| Attacker IP 1 | 64.44.131.168 — OWA browser access, Chicago IL, AS20278 Nexeon Technologies (VPN/hosting) |
|
||
| Attacker IP 2 | 40.126.41.96 — Contact scraping via python-httpx, Microsoft Azure |
|
||
| Attacker IP 3 | 45.134.224.220 — Bulk phishing send, Kansas City MO, AS147049 PacketHub S.A. |
|
||
| Attacker tool | python-httpx/0.28.1, OAuth token for Microsoft Desktop app `d3590ed6-52b3-4102-aeff-aad2292ab01c` |
|
||
| Attacker AAD session | `0031c64a-94a8-7629-20ad-c42db69d76c7` |
|
||
| Compromise window | 13:24–21:41 UTC 2026-06-08 |
|
||
|
||
### MSP App Credentials Used
|
||
|
||
| App | Client ID |
|
||
|-----|-----------|
|
||
| Security Investigator | `bfbc12a4-f0dd-4e12-b06d-997e7271e10c` |
|
||
| Exchange Operator | `b43e7342-5b4b-492f-890f-bb5a4f7f40e9` |
|
||
| User Manager | `64fac46b-8b44-41ad-93ee-7da03927576c` |
|
||
| Tenant Admin | `709e6eed-0711-4875-9c44-2d3518c47063` |
|
||
|
||
---
|
||
|
||
## Commands & Outputs
|
||
|
||
### Attack timeline (from UAL + message trace)
|
||
|
||
```
|
||
09:03 UTC Normal Outlook sync (Microsoft IPs) — pre-compromise
|
||
13:24 UTC [BREACH START] OWA login — 64.44.131.168 (Chicago, Nexeon VPN)
|
||
13:37 UTC Ken's T-Mobile phone (legitimate, unaware)
|
||
15:00 UTC Attacker returns — 64.44.131.168
|
||
15:17 UTC Ken sends legitimate email via Cox (Phoenix AZ)
|
||
15:32 UTC Attacker sends test email from OWA — concurrent with Ken
|
||
16:14 UTC Attacker sends second test email
|
||
18:36 UTC Contact harvest starts — python-httpx/0.28.1 from Azure 40.126.41.96
|
||
18:52 UTC Attacker reviews Sent/Deleted/RSS Feeds from OWA
|
||
18:53 UTC Contact harvest ends (250+ MailItemsAccessed events)
|
||
21:14 UTC Phishing batch 1: 17 recipients
|
||
21:16 UTC Phishing batch 2: 300 recipients
|
||
21:20 UTC Phishing batch 3: 300 recipients
|
||
21:23 UTC Phishing batch 4: 300 recipients
|
||
21:26 UTC Phishing batch 5: 83 recipients — from 45.134.224.220 (PacketHub)
|
||
21:27 UTC Ken's SSPR password reset attempt
|
||
~21:30 UTC Howard (ACG) receives phishing email — incident detected
|
||
21:41 UTC Mike manually blocks Ken sign-in in portal, sets temp password
|
||
~22:00 UTC ACG investigation and remediation begins
|
||
22:05 UTC ACG removes Lori's 10 admin roles (Tenant Admin app)
|
||
22:06 UTC Lori sessions revoked
|
||
```
|
||
|
||
### Malicious inbox rules deleted
|
||
|
||
```
|
||
Ken@kittlearizona.com "." Move ALL mail → RSS Feeds, MarkAsRead, StopProcessing [DELETED]
|
||
Ken@kittlearizona.com "Admin" Move ALL mail → RSS Feeds, MarkAsRead, StopProcessing [DELETED]
|
||
alexis@kittlearizona.com "..." Move ALL mail → RSS Feeds, MarkAsRead, StopProcessing [DELETED]
|
||
Accounting@kittlearizona.com ".." Move mail FROM Ken → RSS Feeds, Priority 1 [DELETED]
|
||
Accounting@kittlearizona.com "..." Move ALL mail → RSS Feeds, Priority 2 [DELETED]
|
||
```
|
||
Note: Accounting ".." + "..." were actively suppressing ALL incoming mail at time of discovery.
|
||
|
||
### Phishing campaign stats
|
||
|
||
```
|
||
Total sent: 1,000
|
||
Delivered: 747
|
||
Failed/bounced: 227
|
||
Pending: 25
|
||
Notifications sent: 740 (7 automated addresses filtered)
|
||
Subject: "Ken Schagel shared a file with you"
|
||
```
|
||
|
||
### OAuth grants revoked
|
||
|
||
```
|
||
PERFECTDATA app Mail.ReadWrite, Files.ReadWrite, offline_access [REVOKED]
|
||
Alignable app offline_access, User.Read, Contacts.Read [REVOKED]
|
||
Grant ID: jB3LklISEEOHpW2kH5IbQLz8wKqAnj1KmLeBzb1HLJrh6qF03cBERamPOhj4CXha
|
||
Client SP: 92cb1d8c-1252-4310-87a5-6da41f921b40
|
||
```
|
||
|
||
### Lori role changes
|
||
|
||
```
|
||
Stripped (all pre-existing): Global Administrator, Exchange Administrator, User Administrator,
|
||
Teams Administrator, SharePoint Administrator, Helpdesk Administrator, AI Administrator,
|
||
Global Reader, Service Support Administrator, User Experience Success Manager
|
||
Re-assigned: User Administrator (roleTemplateId: fe930be7-5e62-47db-91af-98c3a49a38b1)
|
||
Role object ID in tenant: 1321d5cd-17bb-40de-891b-1e85667e1c5a
|
||
Lori user ID: 5817629b-5832-43c6-b74c-86a05c29c852
|
||
```
|
||
|
||
### Key user IDs
|
||
|
||
```
|
||
Ken Schagel Ken@kittlearizona.com (admin roles stripped, re-enabled by Mike)
|
||
Lori Schagel Lori@kittlearizona.com ID: 5817629b-5832-43c6-b74c-86a05c29c852
|
||
Alexis alexis@kittlearizona.com ID: 74a1eae1-c0dd-4544-a98f-3a18f809785a
|
||
Wrex Wrex@kittlearizona.com (sessions revoked, password reset)
|
||
```
|
||
|
||
---
|
||
|
||
## Pending / Incomplete Tasks
|
||
|
||
1. **Christina Micek inbox rule on Ken** — `StopProcessingRules:true`, no action, no filter. Unknown if legitimate or attacker-planted. Needs Ken to confirm. Delete if he doesn't recognize it.
|
||
|
||
2. **Re-add admin roles to Ken** — All 10 stripped during containment. Ken is Global Admin by function; re-add Global Administrator and Exchange Administrator once incident is declared closed and Ken's account is fully verified clean.
|
||
|
||
3. **Lori GA access review** — Confirmed pre-existing (not attacker-planted). Recommend discussing with Ken whether Lori legitimately needs any admin role at all. Downscoped to User Administrator for now.
|
||
|
||
4. **MFA cleanup** — Alexis has duplicate Authenticator registrations ("iPhone 12 Pro Max" x2). Lori has two Authenticator devices (SM-G975U + SM-F766U, likely old phone not removed). Users can self-serve at mysignins.microsoft.com or ACG can reset.
|
||
|
||
5. **Phishing URL unknown** — Email body purged when Ken's account was disabled. Send IP 45.134.224.220 (PacketHub S.A., AS147049) is known. Submit to threat intel if needed.
|
||
|
||
6. **Entra P1 licensing** — Sign-in logs blind without it. Tenant on O365 E3, not M365 E3. Recommend Entra P1 add-on or upgrade. Without P1, a Conditional Access policy for foreign IP blocking also cannot be enforced.
|
||
|
||
7. **DKIM/DMARC** — Not configured on kittlearizona.com. DMARC reject/quarantine would reduce future phishing deliverability from this domain.
|
||
|
||
8. **Alexis MFA duplicate** — One "iPhone 12 Pro Max" Authenticator entry is likely a stale registration. Should be cleaned up but is low priority now that PERFECTDATA and Alignable are revoked.
|
||
|
||
---
|
||
|
||
## Reference Information
|
||
|
||
- **Syncro ticket:** #32393 (public comment added with temp passwords for Ken + Wrex)
|
||
- **Breach report:** `clients/kittle/reports/2026-06-08-breach-check.md`
|
||
- **Vault entry:** `vault/clients/kittle/m365-ken-schagel-incident.sops.yaml`
|
||
- **Victim notification:** sent from admin@kittlearizona.com via EWS SOAP, 3 batches, 740 recipients
|
||
- **Attacker OAuth app (Microsoft Desktop):** `d3590ed6-52b3-4102-aeff-aad2292ab01c`
|
||
- **Attacker AAD session:** `0031c64a-94a8-7629-20ad-c42db69d76c7`
|
||
- **Security Defaults:** enabled (`isEnabled: true`) on this tenant
|
||
- **SharePoint:** clean — no attacker-created files, pages, or external sharing links
|
||
- **EXO REST endpoint:** `https://outlook.office365.com/adminapi/beta/{tenantId}/InvokeCommand`
|
||
- **EWS endpoint:** `https://outlook.office365.com/EWS/Exchange.asmx`
|
||
- **Graph directoryAudits:** confirmed working with Security Investigator token (no P1 needed for this endpoint)
|
||
- **Graph signIns:** blocked — requires Entra P1 (`Authentication_RequestFromNonPremiumTenantOrB2CTenant`)
|
||
|
||
---
|
||
|
||
## Update: 16:23 PT — Post-save follow-up actions
|
||
|
||
### Additional actions completed after initial save
|
||
|
||
**Alignable OAuth on Alexis — revoked:**
|
||
At Mike's direction, revoked the Alignable OAuth grant on alexis@kittlearizona.com.
|
||
- Grant ID: `jB3LklISEEOHpW2kH5IbQLz8wKqAnj1KmLeBzb1HLJrh6qF03cBERamPOhj4CXha`
|
||
- App SP client ID: `92cb1d8c-1252-4310-87a5-6da41f921b40`
|
||
- Scopes: `offline_access User.Read Contacts.Read`
|
||
- HTTP 204 [OK]
|
||
|
||
**Lori Schagel — downscoped to User Administrator:**
|
||
All 10 pre-existing admin roles had been stripped during containment. Re-assigned User Administrator only (confirmed pre-existing GA was unnecessary exposure, not attacker-planted).
|
||
- Lori user ID: `5817629b-5832-43c6-b74c-86a05c29c852`
|
||
- User Administrator role object ID: `1321d5cd-17bb-40de-891b-1e85667e1c5a`
|
||
- roleTemplateId: `fe930be7-5e62-47db-91af-98c3a49a38b1`
|
||
- HTTP 204 [OK]
|
||
|
||
**Lori role assignment timing — confirmed pre-existing:**
|
||
directoryAudits query (last 30 days, all RoleManagement events) returned only ACG-initiated removals. Zero "Add member to role" events from any other initiator in the 30-day window. Lori's GA was assigned more than 30 days before the incident — pre-existing, not attacker-planted. Breach report updated to reflect this finding.
|
||
|
||
**Victim notification confirmed sent:**
|
||
Mike asked explicitly — confirmed 740 emails sent via EWS SOAP from admin@kittlearizona.com in 3 batches. All NoError. 747 delivered by phishing campaign; 7 automated addresses filtered (vzwpix, att.net MMS, Microsoft internal, streaming service) before notification send.
|
||
|
||
**All consent links accepted by Mike** — confirmed all 5 MSP apps (Security Investigator, Exchange Operator, User Manager, Tenant Admin, Defender Add-on) have admin consent in Kittle tenant.
|
||
|
||
**Thread deletion failed:** Bot received HTTP 403 Missing Permissions attempting to delete Discord thread 1513652236444504197. Thread remains open.
|
||
|
||
### Breach report updates
|
||
|
||
`clients/kittle/reports/2026-06-08-breach-check.md` updated with:
|
||
- Lori role assignment confirmed pre-existing (not attacker-planted)
|
||
- Alignable OAuth revocation added to remediation table
|
||
- Lori User Administrator re-assignment added to remediation table
|
||
- Open items updated to reflect resolved items (Ken MFA clean, Lori role timing resolved)
|
||
- New open item: Lori GA access review (recommend discussing with Ken whether she needs any admin role)
|