sync: auto-sync from GURU-5070 at 2026-07-17 05:35:10

Author: Mike Swanson Machine: GURU-5070 Timestamp: 2026-07-17 05:35:10
2026-07-17 05:35:39 -07:00
parent 7db991b037
commit 51b09dacbb
11 changed files with 200 additions and 7 deletions
--- a/.claude/memory/MEMORY.md
+++ b/.claude/memory/MEMORY.md
@@ -41,6 +41,7 @@
 - [GuruRMM technical reference](reference_gururmm.md) — Server (172.16.3.30) layout + downloads dir `/var/www/gururmm/downloads` + `.channel` sidecar rollout control (stable/beta) + privileged server access via the server's OWN root RMM agent (hostname `gururmm`, no SSH needed; plink fallback) + API + `context=user_session` (WTS impersonation) + build-pipeline vendoring at `deploy/build-pipeline/` + Linux agent systemd sandbox trap.
 - [GuruRMM command timeout_seconds](reference_gururmm_command_timeout_seconds.md) — agent command dispatch honors `timeout_seconds`, NOT `timeout`; long jobs die ~300s / go zombie (`running`, empty stdout) otherwise. Cost Birth Biologic a full day.
 - [SharePoint Graph large-file upload](reference_sharepoint_graph_large_file_upload.md) — <4MB simple PUT, >=4MB MUST use chunked upload session (Content-Range); `\\?\` long paths; idempotent size-check; verify counts via /root/delta; single stream ~40Mbps (SPO throttle).
+- [Exchange REST uses exchange-op tier](reference_exchange_online_exchangeop_tier.md) — ALL Exchange adminapi (Get-Mailbox/InboxRule/MailboxPermission, Set-*) goes through `exchange-op`, NEVER `investigator-exo` (401s: Security Investigator app lacks `Exchange.ManageAsApp`). 401=wrong app→exchange-op; 403=missing Exchange Admin role. Recurred 6+ times.
 - [RMM-spawn headless Claude](reference_rmm_spawn_headless_claude.md) — run `claude -p` on any RMM-managed Windows box with Claude Code (reaches coord-isolated sites like AD2); use `context:user_session`, UNSET the stale machine `ANTHROPIC_API_KEY` (shadows OAuth → "Invalid API key"), detach + poll a DONE marker. Validated on AD2 2026-07-01.
 - [RMM agent update model](rmm-agent-update-model.md) — Agent updates are server-PUSH on heartbeat (no self-poll); available versions = filesystem scan needing a `.sha256`; promote flips `.channel` sidecars beta→stable globally. Two stranders: beta-first freezes stable until an explicit promote; agents older than ~0.6.50 re-enroll with a NEW device_id/agent row when updated.
 - [GuruRMM physical server storage](gururmm-physical-server-storage.md) — New box 172.16.1.231 (temp IP→will be .30), Ubuntu 26.04, ssh key `gururmm-physical`/alias `gururmm-new`. SSD (915G root) = HOT (PG default tablespace + WAL + builds); HDD ext4 at `/data` = COLD (`gururmm_cold` PG tablespace for aged `agent_logs` partitions + downloads + backups + archive). The #3 retention answer.
@@ -230,5 +231,5 @@
 - [Background tasks — no shell `&`](feedback_background_task_no_ampersand.md) — with run_in_background:true, run the command in the foreground of the shell; adding `&`/`disown` forks + exits 0 instantly, orphaning a blocking wait so it never delivers (hit twice building ask-forum)
 - [ask-forum — human-in-the-loop via #ct-forum](../skills/ask-forum/SKILL.md) — ask a teammate a question in the private ct-forum Discord forum, get their human answer back in-session; forum-only, any human can answer, run the --wait as a proper background task
 - [Claude Code Bun crash -> pin 2.1.110](reference_claude_code_bun_crash_pin.md) - v2.1.114+ segfaults on Win11 at new-session start; npm pin 2.1.110 + DISABLE_AUTOUPDATER=1; unpin on #55219 fix
- [GuruRMM build-server SSH key](reference_gururmm_build_server_ssh.md) — guru@172.16.3.30 key is ~/.ssh/gururmm-physical (not id_*); password fallback vault infrastructure/gururmm-server
- [LAB-SVR retired](feedback_labsvr_retired.md) — Len's Auto LAB-SVR replaced by LAB-SERVER; never chase LAB-SVR
+- [GuruRMM build-server SSH key](reference_gururmm_build_server_ssh.md) — guru@172.16.3.30 key is ~/.ssh/gururmm-physical (not id_*); password fallback vault infrastructure/gururmm-server
+- [LAB-SVR retired](feedback_labsvr_retired.md) — Len's Auto LAB-SVR replaced by LAB-SERVER; never chase LAB-SVR
--- a/.claude/memory/reference_exchange_online_exchangeop_tier.md
+++ b/.claude/memory/reference_exchange_online_exchangeop_tier.md
@@ -0,0 +1,32 @@
+---
+name: reference_exchange_online_exchangeop_tier
+description: Exchange REST (adminapi InvokeCommand) ALWAYS uses the exchange-op tier, never investigator-exo (which 401s — lacks Exchange.ManageAsApp)
+metadata:
+  type: reference
+---
+
+**Every Exchange Online REST call in the remediation-tool suite — READ and WRITE — goes through the
+`exchange-op` tier. NEVER `investigator-exo`.** This friction has recurred 6+ times (Mike, 2026-07-16).
+
+Why: the Exchange adminapi (`https://outlook.office365.com/adminapi/beta/{tenant}/InvokeCommand`, i.e.
+Get-Mailbox / Get-InboxRule / Get-MailboxPermission / Get-RecipientPermission / Set-Mailbox / etc.)
+requires the calling app to hold the **`Exchange.ManageAsApp`** application permission (Office 365
+Exchange Online app-role `dc50a0fb-09a3-484d-be87-e023b12c6440`) **in addition to** the Exchange
+Administrator directory role.
+
+- **`exchange-op`** = Exchange Operator app `b43e7342-5b4b-492f-890f-bb5a4f7f40e9` — has BOTH `Exchange.ManageAsApp`
+  AND the Exchange Administrator role. This is the ONLY working tier for Exchange REST (read or write).
+- **`investigator-exo`** = Security Investigator app `bfbc12a4-...` — has the Exchange Admin *role* but NOT
+  `Exchange.ManageAsApp`, so it returns **401 on every** adminapi call, including read-only `Get-*`. The
+  "Exchange Online read" label on this tier is aspirational/wrong.
+
+**Diagnosis rule:** `401` on adminapi = wrong app (missing `Exchange.ManageAsApp`) → switch to `exchange-op`.
+`403` = the `exchange-op` SP is missing the Exchange Administrator *directory role* on that tenant → assign it
+(Entra > Roles > Exchange Administrator > add the app). Assigning Exchange Admin to the Security Investigator
+SP does NOT fix the 401 — the app-permission (baked into the app registration), not the role, is the blocker.
+
+Verified live on ACG (`azcomputerguru.com`, tenant `ce61461e-81a0-4c84-bb4a-7b354a9a356d`) 2026-07-16:
+Investigator SP `9d242c15-6cd3-46ec-96d3-bcafaaaca333` `Exchange.ManageAsApp`=NO; Operator SP
+`83c225f1-b38d-4063-9fdd-642b6b09ae8b`=YES. Docs corrected same day: `SKILL.md`, `references/gotchas.md`,
+and `scripts/user-breach-check.sh` (was minting `investigator-exo` for the 03a-d mailbox checks → they
+silently returned empty on every run; now uses `exchange-op`). See also [[reference_remediation_tool_365_access]].
--- a/.claude/skills/remediation-tool/.gitignore
+++ b/.claude/skills/remediation-tool/.gitignore
@@ -0,0 +1 @@
+.bb-tenant.json
--- a/.claude/skills/remediation-tool/SKILL.md
+++ b/.claude/skills/remediation-tool/SKILL.md
@@ -14,7 +14,7 @@ Five multi-tenant apps cover distinct privilege tiers. Use only what the task re
 | Tier | App display name | App ID | Vault file | Scope |
 |---|---|---|---|---|
 | `investigator` | ComputerGuru Security Investigator | `bfbc12a4-f0dd-4e12-b06d-997e7271e10c` | `computerguru-security-investigator.sops.yaml` | Graph read-only |
-| `investigator-exo` | ComputerGuru Security Investigator | `bfbc12a4-f0dd-4e12-b06d-997e7271e10c` | `computerguru-security-investigator.sops.yaml` | Exchange Online read |
+| `investigator-exo` | ComputerGuru Security Investigator | `bfbc12a4-f0dd-4e12-b06d-997e7271e10c` | `computerguru-security-investigator.sops.yaml` | Exchange Online read — **BROKEN: app lacks `Exchange.ManageAsApp`, adminapi `InvokeCommand` 401s. Use `exchange-op` for ALL Exchange REST (read + write).** |
 | `exchange-op` | ComputerGuru Exchange Operator | `b43e7342-5b4b-492f-890f-bb5a4f7f40e9` | `computerguru-exchange-operator.sops.yaml` | Exchange Online write |
 | `user-manager` | ComputerGuru User Manager | `64fac46b-8b44-41ad-93ee-7da03927576c` | `computerguru-user-manager.sops.yaml` | Graph user/group write |
 | `tenant-admin` | ComputerGuru Tenant Admin | `709e6eed-0711-4875-9c44-2d3518c47063` | `computerguru-tenant-admin.sops.yaml` | Graph high-privilege |
@@ -23,7 +23,7 @@ Five multi-tenant apps cover distinct privilege tiers. Use only what the task re

 **The suite has broad, working access across ALL of M365 — Graph, Exchange Online, Defender, AND SharePoint Online.** Before concluding "the tool can't do X / access denied," verify against the live permission map in `references/app-permissions-and-sharepoint.md` (decode the token `roles` claim). An `accessDenied` usually means wrong tier or wrong endpoint for a scope we DO hold — not a real gap. Two recurring traps: (1) **SharePoint app-only requires a certificate** — a `client_secret` token is rejected everywhere in SharePoint with `"Unsupported app only token"` (get-token.sh forces cert for the `sharepoint*` tiers); (2) Graph `GET /admin/sharepoint/settings` needs a scope no app holds — read/write SharePoint tenant settings via the **CSOM/REST admin API** (`sharepoint-admin` tier) instead. Full map, gotchas, and CSOM examples: `references/app-permissions-and-sharepoint.md`.

-**Default for breach checks:** use `investigator` (Graph) + `investigator-exo` (Exchange read). Escalate to write tiers only when remediating.
+**Default for breach checks:** use `investigator` (Graph) + **`exchange-op`** for ALL Exchange REST (read AND write). NOTE: `investigator-exo` 401s on the Exchange adminapi — the Security Investigator app holds the Exchange Admin *role* but NOT the `Exchange.ManageAsApp` *app-permission* that `InvokeCommand` requires — so even read-only `Get-*` cmdlets go through `exchange-op`. Escalate nothing extra; `exchange-op` is the read path too. Full detail: `references/gotchas.md`.

 ## Auto-Invocation Behavior

@@ -41,7 +41,7 @@ When triggered automatically (vs. via `/remediation-tool`), follow the same work

 - The SOPS vault is accessible via `.claude/identity.json` `vault_path` field. The scripts auto-resolve the vault location from identity.json — no hardcoded paths.
 - `jq`, `curl`, `bash` are available.
- For Exchange REST checks: confirm the target tenant has **Exchange Administrator** role assigned to the **Security Investigator** SP (for reads) or **Exchange Operator** SP (for writes). If any Exchange REST call returns 403, emit the tenant-scoped Entra Roles link from `references/gotchas.md`.
+- **Exchange REST (adminapi `InvokeCommand`) — read AND write — ALWAYS use the `exchange-op` tier.** The Exchange Operator app (`b43e7342-5b4b-492f-890f-bb5a4f7f40e9`) is the ONLY suite app that holds `Exchange.ManageAsApp` (required for adminapi) PLUS the Exchange Administrator directory role. `investigator-exo` returns **401** on every Exchange cmdlet — the Security Investigator app has the Exchange Admin *role* but NOT the `Exchange.ManageAsApp` *permission* (verified live on ACG 2026-07-16: Investigator SP `Exchange.ManageAsApp`=NO, Operator SP=YES). Diagnosis rule: **401 = wrong app (missing `Exchange.ManageAsApp`) → switch to `exchange-op`; 403 = the `exchange-op` SP is missing the Exchange Administrator directory role on that tenant** → emit the Entra Roles link from `references/gotchas.md`. Do NOT "assign Exchange Admin to the Security Investigator SP" — it does not fix the 401 (the app-permission, not the role, is the blocker).
 - For Identity Protection checks: `IdentityRiskyUser.Read.All` is in the Security Investigator manifest AND the tenant has consented to that app. If 403, emit the per-app consent URL from `references/gotchas.md`.
 - For Defender checks: confirm tenant has Microsoft Defender for Endpoint (MDE) license before using `defender` tier — it returns AADSTS650052 otherwise.

--- a/.claude/skills/remediation-tool/references/gotchas.md
+++ b/.claude/skills/remediation-tool/references/gotchas.md
@@ -38,13 +38,32 @@ Graph API permissions alone are not enough. Most privileged operations require d

 | Operation | App tier | Required directory role on that SP |
 |---|---|---|
-| Exchange REST read (Get-InboxRule, Get-Mailbox) | `investigator-exo` | Exchange Administrator |
+| Exchange REST read (Get-InboxRule, Get-Mailbox, Get-MailboxPermission, Get-RecipientPermission) | **`exchange-op`** (NOT `investigator-exo` — see callout) | Exchange Administrator |
 | Exchange REST write (Set-Mailbox, Remove-InboxRule) | `exchange-op` | Exchange Administrator |
 | Password reset, user property updates | `user-manager` | User Administrator |
 | MFA method reset | `user-manager` | Authentication Administrator |
 | Conditional Access reads/writes | `tenant-admin` | Conditional Access Administrator OR Security Administrator |
 | Teams policies | `tenant-admin` | Teams Administrator |

+> **[AUTHORITATIVE — stop re-deriving this] Exchange REST goes through `exchange-op`, NEVER `investigator-exo`.**
+> The Exchange adminapi (`https://outlook.office365.com/adminapi/beta/{tenant}/InvokeCommand`) requires the
+> calling app to hold the **`Exchange.ManageAsApp`** application permission (Office 365 Exchange Online app role
+> `dc50a0fb-09a3-484d-be87-e023b12c6440`) **in addition to** the Exchange Administrator directory role. Only the
+> **Exchange Operator** app (`exchange-op`, `b43e7342-5b4b-492f-890f-bb5a4f7f40e9`) has BOTH. The **Security
+> Investigator** app (`investigator`/`investigator-exo`, `bfbc12a4-...`) has the Exchange Admin *role* but NOT
+> `Exchange.ManageAsApp`, so **every** `investigator-exo` adminapi call returns **401** — including read-only
+> `Get-*` cmdlets. Verified live on ACG (`azcomputerguru.com`) 2026-07-16: Investigator SP `Exchange.ManageAsApp`=**NO**,
+> Operator SP=**YES**. **So use `exchange-op` for ALL Exchange cmdlets, read and write.**
+>
+> **401 vs 403:** `401` on adminapi = wrong app (missing `Exchange.ManageAsApp`) → you are on `investigator-exo`,
+> switch to `exchange-op`. `403` = the `exchange-op` SP is missing the **Exchange Administrator** directory role on
+> that specific tenant → assign it (steps below). Assigning Exchange Admin to the *Security Investigator* SP does
+> **not** help — the missing piece there is the app-permission, which is baked into the app registration, not a
+> per-tenant role. Authoritative ACG SP→role map (verified 2026-07-16): `exchange-op` SP `83c225f1-b38d-4063-9fdd-642b6b09ae8b`
+> = Exchange Administrator (+ `Exchange.ManageAsApp`); `investigator` SP `9d242c15-6cd3-46ec-96d3-bcafaaaca333` = Exchange
+> Administrator only (no ManageAsApp); `user-manager` SP `04020a4e-...` = Authentication Administrator + User Administrator;
+> `tenant-admin` SP `046a7f70-...` = Conditional Access Administrator.
+
 ### How to assign a role to an SP in a customer tenant

 1. Sign into the customer's Entra admin center as Global Admin:
--- a/.claude/skills/remediation-tool/scripts/.gitignore
+++ b/.claude/skills/remediation-tool/scripts/.gitignore
@@ -0,0 +1,2 @@
+.breakglass-lastcheck
+.breakglass-alert.log
--- a/.claude/skills/remediation-tool/scripts/breakglass-signin-alert.sh
+++ b/.claude/skills/remediation-tool/scripts/breakglass-signin-alert.sh
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+# breakglass-signin-alert.sh — alert #dev-alerts on ANY sign-in by the ACG M365
+# break-glass account (sysadmin@azcomputerguru.onmicrosoft.com). A break-glass login
+# should be rare and always investigated. Run on a schedule (hourly/daily).
+#
+# State: tracks the last-checked UTC timestamp in a sidecar so it only alerts on NEW
+# sign-ins. First run seeds the timestamp (no alert for history) unless --backfill <ISO>.
+# DRY_RUN=1 prints what it would post instead of posting.
+#
+# Usage: breakglass-signin-alert.sh [--since <ISO8601>]
+set -u
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ROOT="$(cd "$SCRIPT_DIR/../../../.." && pwd)"   # skill scripts -> repo root
+TEN="ce61461e-81a0-4c84-bb4a-7b354a9a356d"       # azcomputerguru.com
+BG="8c363d17-02ed-45e1-97d0-49e5819e6ff6"        # sysadmin@azcomputerguru.onmicrosoft.com
+STATE="$SCRIPT_DIR/.breakglass-lastcheck"
+
+NOW="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
+if [ "${1:-}" = "--since" ] && [ -n "${2:-}" ]; then SINCE="$2";
+elif [ -f "$STATE" ]; then SINCE="$(cat "$STATE")";
+else SINCE="$NOW"; echo "$NOW" > "$STATE"; echo "[seed] first run — seeded $NOW, no history alerted"; exit 0; fi
+
+G=$("$SCRIPT_DIR/get-token.sh" "$TEN" investigator 2>/dev/null) || { echo "[err] token"; exit 1; }
+RESP=$(curl -s "https://graph.microsoft.com/v1.0/auditLogs/signIns?\$filter=userId+eq+'$BG'+and+createdDateTime+ge+$SINCE&\$top=50" -H "Authorization: Bearer $G" | tr -d '\000-\037')
+N=$(echo "$RESP" | jq -r '.value | length' 2>/dev/null)
+[ -z "$N" ] && { echo "[err] query"; exit 1; }
+
+if [ "$N" -gt 0 ]; then
+  while IFS= read -r line; do
+    [ -z "$line" ] && continue
+    MSG="[SECURITY] BREAK-GLASS SIGN-IN - sysadmin@azcomputerguru.onmicrosoft.com used: $line -- investigate if unplanned."
+    if [ "${DRY_RUN:-0}" = "1" ]; then echo "WOULD POST: $MSG";
+    else bash "$ROOT/.claude/scripts/post-bot-alert.sh" "$MSG" >/dev/null 2>&1; echo "[alert] posted: $line"; fi
+  done < <(echo "$RESP" | jq -r '.value[] | "\(.createdDateTime) result=\(.status.errorCode) app=\(.appDisplayName) ip=\(.ipAddress) loc=\(.location.countryOrRegion)/\(.location.city // "?")"')
+else
+  echo "[ok] no break-glass sign-ins since $SINCE"
+fi
+echo "$NOW" > "$STATE"
--- a/.claude/skills/remediation-tool/scripts/run-breakglass-alert.cmd
+++ b/.claude/skills/remediation-tool/scripts/run-breakglass-alert.cmd
@@ -0,0 +1,5 @@
+@echo off
+REM Wrapper for the ACG M365 break-glass sign-in alert. Invoked by the
+REM "ACG-BreakGlass-SignIn-Alert" scheduled task. Uses a Git login shell so the
+REM SOPS/age vault env loads, then runs the check from the repo root and logs output.
+"C:\Program Files\Git\bin\bash.exe" -lc "cd /d/ClaudeTools && bash .claude/skills/remediation-tool/scripts/breakglass-signin-alert.sh >> .claude/skills/remediation-tool/scripts/.breakglass-alert.log 2>&1"
--- a/.claude/skills/remediation-tool/scripts/user-breach-check.sh
+++ b/.claude/skills/remediation-tool/scripts/user-breach-check.sh
@@ -13,7 +13,10 @@ UPN="${2:?usage: user-breach-check.sh <tenant-id|domain> <upn>}"

 TENANT_ID=$("$SCRIPT_DIR/resolve-tenant.sh" "$TENANT_INPUT")
 GT=$("$SCRIPT_DIR/get-token.sh" "$TENANT_ID" investigator)
-EXO=$("$SCRIPT_DIR/get-token.sh" "$TENANT_ID" investigator-exo) || EXO=""
+# Exchange REST (adminapi InvokeCommand) MUST use exchange-op: it holds Exchange.ManageAsApp
+# + Exchange Administrator. investigator-exo 401s (Security Investigator app lacks ManageAsApp)
+# — see references/gotchas.md "Exchange REST goes through exchange-op". (fixed 2026-07-16)
+EXO=$("$SCRIPT_DIR/get-token.sh" "$TENANT_ID" exchange-op) || EXO=""

 USER_SLUG=$(echo "$UPN" | tr '@.' '__')
 OUT="/tmp/remediation-tool/$TENANT_ID/user-breach/$USER_SLUG"
--- a/clients/azcomputerguru.com/session-logs/2026-07-16-mike-mailbox-security-investigation-exchange-tier-breakglass.md
+++ b/clients/azcomputerguru.com/session-logs/2026-07-16-mike-mailbox-security-investigation-exchange-tier-breakglass.md
@@ -0,0 +1,88 @@
+# ACG mailbox security investigation + hardening (mike@)
+
+**Date:** 2026-07-16 (UTC)
+
+## User
+- **User:** Mike Swanson (mike)
+- **Machine:** GURU-5070
+- **Role:** admin
+
+## Trigger
+Mike's Outlook "view" kept resetting — suspected someone else was changing his mailbox
+settings. Requested a full investigation of `mike@azcomputerguru.com`.
+
+## Outcome (TL;DR)
+- **No compromise, no delegates.** Nobody else has access to the mailbox. The changing view
+  is Mike's own many clients (New Outlook vs classic, OWA, Apple Mail, mobile, 3 AAD-joined PCs)
+  fighting over roaming view settings — not another person.
+- **Real finding:** an active distributed **password-spray attack** on the account — all failing
+  (smart-lockout + enforced MFA + legacy-auth block). No successful foreign sign-in.
+- Hardened: pruned stale credentials, made a true break-glass account, added sign-in alerting.
+- **Also fixed a recurring fleet-wide friction:** Exchange REST must use the `exchange-op` tier
+  (docs + breach-check script corrected).
+
+## Investigation findings (tenant ce61461e-81a0-4c84-bb4a-7b354a9a356d)
+- **Sign-ins (30d, interactive):** 200 records, **all failures** — `50053` smart-lockout (×195),
+  `50126` bad password (×5) — from DE (52), US-spoofed (108), CN, KR, IN, RU, BR, etc. Only 6
+  successful sign-ins, all US (Phoenix/SLC), legitimate apps. **Password spray, contained.**
+- **Delegates:** FullAccess = SELF only; SendAs = SELF only; SendOnBehalf = empty; forwarding = none.
+  **No one else can access the mailbox.**
+- **OAuth grants (4):** all legit — Apple Internet Accounts (iPhone/Mac Mail, holds `EWS.AccessAsUser.All`),
+  Alignable (Contacts.Read), ZeroTier, Tailscale. No rogue app.
+- **Inbox rules (46, all enabled):** Mike's own routing (forwarding vendor/notification mail to
+  howard@/rob@/wwilliams@/webmaster@/info@ + client seastman@glaztech.com). No malicious catch-all.
+  Two external-Gmail forwards to confirm as intentional: `energybilling@tesla.com -> ourfamily7479@gmail.com`,
+  `vailschools.org -> ursulaclark76@gmail.com`.
+- **MFA/CA:** "Require MFA for all users" = enabled/All; "Block legacy authentication" = enabled/All.
+  Posture is sound — spray can't convert a lucky guess.
+- **Auth methods:** Authenticator ×2 (Samsung), phone +1 520-289-1912, SSPR email superguru@gmail.com,
+  21 Windows Hello keys (18 stale/unnamed). (Confirm phone + superguru@ are Mike's.)
+
+## Actions taken
+1. **Pruned 20 stale Windows Hello keys** on mike@ (18 orphaned 2020–2025 + ACG-TECH-02L + ACG-YOGA,
+   both retired per Mike). **Only `GURU-5070` (2026-04-22) remains.** Other methods untouched.
+   (user-manager tier; 20× HTTP 204.)
+2. **Break-glass exclusions** — excluded `sysadmin@azcomputerguru.onmicrosoft.com`
+   (`8c363d17-02ed-45e1-97d0-49e5819e6ff6`, cloud-only GA) from BOTH MFA CA policies so an MFA
+   outage can't lock it out, while every real user stays gated:
+   - "Require multifactor authentication for all users" (`1e912cc2-334a-4f12-9d81-a95dc3822b8e`)
+   - "Require multifactor authentication for admins" (`5f10aa47-e796-40db-89eb-fdae60dea0c7`, 14 roles preserved)
+   (tenant-admin tier; both HTTP 204, verified.)
+3. **Break-glass sign-in alerting — LIVE.** New script
+   `.claude/skills/remediation-tool/scripts/breakglass-signin-alert.sh` posts `[SECURITY]` to
+   #dev-alerts on ANY sysadmin@ sign-in (state-tracked, seeded 2026-07-17T04:23Z, no history alerted).
+   Scheduled task **`ACG-BreakGlass-SignIn-Alert`** on GURU-5070, every 4h, test-run OK (result=0).
+   Baseline: sysadmin@ has **zero sign-ins in 2026** (break-glass untouched).
+
+## Blocked / pending (needs Mike)
+- **Rotate the sysadmin@ break-glass password — DO IN PORTAL.** Graph reset 403'd
+  ("Insufficient privileges") by design: resetting a Global Admin's password needs *Privileged
+  Authentication Administrator*, which our apps deliberately don't hold (granting it would be worse
+  than the gap). Mike (a GA) resets it in Entra: Users -> sysadmin@ -> Reset password; set a long
+  (20+ char) password, `force change = off`. Then **vault it** (intended path
+  `msp-tools/acg-m365-breakglass.sops.yaml`) — hand me the password and I'll store it.
+- Confirm phone `+1 520-289-1912` and SSPR email `superguru@gmail.com` are Mike's.
+- Confirm the two external-Gmail forward rules are intentional.
+
+## Recommendations (open)
+- Alert home: the task runs on GURU-5070 only while it's on — move to an always-on fleet host for
+  robust monitoring.
+- Consider a 2nd break-glass account (Microsoft-recommended pair).
+- No break-glass exclusion existed on ANY CA policy before today; now sysadmin@ is the designated one.
+
+## Fleet fix — Exchange REST tier (recurring friction, 6th time)
+Root-caused the perennial Exchange 401: the **Security Investigator** app (`investigator-exo`) has the
+Exchange Admin *role* but NOT the **`Exchange.ManageAsApp`** app-permission the adminapi requires, so it
+401s on every Exchange cmdlet. Only the **Exchange Operator** app (`exchange-op`) has both →
+**all Exchange REST (read + write) goes through `exchange-op`.** Corrected:
+- `SKILL.md` (tier table, breach-check default, "before calling" note — which had told us to assign the
+  role to the wrong SP)
+- `references/gotchas.md` (role table + authoritative callout + 401-vs-403 rule + SP->role map)
+- `scripts/user-breach-check.sh` (was minting `investigator-exo` -> mailbox delegate/forwarding checks
+  silently returned empty on every run; now `exchange-op`)
+- memory `reference_exchange_online_exchangeop_tier` + MEMORY.md index; errorlog `--correction`
+
+## Other client threads this session (separate contexts)
+Khalsa Montessori RMM onboarding (17 Windows enrolled via SC, client `LOWER-TIGER-5022`);
+BirthBiologic SharePoint permissions audit -> Syncro #32187 comment; Reliant Well Drilling Google
+Business Profile (none exists — GBP creation outlined). Logged/tracked in their own places.
--- a/errorlog.md
+++ b/errorlog.md
@@ -19,6 +19,10 @@ Categories (the `[type]` tag): _(none)_ = skill/command execution failure ·

 <!-- Append entries below this line -->

+2026-07-17 | GURU-5070 | remediation-tool/exchange | [correction] Repeatedly used investigator-exo tier for Exchange REST -> 401 on every adminapi call (Security Investigator app lacks Exchange.ManageAsApp). Correct tier is exchange-op (only app with both Exchange.ManageAsApp + Exchange Admin role). Mike's 6th time hitting this. Fixed docs (SKILL.md, gotchas.md) + user-breach-check.sh (was using investigator-exo -> mailbox delegate/forwarding checks silently returned empty on every breach check) + memory reference_exchange_online_exchangeop_tier. [ctx: tenant=azcomputerguru.com ref=reference_exchange_online_exchangeop_tier]
+
+2026-07-17 | GURU-5070 | vault/secret-echo | [friction] grep filter to redact vault output missed the JSON field 'private_key' (filter had 'key:' with colon + 'apikey', not bare 'private_key') -> SA private key printed to transcript. Fix: when displaying vault entries, allowlist known plaintext fields (name/status/username/notes/url) instead of denylisting secret patterns; or pipe through 'sops -d' with an explicit field select. [ctx: ref=vault-skill file=msp-tools/acg-msp-access-google-workspace.sops.yaml]
+
 2026-07-16 | GURU-BEAST-ROG | syncro/comment | [friction] POST /tickets/{id}/comment response failed jq parse even after tr control-char strip; comment DID post (verified via GET) - check-first rule prevented duplicate [ctx: ticket=32557]

 2026-07-15 | Howard-Home | msp360/backups | [correction] assumed LAB-SVR (Len's Auto) is a live machine needing attention; correct is LAB-SVR was RETIRED and replaced by LAB-SERVER — repeated correction from Howard