From 79789a8815d507b6bf5f34310f7e2191c4e3888b Mon Sep 17 00:00:00 2001 From: Mike Swanson Date: Fri, 26 Jun 2026 04:16:30 -0700 Subject: [PATCH] sync: auto-sync from GURU-5070 at 2026-06-26 04:15:16 Author: Mike Swanson Machine: GURU-5070 Timestamp: 2026-06-26 04:15:16 --- .../memory/reference_tedards_tenant_facts.md | 4 +-- .findfolder.py | 28 ++++++++++++++++ ...6-06-25-mike-bt-delete-folder-and-dedup.md | 33 +++++++++++++++++++ 3 files changed, 63 insertions(+), 2 deletions(-) create mode 100644 .findfolder.py diff --git a/.claude/memory/reference_tedards_tenant_facts.md b/.claude/memory/reference_tedards_tenant_facts.md index ed2385b3..efb43c0b 100644 --- a/.claude/memory/reference_tedards_tenant_facts.md +++ b/.claude/memory/reference_tedards_tenant_facts.md @@ -9,8 +9,8 @@ metadata: Mailboxes: `bt@tedards.net` (Bill, owner), `y226@tedards.net` (Yvonne). Bill files mail by legal matter number into deep Inbox subfolders (e.g. "8445 BOLTON [Farmers TX]", "BOLTON, Lindsay"); a top-level "DUPLICATE need to check" folder (~11,864 items) is junk from a **botched mail import years ago** — ignore it. -Security gaps found 2026-06-25: tenant was **dehydrated** (never customized) — ran `Enable-OrganizationCustomization` (irreversible, one-time) then `Set-AdminAuditLogConfig -UnifiedAuditLogIngestionEnabled $true` (both HTTP 200). Read-back still showed `false` immediately after — propagation lag; verify later that it flipped to true and that Search-UnifiedAuditLog starts returning data (ingestion lag up to ~60min). Before this, UAL was OFF so there was no queryable audit trail for the bt@ deletions. bt@ mailbox syncs to **9 devices** (5 aging iOS Mail/EAS, a Mac Outlook, 2 Outlook-for-iOS added 2026-06-25). Per-mailbox AuditEnabled=true but not queryable since Search-MailboxAuditLog is deprecated + UAL ingestion off. +Security gaps found 2026-06-25: tenant was **dehydrated** (never customized) — ran `Enable-OrganizationCustomization` (irreversible, one-time) then `Set-AdminAuditLogConfig -UnifiedAuditLogIngestionEnabled $true` (both HTTP 200). Flag later confirmed `true`, BUT app-only `Search-UnifiedAuditLog` (via exchange-op InvokeCommand) returns **zero** records for this tenant even hours after ingestion — proven against ~11,800 known dedup MoveToDeletedItems events AND a tenant-wide `RecordType=ExchangeItem` query (both 0). Conclusion: **app-only UAL cannot read mailbox-item records here** — do NOT rely on it for mailbox-item attribution; use device-statistics sync-time + EWS bait timing instead (how the bt@ culprit was found). Before enabling, UAL was OFF entirely. bt@ mailbox syncs to **9 devices** (5 aging iOS Mail/EAS, a Mac Outlook, 2 Outlook-for-iOS added 2026-06-25). Per-mailbox AuditEnabled=true but not queryable since Search-MailboxAuditLog is deprecated + UAL ingestion off. EXO access: use the `exchange-op` tier, not `investigator-exo` — see [[reference_investigator_exo_manageasapp_gap]]. Ongoing matter: Wirechunk/agencyzoomify.com DMARC + the bt@ "delete folder" deletions (ticket #5070, #32228). -**bt@ "delete folder" mystery — SOLVED 2026-06-26 (root cause = client-side device auto-delete).** Lindsay's (lindsay@agencyzoomify.com) Bolton-thread mail auto-moves to Deleted Items. Proven via bait test: restored the 3 msgs to Inbox via EWS, all 3 were re-deleted to Deleted Items at the identical instant (02:54:24Z) — automated, not human. Eliminated: inbox rules (incl. hidden), sweep rules, transport rules, forwarding, delegates, folder perms, and any OAuth app with Mail.ReadWrite (none exist; admin apps only have MailboxSettings.ReadWrite/Mail.Send/Exchange.Manage). Only mail-MOVE capability present = Apple native iOS Mail (appId 32f67a9b, EAS+EWS) + Outlook-for-iOS. 5 Apple/Outlook-iOS devices push-synced within 8s of the re-delete; all activity from Bill's single home IP 69.242.239.94 (NOT a compromise). Bisection test (2026-06-26 03:23): removed/disabled Outlook-for-iOS, re-moved msgs to Inbox — re-deleted again in 4s, AND the re-delete (03:23:31Z) fired BEFORE the Outlook-iOS clients re-synced (03:23:38/48) → **Outlook-iOS EXONERATED; culprit = a NATIVE iOS Mail (EAS) device** with on-device "Block Sender → Move to Trash" for lindsay@agencyzoomify.com. Narrowed to the two devices syncing at the delete instant: **iPhone16C2** and **iPad15C8** (Bill's current iPhone + iPad). On-device block list is NOT server-readable — Bill must remove Lindsay from Blocked on those devices (iOS Settings → Mail → Blocked). Note: Set-CASMailbox OutlookMobileEnabled has heavy propagation lag (didn't enforce during the test window); same lag seen on Set-AdminAuditLogConfig. To pin the exact one device, block one EAS DeviceId and re-bait. Removed 2 Outlook-iOS partnerships (they auto-re-add) + toggled OutlookMobileEnabled (reverted to true, queued). +**bt@ "delete folder" mystery — SOLVED 2026-06-26 (root cause = client-side device auto-delete).** Lindsay's (lindsay@agencyzoomify.com) Bolton-thread mail auto-moves to Deleted Items. Proven via bait test: restored the 3 msgs to Inbox via EWS, all 3 were re-deleted to Deleted Items at the identical instant (02:54:24Z) — automated, not human. Eliminated: inbox rules (incl. hidden), sweep rules, transport rules, forwarding, delegates, folder perms, and any OAuth app with Mail.ReadWrite (none exist; admin apps only have MailboxSettings.ReadWrite/Mail.Send/Exchange.Manage). Only mail-MOVE capability present = Apple native iOS Mail (appId 32f67a9b, EAS+EWS) + Outlook-for-iOS. 5 Apple/Outlook-iOS devices push-synced within 8s of the re-delete; all activity from Bill's single home IP 69.242.239.94 (NOT a compromise). Bisection test (2026-06-26 03:23): removed/disabled Outlook-for-iOS, re-moved msgs to Inbox — re-deleted again in 4s, AND the re-delete (03:23:31Z) fired BEFORE the Outlook-iOS clients re-synced (03:23:38/48) → **Outlook-iOS EXONERATED; culprit = a NATIVE iOS Mail (EAS) device** with on-device "Block Sender → Move to Trash" for lindsay@agencyzoomify.com. Narrowed to the two devices syncing at the delete instant: **iPhone16C2** and **iPad15C8** (Bill's current iPhone + iPad). On-device block list is NOT server-readable — Bill must remove Lindsay from Blocked on those devices (iOS Settings → Mail → Blocked). Note: Set-CASMailbox OutlookMobileEnabled has heavy propagation lag (didn't enforce during the test window); same lag seen on Set-AdminAuditLogConfig. To pin the exact one device, block one EAS DeviceId and re-bait. Removed 2 Outlook-iOS partnerships (they auto-re-add) + toggled OutlookMobileEnabled (reverted to true, queued). **CONFIRMED 2026-06-26 by Yvonne: found Bolton's blocked contact on Bill's NEW iPad (= iPad15C8); unblocked on phone + new iPad, checking other iPads — validates the device-block root cause.** diff --git a/.findfolder.py b/.findfolder.py new file mode 100644 index 00000000..d8680f70 --- /dev/null +++ b/.findfolder.py @@ -0,0 +1,28 @@ +import os, json, urllib.request, urllib.error, time +TOKEN=os.environ["GT"]; USER="bt@tedards.net"; BASE="https://graph.microsoft.com/v1.0" +def g(url): + for _ in range(6): + req=urllib.request.Request(url, headers={"Authorization":"Bearer "+TOKEN}) + try: + with urllib.request.urlopen(req,timeout=60) as r: return json.loads(r.read()) + except urllib.error.HTTPError as e: + if e.code in (429,503,504,500): time.sleep(int(e.headers.get("Retry-After","8"))); continue + raise + raise RuntimeError(url) +# BFS all folders +hits=[]; names={} +def walk(url, path): + while url: + d=g(url) + for f in d["value"]: + nm=f.get("displayName","?"); fid=f["id"]; tot=f.get("totalItemCount",0); ch=f.get("childFolderCount",0) + full=path+"/"+nm + names[fid]=full + low=nm.lower() + if "9000" in low or "duplicat" in low or "06-26" in low or "06/26" in low: + hits.append((full, tot, ch, fid)) + if ch>0: + walk(BASE+"/users/%s/mailFolders/%s/childFolders?$top=100&$select=id,displayName,totalItemCount,childFolderCount"%(USER,fid), full) + url=d.get("@odata.nextLink") +walk(BASE+"/users/%s/mailFolders?$top=100&$select=id,displayName,totalItemCount,childFolderCount"%USER, "") +print(json.dumps([{"path":h[0],"total":h[1],"children":h[2],"id":h[3]} for h in hits], indent=2)) diff --git a/clients/tedards/session-logs/2026-06/2026-06-25-mike-bt-delete-folder-and-dedup.md b/clients/tedards/session-logs/2026-06/2026-06-25-mike-bt-delete-folder-and-dedup.md index 8f9fa018..8b1f5a06 100644 --- a/clients/tedards/session-logs/2026-06/2026-06-25-mike-bt-delete-folder-and-dedup.md +++ b/clients/tedards/session-logs/2026-06/2026-06-25-mike-bt-delete-folder-and-dedup.md @@ -122,3 +122,36 @@ Search-UnifiedAuditLog (bt@, Move/Delete ops) -> entries=0 (still propagating) - **Cron job:** `ce6e3e74` (durable, ~every 19 min) — UAL audit recheck - **Prior session:** `clients/tedards/session-logs/2026-06/2026-06-25-discord-bot-agencyzoomify-dmarc-fix.md` - **Memory:** `reference_tedards_tenant_facts`, `reference_investigator_exo_manageasapp_gap`, `feedback_exchange_op_all_access` + +--- + +## Update: 04:14 PT (2026-06-26) — dedup completion, ticket, root-cause confirmation, second folder + +### Summary +Completed the first dedup: "DUPLICATE need to check" went 11,864 -> 54 uniques; 11,813 duplicates soft-deleted to Deleted Items (recoverable). The background driver was launched twice (the first `nohup &` instance never died), so two idempotent instances ran concurrently and converged — harmless. Posted the approved no-billing notification ticket **#32467** (id 113090881) to Yvonne once the folder reached 54, with a customer-visible emailed comment listing the result and her action items (file the 54, empty Deleted Items when satisfied). + +Closed the audit-capture effort as non-viable: app-only `Search-UnifiedAuditLog` returned 0 across ~8 rechecks even after ~11,800 dedup MoveToDeletedItems events and a tenant-wide `RecordType=ExchangeItem` query (also 0). Conclusion: app-only UAL cannot read mailbox-item records for this tenant/SP. Stopped cron `ce6e3e74` (it had actually persisted only as session-only despite the durable flag). Attribution rests on device-statistics timing + the bisection. + +Yvonne replied confirming the root cause: she found Bolton's blocked contact on **Bill's new iPad** (matches iPad15C8), unblocked it on the phone + new iPad, and will check the other iPads. This validates the device-block diagnosis and the bisection method. + +Yvonne requested a second cleanup: `/Inbox/9000 DUPLICATES - 06-26-2026` (she said it was under "8888 Client Dev" and had 3 subfolders; actually directly under Inbox with 6 loose items + 4 subfolders). Ran the generalized scan over the whole subtree (614 items): 609 true duplicates, 5 uniques. With user approval ("deal with all of them - keep all unique items"), soft-deleted the 609 to Deleted Items (0 errors), kept the 5 uniques (1 in 8376 Baines, 2 in 8390 DUPLICATES (2), 2 in 8395 Hixon). Posted a fresh emailed comment (420851846) to #32467 summarizing the second cleanup and acknowledging the device fix. + +### Configuration Changes (delta) +- bt@ "DUPLICATE need to check": 11,864 -> 54 (11,813 moved to Deleted Items). +- bt@ "9000 DUPLICATES - 06-26-2026" subtree (4 subfolders + loose): 614 -> 5 (609 moved to Deleted Items). +- Syncro #32467 created (Waiting on Customer, no billing); comments 420827280 (first cleanup) + 420851846 (second cleanup), both emailed to Yvonne. +- Cron `ce6e3e74` deleted (UAL probe non-viable). +- Memory `reference_tedards_tenant_facts` updated: UAL app-only limitation + customer-confirmed root cause. +- New scan tooling (gitignored scratch, `.dup*`): `.dupscan.py`, `.dupscan2.py`, `.dupdelete.py`, `.dupdrive.sh`, `.findfolder.py`. + +### Totals +- Duplicates removed across both folders: **12,419** (11,810 + 609). Unique items preserved: **59** (54 + 5). All soft-deletes recoverable in Deleted Items. + +### Pending +- Yvonne: unblock Lindsay on the remaining iPads; file the 59 uniques; empty Deleted Items when satisfied. +- bt@ delete-folder fix awaiting confirmation once all devices are cleared. +- Other duplicates folders exist (e.g. `/Inbox/ZZZZZZZZZZZZZZZ CLIENTS/8376 CCD-Maine [Baines]/8376 DUPLICATES`, 422 items) — not requested; offer if relevant. + +### Reference (delta) +- **Syncro:** #32467 (id 113090881) — https://computerguru.syncromsp.com/tickets/113090881 +- Target folder: `/Inbox/9000 DUPLICATES - 06-26-2026` (id `…AAABGP7iAAA=`)