# Session Log: 2026-03-27 - Dataforth Security Incident, MFA Rollout, Test Datasheet Investigation

## Session Summary

Major incident response and security hardening session at Dataforth Corporation. Three concurrent workstreams:

1. **Security Incident** - Compromised workstation DF-JOEL2 via social engineering / ScreenConnect abuse
2. **MFA Rollout** - Deployed Conditional Access policies for M365 tenant
3. **Test Datasheet Pipeline** - Investigated broken datasheet export pipeline affecting customer shipments

---

## 1. Security Incident: DF-JOEL2 Compromise

### Timeline (March 27, 2026 MST)
- 08:25 - Joel Lohr clicked phishing link in personal Yahoo/Comcast email (appeared to be from Arizona Technology Council)
- 08:28 - ScreenConnect client installed from C:\Users\jlohr\Downloads\ScreenConnect.ClientSetup.msi
- 08:29 - "Angel Raya" connected via ScreenConnect cloud relay (instance-wlb9ga-relay.screenconnect.com)
- 08:29 - Attacker deployed two C2 backdoor ScreenConnect clients via PowerShell
- 08:31 - Attacker downloaded Sordum "Hide From Uninstall List" tool
- 08:32 - Tool used to hide rogue clients, "Angel Raya" disconnected
- 11:55 - "Administrator" connected via 80.76.49.18 C2 backdoor
- 12:40 - "Administrator" disconnected
- ~13:00 - ACG discovered compromise during ScreenConnect session
- 18:51 - Successful unauthorized M365 sign-in from Istanbul, Turkey

### Attacker Infrastructure
- C2 Server 1: 80.76.49.18:8040/8041
- C2 Server 2: 45.88.91.99:8040/8041
- ASN: AS399486, Virtuo (12651980 CANADA INC.), Montreal QC
- Abuse: abuses@virtuo.host, escalation: jordan@virtuo.host
- ScreenConnect Cloud: instance-wlb9ga-relay.screenconnect.com
- C2 client version: 25.2.4.9229 (binaries dated April 8, 2025)

### Rogue ScreenConnect Clients Found
1. 0cad93610010625f - "Angel Raya" initial access (instance-wlb9ga cloud relay)
2. 0dfe1abae029411c - C2 backdoor (80.76.49.18:8041)
3. a897d9a21259d116 - C2 backdoor (45.88.91.99:8041)
4. 1912bf3444b41a08 - LEGITIMATE (ACG, instance-kgc7jt)

### M365 Account Compromise
Sustained brute-force against jlohr@dataforth.com for 7+ days:
- Successful: Istanbul Turkey (91.93.232.236), Croydon UK (82.44.33.210), Germany (IPv6)
- Blocked attempts from: Frankfurt DE (45.86.202.x), Luxembourg, Virginia Beach, Sioux Falls, Camden DE, Charleston WV
- Tools used: Azure AD PowerShell, Azure CLI
- MFA: Password + phone only (520-917-2241), no conditional access

### Remediation Actions Completed
- [DONE] Both C2 IPs blocked at UDM firewall (iptables FORWARD + INPUT, all directions)
- [DONE] Three rogue ScreenConnect clients uninstalled via WinRM
- [DONE] HideUL tool deleted from C:\Users\Public\Pictures\Backup\
- [DONE] Downloaded MSIs cleaned
- [DONE] jlohr AD password reset to Dataforth2026! (force change at logon)
- [DONE] Entra Connect delta sync forced
- [DONE] M365 sessions revoked (twice)
- [DONE] Network-wide scan: 32 machines clean, 28 unreachable (offline)
- [DONE] UDM connection table scan: zero C2 traffic network-wide
- [DONE] No malicious inbox rules, mail forwarding, or OAuth consents
- [DONE] No rogue SSH keys on UDM

### Reports Filed
- FBI IC3: Submission ID 1c32ade367084be9acd548f23705736f (filed 3/27/2026 5:11 PM EST)
- Virtuo Hosting: abuses@virtuo.host - automated suspension of both IPs confirmed
- ConnectWise: Case #03464184 (abuse@connectwise.com)

### Incident Notes File
- PSA ticket notes: /tmp/dataforth-incident-psa-notes.txt
- Virtuo abuse report: clients/dataforth/docs/incident-2026-03-27-abuse-report-virtuo.md
- ConnectWise abuse report: clients/dataforth/docs/incident-2026-03-27-abuse-report-connectwise.md
- IC3 complaint PDF: clients/dataforth/docs/IC3-Complaint-2026-03-27.pdf

---

## 2. MFA Rollout

### Conditional Access Policies Deployed (Report-Only Mode)

| Policy | ID | State |
|--------|-----|-------|
| ACG - Require MFA for All Users | dc920ee4-22e6-402b-b5e3-4f3662d26227 | Report-only |
| ACG - Block Foreign Sign-Ins | 3405f7db-91b6-48da-b3fb-2e0ef1e44d17 | Report-only |
| ACG - Block Legacy Authentication | 82ebbe3b-d151-4cb7-aff7-af893a4915e3 | Report-only |

### Named Locations
- Dataforth Office - Tucson: 0a3e61d7-a544-4a47-961a-a98cd4804613 (67.206.163.122/32, trusted)
- Allowed Countries - US Only: 12706cec-c91b-454e-a24d-c801284b79f7

### Security Groups
- MFA-Excluded-BreakGlass: 75ac10ae-d49e-42b1-aa87-04908a983495
  - Members: Brian Faires, Dataforth Calibration, Dataforth Notifications, Endcap, Tablet 01
- MFA-Travel-Bypass: 094b12c5-b39a-4287-943a-f1175ce61a6f
  - For users traveling internationally, excludes them from foreign sign-in block

### MFA policy behavior
- From office (67.206.163.122): No MFA required
- From elsewhere in US: MFA required
- From outside US: Blocked (unless in MFA-Travel-Bypass group)
- Legacy auth (IMAP/POP/old PowerShell): Blocked everywhere

### Enforcement deadline: April 4, 2026
- MFA notice email sent to all 37 licensed users from sysadmin@dataforth.com via Graph API (draft-then-send method)
- Enrollment status email sent to mike@azcomputerguru.com and ghaubner@dataforth.com

### MFA Enrollment Status (19 ready, 19 need setup)
**Ready:** Ben Wadzinski, Bobbi Whitson, Dan Center, Georg Haubner (Authenticator), Jacque Antar, Jaime Becerra, Joel Lohr, John Lehman, Kellyn Wackerly, Kevin Wackerly, Lee Payne, Otto Fest (Windows Hello), Peter Iliya, Robert Koranek, Sandra Schock, Shipping, sysadmin (Authenticator), Theresa Dean, Yvonne Bejarano

**Need setup:** Andres Oliva-Martinez, Angel Lopez, Ayleen Montijo, Bernardo Laredo, Catalina Vanatta, Cesar Rivas, Chauncey Bell, Concepcion Hernandez, Cynthia Roedig, Elma Trujillo, Jason Yoder, Ken Hoffman, Linda Duarte, Logan Tobey, Lori Schlotterback, Manny Vargas, Martin Florez, Rosalinda Duarte, Thomas Nord

### Entra Cleanup
- 38 test station accounts (TS-*) deleted from Entra (were stale synced objects from CompanyUsers OU no longer in sync scope)
- bfaires@dataforth.com: AD account disabled, M365 still enabled, mailbox converted to shared (5,711 messages preserved)

### Dataforth M365 Licenses
- 50x M365 Business Premium (39 used) - includes Entra ID P1
- 19x Exchange Online Plan 1 (5 used)
- 5x SPB (4 used)

---

## 3. Test Datasheet Pipeline Investigation

### Background
Customer Quatronix (China) refusing shipments of 54+ modules without test datasheets. Originally 328 missing, whittled to 54 by Peter Iliya manually finding some.

### Pipeline Architecture
```
DOS Test Machine (TS-XX) -> QuickBASIC test program
  -> Generates H-prefix TXT file (H=17 decode: A=10,B=11...H=17,I=18,J=19)
  -> Writes to T:\ (mapped to \\D2TESTNAS\TEST)
  -> Syncs to AD2 C:\Shares\test\ (Sync-FromNAS task, every 15 min, WORKING)
  -> TestDataDB import.js ingests DAT files into SQLite database (WORKING)
  -> DFWDS.exe should process and move to WebShare (BROKEN - third party, Hoffman)
  -> TestDataSheetUploader should sync WebShare to website (BROKEN - not running since 2022)
  -> Customer downloads from www.dataforth.com
```

### Current State
- Sync-FromNAS: RUNNING (last ran 5:00 PM today, every 15 min)
- TestDataDB service: RUNNING (Windows service, auto-start, 2.27M records)
- TestDataDB API: http://192.168.0.6:3000 (Express.js, SQLite, better-sqlite3)
- DFWDS.exe: NOT RUNNING (VB6 program on AD2, third-party dev Hoffman unresponsive)
- TestDataSheetUploader: NOT RUNNING (last used Nov 2022, config points to Hoffman's local path)
- datasheet_exported_at: NULL for ALL records - export has never run

### Key Finding
Of the missing Quatronix serials checked, **22 out of 22 that responded are IN the database** with test data. Zero actually missing. The data exists but has never been exported as datasheet files.

### TestDataSheetUploader Details (found in Test Datasheets folder)
- VB.NET console app, .NET Framework 4.7.2
- Services: https://www.dataforth.com/Services/{Uploader,DirectoryManifest,DeleteFile}.aspx
- Auth: DataforthWebShare / Data6277
- Server-side path: C:\inetpub\wwwroot\dataforth.com\WebShare\ProcessCheckOutTestFolder\
- Config: C:\Users\hoffm\Documents\Customer Folders\Dataforth\product lists (Hoffman's old path)

### WebShare on AD2
- Share: \\AD2\webshare -> C:\Shares\webshare
- DFWDS config expects: X:\webshare\Test_Datasheets, X:\webshare\For_Web, X:\webshare\Bad_Datasheets
- DFWDS/Test Files/Original/Test_Datasheets: 2,457 existing datasheets

### H-prefix decode table
A=10, B=11, C=12, D=13, E=14, F=15, G=16, H=17, I=18, J=19
Example: H8601-6.TXT -> 178601-6.TXT

### AD1 Claude Instance Results (same day)
A Claude instance running on AD1/AD2 rebuilt the entire pipeline:
- **Spec Parser built** (parsers/spec-reader.js) - reads 8 binary spec DATs, 1470 models
- **Exact-match TXT formatter** (templates/datasheet-exact.js) - reverse-engineered from QB source
- **Auto-export system** (database/export-datasheets.js) - generates TXT to X:\For_Web after import
- **72 of 73 Quatronix datasheets generated** - in ComputerGuruConnectv2/Files/Quatronix-Datasheets.zip
- **1.4M records reconciled** with existing For_Web files, 500K archived to year subfolders
- **Root cause: CTONWTXT.BAT not called** in AUTOEXEC v4.1 since 2026-03-12. TXT piling up in C:\STAGE since Sept 2025.
- **New pipeline eliminates** CTONWTXT.BAT, DFWDS.exe, and TestDataSheetUploader
- **1 missing datasheet**: 177000-15 (SCM5B49-05) needs empty spec file from John Lehman
- **Website upload still broken**: old ASP.NET endpoints return 404, need replacement
- **7B series**: ~830K records need 7B-specific formatter (specs loaded but layout not implemented)
- All session files saved to clients/dataforth/session-logs/ (from ClaudeSession-2026-03-27.zip)

### Next Steps for Datasheets
- Website upload replacement (old endpoints dead, need new mechanism)
- Fix CTONWTXT.BAT in AUTOEXEC or accept server-side generation as sufficient
- 7B datasheet formatting
- SCM5B49 spec file from John
- New product lines: MAQ20/PWRM (XLS), 10D (JSON, ~May 2026)

---

## 4. Other Work

### UDM Firewall Access
- Generated SSH key: ~/.ssh/id_ed25519_udm
- Public key added to UDM root authorized_keys
- UDM IPs: 192.168.0.254 (also responds on 192.168.0.1, same MAC d0:21:f9:6c:11:02)

### Network Investigation (Internet/Phone Outage Report)
- All infrastructure reachable: UDM, AD1, AD2, D2TESTNAS
- Internet working from NAS (8.8.8.8, google.com)
- DNS working on all DCs and UDM
- ISP: fdtnet.net, Dataforth public IP: 67.206.163.122
- Issue was likely localized to specific machines or resolved itself

### AD1 Claude Instance README
- Created: clients/dataforth/dos-test-machines/Test Datasheets/README.md
- Contains Gitea credentials, network context, investigation steps for AD1 instance

---

## Credentials Used This Session

### Dataforth Network
- AD1 (192.168.0.27): INTRANET\sysadmin / Paper123!@#
- AD2 (192.168.0.6): INTRANET\sysadmin / Paper123!@#
- D2TESTNAS (192.168.0.9): root / SSH key (~/.ssh/id_ed25519 default)
- UDM (192.168.0.254): root / SSH key (~/.ssh/id_ed25519_udm)

### Dataforth M365
- Tenant ID: 7dfa3ce8-c496-4b51-ab8d-bd3dcd78b584
- Admin: sysadmin@dataforth.com / Paper123!@# (synced with AD)
- Entra App (Claude-Code-M365): App ID 7a8c0b2e-57fb-4d79-9b5a-4b88d21b1f29 / Secret tXo8Q~ZNG9zoBpbK9HwJTkzx.YEigZ9AynoSrca3

### MSP Multi-Tenant App (Claude-MSP-Access)
- MSP Tenant: ce61461e-81a0-4c84-bb4a-7b354a9a356d
- App ID: fabb3421-8b34-484b-bc17-e46de9703418
- Client Secret: ~QJ8Q~NyQSs4OcGqHZyPrA2CVnq9KBfKiimntbMO
- Permissions granted for Dataforth tenant: CA policies, mail, users, auth methods, security events

### TestDataSheetUploader Web Services
- URL: https://www.dataforth.com/Services/{Uploader,DirectoryManifest,DeleteFile}.aspx
- Auth: DataforthWebShare / Data6277

### Rsync (Sync-FromNAS on AD2)
- NAS rsync user: rsync / IQ203s32119
- Module: test (maps to /data/test on NAS)

### Gitea
- URL: https://git.azcomputerguru.com
- User: mike@azcomputerguru.com
- Password: Gptf*77ttb123!@#-git

### WinRM Python Environment
- /tmp/winrm-env/bin/python3 (pywinrm, msal, requests installed)

---

## Outstanding / Incomplete

### Security
- [ ] 28 machines unreachable during ScreenConnect scan - check when online
- [ ] Joel should reset Yahoo/Comcast personal email password
- [ ] Check Chrome saved passwords on DF-JOEL2 (may have been exported by attacker)
- [ ] Fix Windows Defender on DF-JOEL2 (error 0x800106ba, McAfee conflict)
- [ ] Add C2 IPs to UDM permanent block list via UniFi UI (iptables rules don't survive reboot)
- [ ] Monitor ConnectWise case 03464184 for Angel Raya identity
- [ ] Consider disabling jlohr account entirely (retiring March 31)
- [ ] Original phishing email never found (sent to personal Yahoo, Joel can't locate it)

### MFA
- [ ] April 4: Switch all 3 CA policies from report-only to enabled
- [ ] 19 users still need to register MFA
- [ ] Delete orphan scheduled tasks on AD2: TestDataDB Server, TestDataDB_NodeServer
- [ ] Clean up remaining unlicensed enabled accounts in Entra (former employees)

### Test Datasheets
- [DONE] Pipeline rebuilt by AD1 instance -- server-side TXT generation from DAT data
- [DONE] 72 of 73 Quatronix datasheets generated (in ComputerGuruConnectv2/Files/Quatronix-Datasheets.zip)
- [ ] 1 remaining: 177000-15 (SCM5B49-05) needs empty spec file from John Lehman
- [ ] Website upload replacement -- old ASP.NET endpoints return 404
- [ ] 7B datasheet formatting (~830K records, specs loaded but formatter needs 7B layout)
- [ ] Fix CTONWTXT.BAT in AUTOEXEC or accept server-side generation as sufficient
- [ ] New product lines: MAQ20/PWRM (XLS), 10D (JSON, ~May 2026)
- [ ] Service permissions cleanup (runs as SYSTEM)

---

## Update: 17:45 - AD1 Session Import and Final Context Save

### AD1 Claude Instance Session (same day)
Imported full session from /home/guru/ComputerGuruConnectv2/Files/ClaudeSession-2026-03-27.zip. The AD1 instance running on AD2 rebuilt the entire test datasheet pipeline:

**Key accomplishments:**
- Built spec parser (parsers/spec-reader.js) - reads 8 binary spec DATs, 1470 models across 6 families
- Built exact-match TXT formatter (templates/datasheet-exact.js) - reverse-engineered from QuickBASIC source
- Built auto-export system (database/export-datasheets.js) - generates TXT to X:\For_Web after import
- Added PDF on-demand endpoint (/api/datasheet/:id/pdf)
- Generated 72 of 73 Quatronix datasheets
- Reconciled 1.4M records with existing For_Web files
- Archived 500K pre-2026 files into year subfolders
- Found root cause: CTONWTXT.BAT not called in AUTOEXEC v4.1 since 2026-03-12

**AD1 session files saved to:**
- clients/dataforth/session-logs/SESSION-SUMMARY.md
- clients/dataforth/session-logs/project_pipeline_rebuilt.md
- clients/dataforth/session-logs/project_test_datasheet_pipeline.md
- clients/dataforth/session-logs/project_ctonwtxt_gap.md
- clients/dataforth/session-logs/project_ad2_context.md
- clients/dataforth/session-logs/project_engr_share_exploration.md
- clients/dataforth/session-logs/project_new_product_lines.md
- clients/dataforth/session-logs/reference_haubner_backup.md
- clients/dataforth/session-logs/feedback_sn_from_data.md
- clients/dataforth/session-logs/email-draft-updated.md
- clients/dataforth/session-logs/email-draft-morning.md
- clients/dataforth/session-logs/user_mike_swanson.md
- clients/dataforth/session-logs/MEMORY.md (AD1 instance memory index)

### Memory Entries Created
- .claude/memory/project_datasheet_pipeline.md - Full pipeline architecture and status
- .claude/memory/project_dataforth_incident_2026-03-27.md - Security incident and MFA summary
- MEMORY.md index updated with both entries

### MFA Note
- SMS/Voice calling acceptable as MFA method for users who push back on Authenticator app
- CA policy requires MFA but doesn't mandate specific method - phone registration is sufficient
- MFA-Travel-Bypass group (094b12c5) created for users traveling internationally

### Emails Sent via Graph API (draft-then-send from sysadmin@dataforth.com)
1. MFA notice to all 37 licensed users (deadline April 4, 2026)
2. Enrollment status report to mike@azcomputerguru.com and ghaubner@dataforth.com

### Gitea
- 502 error on push - Gitea container may need restart on Jupiter
- All changes committed locally, will push when available

---

## Files Created/Modified This Session
- clients/dataforth/dos-test-machines/Test Datasheets/README.md (AD1 Claude instance context)
- clients/dataforth/docs/incident-2026-03-27-abuse-report-virtuo.md
- clients/dataforth/docs/incident-2026-03-27-abuse-report-connectwise.md
- clients/dataforth/docs/IC3-Complaint-2026-03-27.pdf
- .claude/memory/project_datasheet_pipeline.md
- .claude/memory/project_dataforth_incident_2026-03-27.md
- .claude/memory/MEMORY.md (updated index)
- clients/dataforth/session-logs/* (13 AD1 session files imported)
- /tmp/dataforth-incident-psa-notes.txt (PSA ticket notes)
- /tmp/dataforth-mfa-notice.txt (MFA user communication)
- /tmp/ic3-technical-details.txt (IC3 technical details)
- ~/.ssh/id_ed25519_udm (SSH key for UDM access)
- /tmp/winrm-env/ (Python venv with pywinrm, msal, requests)
