claudetools/wiki/clients/dataforth.md

---
type: client
name: dataforth
display_name: Dataforth Corporation
last_compiled: 2026-06-02
compiled_by: DESKTOP-0O8A1RL/claude-main
sources:
  - clients/dataforth/docs/overview.md
  - clients/dataforth/docs/active-directory.md
  - clients/dataforth/docs/workstations.md
  - clients/dataforth/docs/manufacturing.md
  - clients/dataforth/docs/billing-log.md
  - clients/dataforth/docs/SYNC_SCRIPT_UPDATE_SUMMARY.md
  - clients/dataforth/docs/incident-2026-03-27-abuse-report-virtuo.md
  - clients/dataforth/docs/incident-2026-03-27-abuse-report-connectwise.md
  - clients/dataforth/session-logs/2026-03-23-galactic-advisors-report.md
  - clients/dataforth/session-logs/2026-03-27-security-incident-mfa-datasheets.md
  - clients/dataforth/session-logs/SESSION-SUMMARY.md
  - clients/dataforth/session-logs/MEMORY.md
  - clients/dataforth/session-logs/2026-04-12-session.md
  - clients/dataforth/session-logs/2026-04-13-session.md
  - clients/dataforth/session-logs/2026-04-14-session.md
  - clients/dataforth/session-logs/2026-04-23-session.md
  - clients/dataforth/session-logs/2026-05-03-session.md
  - clients/dataforth/session-logs/2026-05-04-lobby-phone-vlan-fix.md
  - clients/dataforth/session-logs/2026-05-06-session.md
  - clients/dataforth/session-logs/2026-05-12-session.md
  - clients/dataforth/session-logs/project_ad2_context.md
  - clients/dataforth/session-logs/project_pipeline_rebuilt.md
  - clients/dataforth/session-logs/project_test_datasheet_pipeline.md
  - clients/dataforth/session-logs/project_new_product_lines.md
  - projects/dataforth-dos/CONTEXT.md
  - .claude/memory/project_dataforth_incident_2026-03-27.md
  - .claude/memory/project_datasheet_pipeline.md
  - .claude/memory/project_neptune_sbr_email_routing.md
  - .claude/memory/reference_dataforth_contact.md
  - .claude/memory/reference_neptune_access_d2testnas.md
  - .claude/memory/feedback_d2testnas_ssh.md
  - .claude/memory/infra_office_network.md
  - clients/dataforth/session-logs/2026-06-01-aoi-xp-vlan-share.md
  - clients/dataforth/docs/aoi-xp-vlan-backup-runbook.md
  - clients/dataforth/session-logs/2026-06-01-cbell-m365-bobbi-outlook.md
  - clients/dataforth/session-logs/2026-06-02-session.md
backlinks:
  - projects/dataforth-dos
  - systems/jupiter
---

# Dataforth Corporation

Signal conditioning / data acquisition manufacturer in Tucson, AZ. Long-standing ACG client. Active managed relationship — monthly prepaid block. Notable for 64 MS-DOS 6.22 test stations, a major security incident in March 2026, and an ongoing test datasheet pipeline modernization project.

---

## Profile

- **Contract type:** Prepaid hour block (monthly replenishment invoice $2,098.87)
- **Key contacts:**

| Name | Username | Role | Email |
|---|---|---|---|
| Dan Center | dcenter | Operations (primary IT contact) | dcenter@dataforth.com |
| John Lehman | jlehman | Engineering, QB code, test specs | jlehman@dataforth.com |
| Peter Iliya | pIliya | Applications Engineer | pIliya@dataforth.com |
| Georg Haubner | ghaubner | Engineering; D: drive on HGHAUBNER has pre-ransomware-attack backup | ghaubner@dataforth.com |
| Kevin Wackerly | kwackerly | IT/Admin, handles calibration@ account | kwackerly@dataforth.com |
| Logan Tobey | ltobey | Support/Sales | ltobey@dataforth.com |
| Ben Wadzinski | bwadzinski | Engineering | — |
| Lee Payne | lpayne | Engineering | — |
| Theresa Dean | tdean | Admin | tdean@dataforth.com |
| Joel Lohr | jlohr | **RETIRED 2026-03-31** — account intentionally kept enabled; inbox rule forwards ntirety.com notifications to mike@azcomputerguru.com | jlohr@dataforth.com |
| Ken Hoffman | khoffman / oemdata | TestDataSheetUploader author, external; also owns Dataforth product API | — |
| Winter | — | Dataforth contact who requested Syncro asset cleanup 2026-06-02 | — |

- **External distributor:** Ginger (gy@quatronix-cn.com) — Quatronix China; receives datasheets
- **Billing rate:** Prepaid block; all invoices show $0.00 — hours drawn from block
- **Hours remaining:** 46.5 hrs as of 2026-05-03 (after 1 hr billed that session). Always live-check Syncro before billing — `GET /customers/578095`.
- **Syncro customer ID:** 578095
- **Invoice CC:** jantar@dataforth.com

---

## Infrastructure

### Servers & Services

| Host | IP | Role | OS | Notes |
|---|---|---|---|---|
| AD1 | 192.168.0.27 | Primary DC, DNS, FSMO roles, Engineering share | Windows Server 2016 | C:\ at **90%** capacity (C:\Engineering = 787 GB) — critical risk. FSMO roles (assumed all). |
| AD2 | 192.168.0.6 | Secondary DC, TestDataDB service host, NAS mirror, WebShare | Windows Server 2022 | Hosts testdatadb Node.js service on :3000. Wiped by crypto attack 2025 — rebuilt. Windows Firewall disabled (all profiles). |
| FILES-D1 | — | File server | — | Sales docs (W:), archive (Y:) |
| SAGE-SQL | 192.168.0.153 | Sage ERP (S:), RDS Session Host/Connection Broker/Web Access | Windows Server | RDS licensing grace period was expired (reset 2026-05-06). TSGateway disabled (server not externally exposed). New self-signed RDS cert installed. Bitdefender GravityZone managed AV. |
| 3CX | 192.168.0.125 | Phone system | — | Last logon Oct 2025 — possibly inactive |
| DF-HYPERV-B | — | Hyper-V hypervisor | — | — |
| D2TESTNAS | 192.168.0.9 | SMB1 bridge for DOS test stations + AOI XP backup; Neptune Exchange physically colocated | Debian 13 (trixie), Samba 4.22.6 | **Repurposed Netgear ReadyNAS** (earlier "CachyOS"/"Netgear ReadyNAS" records were stale). SMB1 enabled globally (CORE..SMB3, NTLMv1) — required for DOS 6.22 stations. rsync daemon on port 873 (module `test`, user `rsync`, hosts allow 192.168.0.0/24 + 172.16.0.0/12). SSH: `root@192.168.0.9`. Tailscale route for 172.16.0.0/22. **Shares:** `test`/`datasheets`/`snapshots` (guest; now `hosts deny 192.168.1.175`), `aoibackup` (XP-only — see Access). |
| ESXi hosts | 192.168.0.122, 192.168.0.124 | VMware ESXi hypervisors | ESXi | — |
| UDM Firewall | 192.168.0.254 | Perimeter firewall/router | UniFi OS | MAC d0:21:f9:6c:11:02. Also responds on 192.168.0.1. SSH key: `~/.ssh/id_ed25519_udm`. C2 IPs blocked via iptables (NOT permanent — need to add to UniFi UI). |
| PBX (3CX/Sangoma) | 192.168.100.2 (also .196) | VoIP PBX — production phones on 192.168.100.0/24 | — | TFTP provisioning for Cisco SPA502G phones. Access via SSH: `sangoma@192.168.100.2`. Vault: `clients/dataforth/pbx.sops.yaml` |

**Neptune Exchange (ACG infrastructure, physically at Dataforth D2):**
- `neptune.acghosting.com` | internal `172.16.3.11` | external inbound `67.206.163.124` / outbound `67.206.163.122`
- Exchange Server 2016, active ACG-hosted mail server for multiple clients
- Physically colocated at Dataforth's D2 facility — NOT on ACG office LAN despite 172.16.x.x IP
- Access requires routing through D2TESTNAS (192.168.0.9): Dataforth UDM has a 172.16.x.x subnet that overlaps ACG office LAN, making direct routing ambiguous
- SNAT rule on Dataforth UDM at `/data/on_boot.d/10-neptune-snat.sh` should force Neptune outbound to use `.124` (not always active — verify)
- Vault: `clients/dataforth/neptune-exchange.sops.yaml`
- [WARNING] TODO: Resubnet Dataforth UDM to a non-overlapping range to permanently fix Neptune routing

### Workstations (summary)

| Category | Count | OS | Notable |
|---|---|---|---|
| Engineering | ~12 | Win 10/11 Pro | HGHAUBNER (192.168.0.148) has pre-attack D: backup. D1-PWRM for PWRM10 test. |
| Manufacturing/Assembly | ~14 | Win 10/11 Pro | AS24, AS26 + various assembly/hi-pot stations |
| Office/Admin | ~12 | Win 10/11 Pro | DF-GAGETRAK (192.168.0.102) — GAGEtrak calibration host. DF-JOEL2 (192.168.0.174) — compromised 2026-03-27, remediated. |
| End-of-Life (Win 7) | 3 | Windows 7 Pro | LABELPC (192.168.0.100), LABELPC2 (192.168.0.98), D2-RCVG-003 (192.168.0.47) — EOL, on network |
| AOI Optical Inspection (XP) | 1 | Windows XP | WinXPBE-724667 @ **192.168.1.175** on VLAN 2 (mydata/SMT). Holds the AOI machine's external drive; backs up to `\\192.168.0.9\aoibackup` (SMB1, XP-only). EOL. See AOI runbook + 2026-06-01 session log. |
| DOS Test Stations | 64 | MS-DOS 6.22 | TS-1 through TS-30 + variants. Not domain-joined. SMB1 via D2TESTNAS. |

### Email & Identity

- **M365 tenant:** dataforth.com | Tenant ID: `7dfa3ce8-c496-4b51-ab8d-bd3dcd78b584`
- **Entra ID Sync:** Yes — Azure AD Connect. Synced OUs include **OU=SyncedUsers** and **OU=Azure_Users** (cbell confirmed in OU=Azure_Users and syncing, 2026-06-01) — the earlier "SyncedUsers only" note was incomplete.
- **M365 licenses:** 50x Business Premium (39 used), 19x Exchange Online Plan 1 (5 used), 5x SPB (4 used)
- **SMTP settings:** smtp.office365.com, port 587, STARTTLS — use `sysadmin@dataforth.com`
- **SMTP AUTH status:** Tenant-level not disabled; per-mailbox varies. `calibration@dataforth.com` had SmtpClientAuthentication=true re-enabled 2026-04-23. `sysadmin@dataforth.com` SMTP AUTH is blocked by Exchange Online default — testdatadb uses Graph API for email (Mail.Send permission granted to Claude-Code-M365 app 2026-05-12).
- **DKIM:** Both selector1 and selector2 published. Rotated 2026-05-12; cutover to selector2 on 2026-05-16.
  - `selector1._domainkey.dataforth.com` → selector1-dataforth-com._domainkey.dataforthcom.onmicrosoft.com
  - `selector2._domainkey.dataforth.com` → selector2-dataforth-com._domainkey.dataforthcom.onmicrosoft.com
- **DNS Host:** ntirety.com — Dataforth's public DNS zone managed through ntirety's portal (not a standard registrar). DNS change requests go to ntirety, not a domain control panel. Joel Lohr's account retained to receive ntirety.com infrastructure notifications (inbox rule → mike@azcomputerguru.com).
- **INKY PhishFence:** Active transport rule `B859327F-3FBD-4BE7-A47A-97D02F1558A7` fires first and calls StopProcessingRules=true — blocks all subsequent custom transport rules. Use inbox rules for per-user mail routing.
- **MFA:** 3 Conditional Access policies created 2026-03-27 (initially report-only; enforced 2026-04-04):
  - "ACG - Require MFA for All Users" — skip from office IP 67.206.163.122
  - "ACG - Block Foreign Sign-Ins" — US-only; MFA-Travel-Bypass group for exceptions
  - "ACG - Block Legacy Authentication"
- **Named locations:** Dataforth Office - Tucson (67.206.163.122/32, trusted), Allowed Countries - US Only
- **MFA-Excluded-BreakGlass group:** Brian Faires, Dataforth Calibration, Dataforth Notifications, Endcap, Tablet 01
- **MFA enrollment (as of 2026-03-27):** 19/38 ready, 19 needed setup — deadline April 4, 2026

### Network

- **Domain:** intranet.dataforth.com | Forest/Domain Level: Windows Server 2016
- **ISP:** fdtnet.net | Public IP: 67.206.163.122 (outbound), 67.206.163.124 (Neptune inbound)
- **Firewall/Router:** UniFi Dream Machine at 192.168.0.254 (also 192.168.0.1)
- **Network:** Flat (no VLANs on main LAN — 192.168.0.0/24). Voice/PBX VLAN: 192.168.100.0/24 — production phones live here. **VLAN 2 "mydata" (192.168.1.0/24)** = SMT production-line network (gateway 192.168.1.1); members on the *D2-SMT Switch* (USW Enterprise 8) + *D2-Breakroom* port 12. Supersedes the earlier note that 192.168.1.0/24 was an unused UDM default voice VLAN — it is in active use by SMT. Inter-VLAN routing from mydata → main LAN is currently OPEN.
  - **mydata members (2026-06-01):** WinXPBE-724667 (AOI XP, .175), goldstar19, DESKTOP-FT0T4MK, My9-PC, + 3 unnamed industrial/SMT devices (MAC 00:90:fb:80:f0:c6, 00:80:79:05:23:f2, 00:80:79:04:47:e7).
- **VPN:** FortiClient required for remote access to 192.168.0.x. VPN can drop mid-session — save work frequently.
- **Drive mappings (GPO):** B: (\\ad1\itsvc), Q: (\\ad2\c-drive), S: (\\SAGE-SQL\sage), T: (\\ad2\e-drive), W: (\\files-d1\sales), X: (\\ad2\webshare), Y: (\\files-d1\archive). DOS test stations: T: (\\D2TESTNAS\test), X: (\\D2TESTNAS\datasheets)

### GuruRMM Enrollment

- **Site name:** Dataforth D1 | Site ID: `3a2f6866-26cd-452c-9806-a8df21475c3c`
- **Site API key:** vault `clients/dataforth/...` [check vault for current entry]
- **DF-GAGETRAK enrolled:** Agent ID `7626d82c-0736-47a6-8bc6-68e39859caed`, device ID `win-901ce38b-fb6e-44b8-a577-7c0bdf269a9a` — enrolled 2026-04-23
- **[WARNING] GuruRMM enrollment workaround:** WebSocket auth in `ws/mod.rs` does not validate `enrolled_agents.agent_key_hash`. New agent installs must overwrite registry AgentKey with the site API key (not the enrollment AgentKey) and restart service. See Gitea issue #8.

### Key Applications

| Application | Host | URL/Port | Notes |
|---|---|---|---|
| TestDataDB | AD2 | http://192.168.0.6:3000 | Node.js + Express, PostgreSQL 18, 469K records. Internal LAN only. |
| Sage ERP | SAGE-SQL | \\SAGE-SQL\sage (S:) | RDS-served RemoteApp |
| GageTrak | DF-GAGETRAK (192.168.0.102) | — | Calibration tracking. Sends email via calibration@dataforth.com (SMTP). GuruRMM enrolled. |
| Dataforth Product API | Hoffman's servers | https://www.dataforth.com/api/v1/TestReportDataFiles | OAuth2 client_credentials. Vault: `clients/dataforth/api-oauth.sops.yaml` |
| QuickBASIC 4.5 ATE | 64 DOS stations | T:\ (\\D2TESTNAS\test) | Automated test equipment programs. 1,470+ product model specs. |

---

## Syncro Asset Inventory (2026-06-02 Reconciliation)

Pulled full Syncro asset list for customer_id `578095`: **78 assets** across 2 pages.

### Reconciliation Result

| Bucket | Count | Meaning |
|---|---|---|
| KEEP | 20 | Active in Syncro (<150 days since last check-in) |
| SAVE + FLAG | 21 | Alive in ScreenConnect or Bitdefender but Syncro agent broken; do NOT delete — reinstall agent |
| REMOVE | 28 | Dead in all three systems (Syncro + ScreenConnect + Bitdefender) |
| VERIFY | 9 | Servers with no agent anywhere; could be live console-only; confirm before removing |

**Governing rule (Howard's 3-system OR):** A machine is saved if it has been online within 150 days in ANY of Syncro, ScreenConnect, or Bitdefender. Removal only if dead in all three.

### SAVE + FLAG — alive but Syncro agent broken (21 machines)

AD1, AD2, SAGE-SQL, FILES-D1, ENG-DEV-SERVER, D2-MFG-001, D1-ENGI-012, MY9-PC, D1-CUST-003, DANC0619, DFORTH-SHIP, DF-LEE11-I9, DFASLB0519, D2-AS-26, HGHAUBNER, D1-PWRM, D1-ENGI-EMCLAB1, D1-CONF-002, D2-HIPOT-SURFAC, D2-AS-34, TS-41 (shows as STATION_41 in ScreenConnect)

### VERIFY — servers with no agent (9 machines)

APPS, EXCHANGE, EXCHANGE16, AD-3, AD-4, OLD-AD2, SAGETS-1, EPICOR, D2-ASSY-001

Likely dead: OLD-AD2, EXCHANGE16, SAGETS-1. Confirm before removing: APPS, AD-3, AD-4, EXCHANGE, EPICOR, D2-ASSY-001.

### REMOVE — confirmed dead in all systems (28 asset IDs)

Syncro asset IDs: 23845, 149614, 9708445, 9357407, 9276901, 9212922, 9078651, 8824875, 8824867, 8726494, 8726485, 8657233, 8606209, 8572160, 8523941, 8411908, 8410614, 8632009, 8726495, 8421223, 9081717, 8726493, 8423782, 8726481, 8525650, 8622969, 8361459, 8670944

**Deletion method:** Syncro GUI only (`https://computerguru.syncromsp.com/customer_assets?customer_id=578095`). API route `DELETE /customer_assets/{id}` returns HTML 404 for this integration token — not exposed.

### Root Cause — Fleet-wide Syncro Agent Break ~2025-10-06

57 of 78 assets show `updated_at` frozen at or before 2025-10-06, while the remaining 21 show recent check-ins. This is a hard cutoff, not gradual attrition — indicating a fleet-wide Syncro agent failure around that date. The machines stayed online (visible in ScreenConnect); only the Syncro agent stopped reporting. Root cause not yet investigated. Flag for Dan Center / Winter when replying.

### Pending Actions (Coord todo tree, parent `103c48ad-7b31-4967-9388-065a91888e7c`, assigned to Howard)

1. Delete the 28 confirmed-dead assets in Syncro GUI.
2. Decide the 9 VERIFY servers.
3. Reinstall Syncro agent on the 21 SAVE+FLAG machines.
4. Switch Dataforth to metered Syncro asset billing once clean.
5. Reply to Winter; flag the ~2025-10-06 fleet-wide agent break for investigation.

---

## Third-Party Tool Inventory

### Bitdefender GravityZone

- **Company ID:** `64c94ef310db128bfa0d908f` (suffix `_578095` confirms Dataforth mapping)
- **Status:** Dataforth is being **phased off Bitdefender**. Only 4 of 57 GravityZone endpoints remain in "Custom Groups" (actively managed); 53 are in the "Deleted" folder (mostly unmanaged).
- **[WARNING] Bitdefender absence is NOT a decommission signal for Dataforth.** A machine missing from BD may simply have had its BD agent uninstalled as part of the phase-off. Use Syncro or ScreenConnect as liveness indicators.
- GravityZone company owner field: Lee Payne.

### ScreenConnect

- **Host:** `https://computerguru.screenconnect.com`
- **Extension GUID:** `2d558935-686a-4bd0-9991-07539f5fe749`
- **Vault:** `msp-tools/screenconnect.sops.yaml` (fields `credentials.username`, `credentials.api_secret`)
- **Working API auth (determined 2026-06-02):** `CTRLAuthHeader: <raw api_secret>` (NO "Basic " prefix) + `Origin: https://computerguru.screenconnect.com`. Basic-auth or "Basic <b64>" in CTRLAuthHeader both return 401.
- **Only exposed method:** `POST /App_Extensions/<guid>/Service.ashx/GetSessionsByName` with body `{"sessionName":"<name>"}`. All other Get* method names return 500. Agent `Name` fields are blank for unattended sessions — this API cannot enumerate the full Dataforth fleet; name-based lookup only.
- Custom session properties: CP1=Company, CP2=Site, CP3=Tag.

---

## Access

### Domain / Server Access
- **AD2 SSH:** `ssh sysadmin@192.168.0.6` (port 22) — vault: `clients/dataforth/ad2.sops.yaml` → `credentials.password` — NOTE: stale backslash escape in vault entry; strip with `sed 's/\\//g'`
- **AD1 SSH:** `ssh sysadmin@192.168.0.27` — vault: `clients/dataforth/ad1.sops.yaml`
- **D2TESTNAS SSH:** `ssh root@192.168.0.9` — vault: `clients/dataforth/d2testnas.sops.yaml`. Use root, NOT sysadmin (sysadmin SSH fails on D2TESTNAS). SSH key from acg-guru-5070 authorized. (Password auth works for root; UDM does NOT — UDM is publickey/keyboard-interactive only, 2FA push, key `id_ed25519_udm`.)
- **D2TESTNAS `aoibackup` share (AOI XP backup):** `\\192.168.0.9\aoibackup` — Samba user `admin` (password matches the XP's local login), `hosts allow = 192.168.1.175` only, `browseable = no`. Other NAS shares (`test`/`datasheets`/`snapshots`) explicitly deny 192.168.1.175. Creds in vault: `clients/dataforth/d2testnas.sops.yaml → credentials.smb.aoi-user` / `.aoi-password` / `.aoi-share`.
- **UDM SSH:** `ssh root@192.168.0.254` — SSH key `~/.ssh/id_ed25519_udm` (generated 2026-03-27)
- **SAGE-SQL SSH:** `ssh sysadmin@192.168.0.153` — SSH key (`C:\ProgramData\ssh\administrators_authorized_keys` on SAGE-SQL)
- **All server passwords:** `Paper123!@#` (domain admin sysadmin account — stored in individual vault entries per server)
- **WinRM (AD2/AD1):** port 5985 — pywinrm with NTLM, user `INTRANET\sysadmin`

### M365 / Entra
- **M365 admin:** sysadmin@dataforth.com — vault: `clients/dataforth/m365.sops.yaml`
- **Tenant ID:** `7dfa3ce8-c496-4b51-ab8d-bd3dcd78b584`
- **Claude-Code-M365 Entra App:** App ID `7a8c0b2e-57fb-4d79-9b5a-4b88d21b1f29`, secret expires 2027-12-22 — vault: `clients/dataforth/m365.sops.yaml → credentials.entra-app`
- **MSP Multi-Tenant App (Claude-MSP-Access):** MSP tenant `ce61461e-81a0-4c84-bb4a-7b354a9a356d`, App ID `fabb3421-8b34-484b-bc17-e46de9703418` — vault: msp-tools SOPS file
- **ComputerGuru tiered apps:** All 5 apps consented 2026-04-23. Exchange Operator SP (b43e7342) had Exchange Admin role added manually (gap in onboard-tenant.sh — not auto-assigned for Exch Operator).

### Dataforth Product API (Hoffman)
- **Vault:** `clients/dataforth/api-oauth.sops.yaml`
- Token URL: `https://login.dataforth.com/connect/token`
- Grant: `client_credentials`, Client ID: `dataforth.onprem.sync`, Scope: `dataforth.web`
- Token TTL: 1 hour
- Swagger: `https://www.dataforth.com/swagger/index.html`

### ESXi / Hypervisors
- ESXi-122: 192.168.0.122 — vault: `clients/dataforth/esxi-122.sops.yaml`
- ESXi-124: 192.168.0.124 — vault: `clients/dataforth/esxi-124.sops.yaml`

### PBX
- Vault: `clients/dataforth/pbx.sops.yaml`

---

## Patterns & Known Issues

### Active Directory
- **No custom security groups** — only default Windows groups. Service accounts in OU=ServiceAccounts.
- **ClaudeTools-ReadOnly AD account** — purpose unclear. Investigate.
- **Ken Hoffman has two accounts** (khoffman + oemdata) — not consolidated.
- **jlohr account retained** — post-retirement (2026-03-31), kept enabled specifically to receive ntirety.com infrastructure notifications. Inbox rule forwards to mike@azcomputerguru.com. Do NOT disable.
- **Entra sync scope:** OU=SyncedUsers **and OU=Azure_Users** sync to Entra (cbell confirmed in OU=Azure_Users, synced — 2026-06-01; the prior "SyncedUsers only" note was incomplete). CompanyUsers OU does NOT sync. 38 stale TS-* test station accounts were cleaned from Entra 2026-03-27.

### RDS / SAGE-SQL
- **RDS licensing:** Grace period reset 2026-05-06 by deleting GracePeriod registry key. Grace period expires again without proper CALs. Purchase RDS CALs (Per User mode, LicensingType=4).
- **TSGateway:** Disabled on SAGE-SQL (server not externally exposed at firewall). Do NOT re-enable without reason.
- **SSL cert:** Self-signed, subject `CN=sage-sql.intranet.dataforth.com`. Non-domain machines must manually import to Trusted Root + Trusted Publishers.
- **GPO cert distribution:** Not completed (AD2 SYSVOL write blocked from non-domain workstation). Pending.
- **Bitdefender GravityZone:** Managed AV on SAGE-SQL. Can block PowerShell execution — may need temporary disable for admin work.

### Voice / Phones
- **Production phones VLAN:** 192.168.100.0/24. PBX at .196 / .2. All production phones live here.
- **Unifi default voice VLAN (192.168.1.0/24):** NOT used for production — phones landing here cannot reach PBX. Switch port misconfiguration symptom: phone shows wrong date/time (NTP failure) and no dial tone.
- **D1-Server-Room port 1:** Controls lobby drop → must stay on VLAN 100. Reverted to default once before (2026-05-04 incident).

### Exchange Online / Email
- **INKY PhishFence StopProcessingRules:** Kills all subsequent transport rules. Use inbox rules for per-mailbox forwarding, NOT transport rules.
- **AutoForwarding blocked by default** (tenant outbound spam policy). If per-user forwarding needed, create scoped HostedOutboundSpamFilterPolicy for that sender with AutoForwardingMode=On.
- **Get-MessageTrace deprecated Sept 2025:** Use Get-MessageTraceV2 and Get-MessageTraceDetailV2 in Exchange PowerShell.

### GuruRMM Agent Deployment
- **WebSocket auth bug (Issue #8):** enrolled_agents.agent_key_hash is never checked by ws/mod.rs. Workaround: after MSI install, overwrite registry `HKLM:\SOFTWARE\GuruRMM\AgentKey` with the site API key (not enrollment AgentKey), then restart service.
- **rmm-api.azcomputerguru.com must be grey-clouded** (DNS-only, not proxied) — Cloudflare proxy blocks WebSocket. Do NOT re-enable orange cloud. Gitea Issue #9.

### Security
- **C2 IP blocks are iptables only** — do not survive UDM reboot. Must add to permanent UniFi block list via UI. C2 IPs: 80.76.49.18, 45.88.91.99 (AS399486 Virtuo, Montreal).
- **AD1 disk 90% full** — C:\Engineering = 787 GB of 1023 GB. Risk of replication failures.
- **Windows Firewall disabled on AD2** (all profiles) — known risk, not yet remediated.
- **3 Windows 7 machines on network** (LABELPC, LABELPC2, D2-RCVG-003) — EOL, unpatched.
- **AD1/AD2 on Windows Server 2016** — end of mainstream support. Plan upgrade.
- **Entra ID P2 not licensed** — IdentityRiskyUser risk check returns 403 even with scope consented. Would need P2 upgrade to enable Identity Protection.
- **IdentityRiskyUser.Read.All scope:** Consented to Security Investigator app but unusable (no P2 license).

### Syncro Asset Management
- **Fleet-wide Syncro agent break ~2025-10-06:** ~half of Dataforth machines stopped reporting to Syncro on or around that date while remaining online in ScreenConnect. Do NOT auto-remove machines frozen at that date without cross-checking ScreenConnect. Root cause unknown — needs investigation.
- **Bitdefender is NOT a liveness signal:** Dataforth is being phased off BD; 53 of 57 GravityZone endpoints are in the "Deleted" folder. Missing from BD = BD agent uninstalled, not machine dead.
- **API delete not available:** `DELETE /customer_assets/{id}` returns HTML 404 for the current integration token. All asset deletions must go through the Syncro GUI.

---

## Active Work

As of 2026-06-02:

- **Syncro asset cleanup (2026-06-02):** 78-asset reconciliation complete. 28 confirmed-dead assets pending GUI deletion; 21 alive-but-broken machines need Syncro agent reinstall; 9 servers in VERIFY bucket. Move to metered billing once clean. Reply to Winter pending. Coord todo tree assigned to Howard (parent `103c48ad-7b31-4967-9388-065a91888e7c`). See [Syncro Asset Inventory](#syncro-asset-inventory-2026-06-02-reconciliation) above.

- **AOI XP backup + isolation (2026-06-01):** AOI optical-inspection XP PC moved to VLAN 2 (mydata/SMT) @ 192.168.1.175; locked-down SMB1 share `aoibackup` on D2TESTNAS (XP-only, user `admin`). Other NAS shares now deny the XP. Mike OK'd full SMT visibility ("it's part of SMT"). **Optional EOL hardening pending:** block XP → company LAN (except NAS 192.168.0.9) + Internet on the UDM, scoped to .175 (won't affect other SMT devices). Todo `37543f7f`.

- **Test Datasheet Pipeline:** Production pipeline healthy. 469K records, 458.5K live on website. Daily task runs 02:30 AM. Email notification deployed but pending SMTP AUTH fix — sysadmin SMTP AUTH disabled in Exchange Online. See `projects/dataforth-dos/CONTEXT.md`.
- **GAGEtrak email (ticket #32142):** calibration@ SMTP re-enabled 2026-04-23. GAGEtrak configured (smtp.office365.com:587, calibration@dataforth.com). Kevin Wackerly verifying schedule on DF-GAGETRAK — expected Monday run appears to run Tuesday.
- **DKIM rotation:** Automatic cutover to selector2 on 2026-05-16 — no action needed; verify signing after that date.
- **jlohr forwarding:** ntirety.com inbox rule active as of 2026-05-12; confirmed delivering to mike@azcomputerguru.com. Defunct transport rule pending cleanup.
- **RDS / SAGE-SQL:** RDS grace period reset. GPO cert distribution pending. RDS CALs purchase needed long-term.
- **28 offline machines** (at time of 2026-03-27 incident) — rescanned status unknown. These should be verified when available.
- **MFA enforcement ongoing** — 19 users were still not enrolled as of April 4 enforcement date; current count unverified.

---

## History Highlights

| Date | Event |
|---|---|
| 2025 | Crypto/ransomware attack — AD2 wiped and rebuilt, many files lost. Test datasheet pipeline broken. |
| 2026-01-19 | DOS Update System built and deployed — NWTOC/CTONW/UPDATE/DEPLOY BAT files, 39 deployments. Sync-FromNAS updated (DEPLOY.BAT). |
| 2026-03-20 | Galactic Advisors security assessment — AD1 C: at 90%, legacy SQL 2008 R2 client noted, 3 computers scanned. |
| 2026-03-23 | Galactic Advisors assessment analyzed by ACG. |
| 2026-03-27 | **Major security incident:** DF-JOEL2 compromised via social engineering/ScreenConnect (attacker "Angel Raya", C2 on Virtuo hosting). M365 sign-in from Turkey. Full remediation. 3 CA policies deployed. MFA notice sent. IC3 filed (1c32ade367084be9acd548f23705736f). |
| 2026-03-27–29 | Test datasheet pipeline rebuilt — 72/73 Quatronix datasheets generated, new Node.js pipeline replaces VB6 DFWDS + VB.NET uploader. |
| 2026-03-31 | Joel Lohr retirement. Brian Faires mailbox converted to shared (5,711 messages preserved). 38 stale Entra TS-* accounts deleted. |
| 2026-04-04 | MFA CA policies enforced (switched from report-only). |
| 2026-04-11–12 | SCMVAS/SCMHVAS pipeline extension — 27,503 records backfilled, 434 Engineering-Tested .txt files imported. |
| 2026-04-12 | TestDataDB PostgreSQL migration verified (2.89M records). Hoffman API discovered (Swagger). |
| 2026-04-13 | API architecture discussion with Hoffman — client_credentials grant confirmed for dataforth.onprem.sync client. |
| 2026-04-14 | DFWDS logic ported to Node.js (dfwds-process.js). 897 staged datasheets drained. 803 new records created on Hoffman API. |
| 2026-04-15 | Major release — DB dedup (2.89M→469K rows), FAIL→PASS retest rule, For_Web filesystem dependency eliminated, 170,984 records bulk-pushed to Hoffman. Dashboard UI upgrades. |
| 2026-04-23 | Full Dataforth tenant onboarded to all 5 ComputerGuru tiered apps. calibration@ SMTP AUTH fixed. DF-GAGETRAK GuruRMM agent enrolled (with auth workaround). Syncro ticket #32142 billed. |
| 2026-05-03 | jantar@dataforth.com darkweb breach check — no indicators of compromise. eM Client OAuth grant and SP revoked/disabled. 1 hr billed. |
| 2026-05-04 | Howard onsite — lobby phone offline (VLAN misconfiguration on D1-Server-Room port 1 → fixed to VLAN 100). |
| 2026-05-06 | SAGE-SQL RDS issues resolved — grace period reset, SSL cert replaced, TSGateway disabled, RemoteApp permission prompts fixed. |
| 2026-05-12 | Pipeline audit + email notifications implemented (Graph API). jlohr forwarding configured (ntirety.com → mike@). DKIM keys rotated. |
| 2026-06-01 | AOI optical-inspection XP PC isolated onto VLAN 2 (mydata/SMT) @ 192.168.1.175; `aoibackup` SMB1 share created on D2TESTNAS locked to the XP only; other NAS shares set to deny the XP. D2TESTNAS confirmed Debian 13 / Samba 4.22.6 (repurposed Netgear ReadyNAS); vault + wiki OS corrected. Mike: AOI may see all of SMT; optional company-LAN/Internet block for the XP still pending. |
| 2026-06-01 | Chauncey Bell (cbell) M365 verified — active mailbox, licensed Microsoft 365 Business Standard (full Office + Exchange); AD password reset on AD2 (synced user, OU=Azure_Users), signed into Office. Bobbi's Outlook printing fixed by switching to Outlook (Classic). Ticket #32364 (0.5 hr onsite). |
| 2026-06-02 | Syncro asset reconciliation (78 assets): 20 keep / 21 save+flag / 28 remove / 9 verify. Root cause identified: fleet-wide Syncro agent break ~2025-10-06 silenced ~half the fleet while boxes stayed online (visible in ScreenConnect). Dataforth confirmed phasing off Bitdefender (only 4 of 57 GravityZone endpoints actively managed; 53 in Deleted folder). GUI delete list and 5-step todo tree handed to Howard. Move to metered billing pending cleanup. ScreenConnect API auth pattern documented (CTRLAuthHeader raw secret + Origin). |

---

## Backlinks

- [[projects/dataforth-dos]] — Active test datasheet pipeline project on AD2
- [[systems/jupiter]] — Neptune Exchange physically colocated at Dataforth D2 facility; D2TESTNAS provides Tailscale routing