wiki: full recompile cascades-tucson + dataforth (RF/voice applied state; mail stack, FreePBX, shares, cert pipeline; live Syncro hours)
This commit is contained in:
@@ -2,8 +2,8 @@
|
||||
type: client
|
||||
name: dataforth
|
||||
display_name: Dataforth Corporation
|
||||
last_compiled: 2026-06-04
|
||||
compiled_by: DESKTOP-0O8A1RL/claude-main
|
||||
last_compiled: 2026-06-20
|
||||
compiled_by: GURU-5070/claude-main
|
||||
sources:
|
||||
- clients/dataforth/docs/overview.md
|
||||
- clients/dataforth/docs/active-directory.md
|
||||
@@ -13,6 +13,25 @@ sources:
|
||||
- clients/dataforth/docs/SYNC_SCRIPT_UPDATE_SUMMARY.md
|
||||
- clients/dataforth/docs/incident-2026-03-27-abuse-report-virtuo.md
|
||||
- clients/dataforth/docs/incident-2026-03-27-abuse-report-connectwise.md
|
||||
- clients/dataforth/docs/cloud/m365.md
|
||||
- clients/dataforth/docs/issues/log.md
|
||||
- clients/dataforth/docs/network/topology.md
|
||||
- clients/dataforth/docs/network/vlans.md
|
||||
- clients/dataforth/docs/network/firewall.md
|
||||
- clients/dataforth/docs/rmm/rmm.md
|
||||
- clients/dataforth/docs/security/antivirus.md
|
||||
- clients/dataforth/docs/security/backup.md
|
||||
- clients/dataforth/docs/servers/ad1.md
|
||||
- clients/dataforth/docs/servers/ad2.md
|
||||
- clients/dataforth/docs/servers/d2testnas.md
|
||||
- clients/dataforth/docs/servers/df-hyperv-b.md
|
||||
- clients/dataforth/docs/servers/files-d1.md
|
||||
- clients/dataforth/docs/servers/sage-sql.md
|
||||
- clients/dataforth/docs/projects/shares-permissions/roadmap.md
|
||||
- clients/dataforth/docs/projects/shares-permissions/current-state-2026-06-10.md
|
||||
- clients/dataforth/docs/projects/shares-permissions/acl-audit-detail-2026-06-10.md
|
||||
- clients/dataforth/docs/projects/shares-permissions/discovery-email-draft.md
|
||||
- clients/dataforth/docs/aoi-xp-vlan-backup-runbook.md
|
||||
- clients/dataforth/session-logs/2026-03-23-galactic-advisors-report.md
|
||||
- clients/dataforth/session-logs/2026-03-27-security-incident-mfa-datasheets.md
|
||||
- clients/dataforth/session-logs/SESSION-SUMMARY.md
|
||||
@@ -25,11 +44,20 @@ sources:
|
||||
- clients/dataforth/session-logs/2026-05-04-lobby-phone-vlan-fix.md
|
||||
- clients/dataforth/session-logs/2026-05-06-session.md
|
||||
- clients/dataforth/session-logs/2026-05-12-session.md
|
||||
- clients/dataforth/session-logs/2026-06-01-aoi-xp-vlan-share.md
|
||||
- clients/dataforth/session-logs/2026-06-01-cbell-m365-bobbi-outlook.md
|
||||
- clients/dataforth/session-logs/2026-06-02-session.md
|
||||
- clients/dataforth/session-logs/2026-06-04-session.md
|
||||
- clients/dataforth/session-logs/project_ad2_context.md
|
||||
- clients/dataforth/session-logs/project_pipeline_rebuilt.md
|
||||
- clients/dataforth/session-logs/project_test_datasheet_pipeline.md
|
||||
- clients/dataforth/session-logs/project_new_product_lines.md
|
||||
- clients/dataforth/migration-gap-diff-RESUME.md
|
||||
- clients/dataforth/CLAUDE.dataforth.md
|
||||
- projects/dataforth-dos/CONTEXT.md
|
||||
- session-logs/2026-06-05-session.md
|
||||
- session-logs/2026-06/2026-06-09-mike-dataforth-freepbx-safesite-forensics.md
|
||||
- session-logs/2026-06/2026-06-18-mike-testdatadb-render-and-security-app.md
|
||||
- .claude/memory/project_dataforth_incident_2026-03-27.md
|
||||
- .claude/memory/project_datasheet_pipeline.md
|
||||
- .claude/memory/project_neptune_sbr_email_routing.md
|
||||
@@ -37,12 +65,11 @@ sources:
|
||||
- .claude/memory/reference_neptune_access_d2testnas.md
|
||||
- .claude/memory/feedback_d2testnas_ssh.md
|
||||
- .claude/memory/infra_office_network.md
|
||||
- clients/dataforth/session-logs/2026-06-01-aoi-xp-vlan-share.md
|
||||
- clients/dataforth/docs/aoi-xp-vlan-backup-runbook.md
|
||||
- clients/dataforth/session-logs/2026-06-01-cbell-m365-bobbi-outlook.md
|
||||
- clients/dataforth/session-logs/2026-06-02-session.md
|
||||
- clients/dataforth/session-logs/2026-06-04-session.md
|
||||
- clients/dataforth/migration-gap-diff-RESUME.md
|
||||
- .claude/memory/project_dataforth.md
|
||||
- .claude/memory/project_dataforth_history.md
|
||||
- .claude/memory/project_ad2_dataforth_fork.md
|
||||
- .claude/memory/ad2-ssh-mtu-blackhole.md
|
||||
- .claude/memory/ad2-comms-via-sync-only.md
|
||||
backlinks:
|
||||
- projects/dataforth-dos
|
||||
- systems/jupiter
|
||||
@@ -50,7 +77,7 @@ backlinks:
|
||||
|
||||
# Dataforth Corporation
|
||||
|
||||
Signal conditioning / data acquisition manufacturer in Tucson, AZ. Long-standing ACG client. Active managed relationship — monthly prepaid block. Notable for 64 MS-DOS 6.22 test stations, a major security incident in March 2026, an ongoing test datasheet pipeline modernization project, and an incomplete 2025 post-ransomware recovery restore that silently dropped files across multiple shares (active audit underway).
|
||||
Signal conditioning / data acquisition manufacturer in Tucson, AZ. Long-standing ACG client. Active managed relationship — monthly prepaid block. Notable for 64 MS-DOS 6.22 test stations, a major security incident in March 2026, an ongoing test datasheet pipeline modernization project, an incomplete 2025 post-ransomware recovery restore that silently dropped files across multiple shares (active audit underway), and a new shares/permissions remediation project (Phase 1 pending client input as of 2026-06-19).
|
||||
|
||||
---
|
||||
|
||||
@@ -76,8 +103,10 @@ Signal conditioning / data acquisition manufacturer in Tucson, AZ. Long-standing
|
||||
|
||||
- **External distributor:** Ginger (gy@quatronix-cn.com) — Quatronix China; receives datasheets
|
||||
- **Billing rate:** Prepaid block; all invoices show $0.00 — hours drawn from block
|
||||
- **Hours remaining:** 34.5 hrs as of 2026-06-04 (after 1.0 hr billed for SP1366 file recovery, ticket #32385). Always live-check Syncro before billing — `GET /customers/578095`.
|
||||
- **Hours remaining:** 31.5 hrs as of 2026-06-19 (live-check Syncro before billing — `GET /customers/578095`)
|
||||
- **Syncro customer ID:** 578095
|
||||
- **Syncro managed assets:** 50
|
||||
- **Open Syncro tickets:** 0 as of 2026-06-19
|
||||
- **Invoice CC:** jantar@dataforth.com
|
||||
|
||||
---
|
||||
@@ -88,18 +117,18 @@ Signal conditioning / data acquisition manufacturer in Tucson, AZ. Long-standing
|
||||
|
||||
| Host | IP | Role | OS | Notes |
|
||||
|---|---|---|---|---|
|
||||
| AD1 | 192.168.0.27 | Primary DC, DNS, FSMO roles, Engineering share | Windows Server 2016 | C:\ at **90%** capacity (C:\Engineering = 787 GB) — critical risk. FSMO roles (assumed all). GuruRMM agent `bf7bc5ee-4167-4a62-912a-c88b11a5943d`. Only `Image2025` backup plan — Files plan pending. |
|
||||
| AD2 | 192.168.0.6 | Secondary DC, TestDataDB service host, NAS mirror, WebShare | Windows Server 2022 | Hosts testdatadb Node.js service on :3000. Wiped by crypto attack 2025 — rebuilt. Windows Firewall disabled (all profiles). Shares: `C:\Shares\{c-drive,e-drive,webshare}`. Old `D:\c-drive` data volume is GONE — D: is now a mounted Windows install ISO. MSP360 agent at `C:\Program Files\Arizona Computer Guru\Online Backup\cbb.exe`; storage account `ACG-Dataforth`. GuruRMM agent `cfa93bb6-0cdc-4d4e-a29e-1609cda6f047`. No shadow copies. |
|
||||
| FILES-D1 | — | File server | — | Shares: `E:\Shares\{sales,archive}`. GuruRMM agent `8566a19d-49a9-4f8b-9c6c-012cc934484b`. **NOTE: `staff` share is missing** on FILES-D1 — separate issue. |
|
||||
| SAGE-SQL | 192.168.0.153 | Sage ERP (S:), RDS Session Host/Connection Broker/Web Access | Windows Server | RDS licensing grace period was expired (reset 2026-05-06). TSGateway disabled (server not externally exposed). New self-signed RDS cert installed. Bitdefender GravityZone managed AV. Share: `C:\sage`. GuruRMM agent `120ba7bf-8544-48a0-98a1-40ed5cdd3e1f`. |
|
||||
| 3CX | 192.168.0.125 | Phone system | — | Last logon Oct 2025 — possibly inactive |
|
||||
| DF-HYPERV-B | — | Hyper-V hypervisor | — | GuruRMM enrolled (agent ID — see GuruRMM fleet below) |
|
||||
| AD1 | 192.168.0.27 | Primary DC, DNS, FSMO roles, Engineering share | Windows Server 2016 | C:\ at **90%** capacity (C:\Engineering = 787 GB) — critical risk. FSMO roles (assumed all). GuruRMM agent `bf7bc5ee-4167-4a62-912a-c88b11a5943d`. Image plan (`Image2025`) + Files plan (NBF, daily 2 AM, 180-day retention — created 2026-06-05). |
|
||||
| AD2 | 192.168.0.6 | Secondary DC, TestDataDB service host, NAS mirror, WebShare | Windows Server 2022 | Hosts testdatadb Node.js service on :3000. Wiped by crypto attack 2025 — rebuilt. Windows Firewall disabled (all profiles). Shares: `C:\Shares\{c-drive,e-drive,webshare,test}`. Old `D:\c-drive` data volume is GONE — D: is now a mounted Windows install ISO. MSP360 agent at `C:\Program Files\Arizona Computer Guru\Online Backup\cbb.exe`; storage account `ACG-Dataforth`. GuruRMM agent `cfa93bb6-0cdc-4d4e-a29e-1609cda6f047`. No shadow copies. Runs ClaudeTools on `ad2` branch (coord-API isolated; comms via git sync only). |
|
||||
| FILES-D1 | 192.168.0.189 | File server | Windows Server 2016 | Shares: `E:\Shares\{sales,archive}`. GuruRMM agent `8566a19d-49a9-4f8b-9c6c-012cc934484b`. **NOTE: `staff` share is missing** on FILES-D1 — separate issue. |
|
||||
| SAGE-SQL | 192.168.0.153 | Sage ERP (S:), RDS Session Host/Connection Broker/Web Access | Windows Server 2016 | RDS licensing grace period was expired (reset 2026-05-06). TSGateway disabled (server not externally exposed). New self-signed RDS cert installed. Bitdefender GravityZone managed AV. Share: `C:\sage`. GuruRMM agent `120ba7bf-8544-48a0-98a1-40ed5cdd3e1f`. |
|
||||
| 3CX | 192.168.0.125 | Phone system (possibly inactive) | — | Last logon Oct 2025. Production phones live on VLAN 100 under the Sangoma/FreePBX PBX — 3CX role likely superseded. |
|
||||
| DF-HYPERV-B | 192.168.0.123 | Hyper-V hypervisor | Windows Server 2025 | GuruRMM enrolled. Newest server in environment. VM inventory not captured. |
|
||||
| DF-SVR-D2-Sync | — | (role TBD) | — | GuruRMM enrolled |
|
||||
| eng-dev-server | — | Engineering dev server | — | GuruRMM enrolled |
|
||||
| D2TESTNAS | 192.168.0.9 | SMB1 bridge for DOS test stations + AOI XP backup; Neptune Exchange physically colocated | Debian 13 (trixie), Samba 4.22.6 | **Repurposed Netgear ReadyNAS** (earlier "CachyOS"/"Netgear ReadyNAS" records were stale). SMB1 enabled globally (CORE..SMB3, NTLMv1) — required for DOS 6.22 stations. rsync daemon on port 873 (module `test`, user `rsync`, hosts allow 192.168.0.0/24 + 172.16.0.0/12). SSH: `root@192.168.0.9`. Tailscale route for 172.16.0.0/22. **Shares:** `test`/`datasheets`/`snapshots` (guest; now `hosts deny 192.168.1.175`), `aoibackup` (XP-only — see Access). |
|
||||
| ENG-DEV-SERVER | 192.168.0.126 | Engineering dev server | Windows 11 Pro | GuruRMM enrolled |
|
||||
| D2TESTNAS | 192.168.0.9 | SMB1 bridge for DOS test stations + AOI XP backup; Neptune Exchange colocation routing | Debian 13 (trixie), Samba 4.22.6 | **Repurposed Netgear ReadyNAS.** SMB1 enabled globally (CORE..SMB3, NTLMv1) — required for DOS 6.22 stations. rsync daemon on port 873 (module `test`, user `rsync`, hosts allow 192.168.0.0/24 + 172.16.0.0/12). SSH: `root@192.168.0.9`. Tailscale route for 172.16.0.0/22. **Shares:** `test`/`datasheets`/`snapshots` (guest; `hosts deny 192.168.1.175`), `aoibackup` (XP-only — see Access). Acts as jump host for UDM SSH (D2TESTNAS direct-tcpip channel to 192.168.0.254). |
|
||||
| ESXi hosts | 192.168.0.122, 192.168.0.124 | VMware ESXi hypervisors | ESXi | — |
|
||||
| UDM Firewall | 192.168.0.254 | Perimeter firewall/router | UniFi OS | MAC d0:21:f9:6c:11:02. Also responds on 192.168.0.1. SSH key: `~/.ssh/id_ed25519_udm`. C2 IPs blocked via iptables (NOT permanent — need to add to UniFi UI). |
|
||||
| PBX (3CX/Sangoma) | 192.168.100.2 (also .196) | VoIP PBX — production phones on 192.168.100.0/24 | — | TFTP provisioning for Cisco SPA502G phones. Access via SSH: `sangoma@192.168.100.2`. Vault: `clients/dataforth/pbx.sops.yaml` |
|
||||
| UDM Firewall | 192.168.0.254 | Perimeter firewall/router | UniFi OS 5.1.15 | MAC d0:21:f9:6c:11:02. Also responds on 192.168.0.1. SSH: `azcomputerguru@192.168.0.254`, root SSH key added 2026-06-08, 2FA push required. Vault: `clients/dataforth/udm.sops.yaml`. C2 IPs blocked via iptables (NOT permanent — need to add to UniFi UI). Boot scripts in `/data/on_boot.d/`: `10-neptune-snat.sh` (Neptune outbound SNAT), `30-freepbx-sip-forward.sh` (SIP DNAT, WAN UDP 5060 source-locked to 66.7.123.0/24 → 192.168.100.2; SIP-only — do NOT add RTP forward). |
|
||||
| PBX (Sangoma FreePBX) | 192.168.100.2 | VoIP PBX — production phones on 192.168.100.0/24 | Sangoma FreePBX 17 / Asterisk 22.5.2 | FirstDigital PJSIP trunk; SBC 66.7.123.215:5060 (Sonus), match 66.7.123.0/24; IP-auth (no registration). `qualify_frequency=0` (FD SBC ignores OPTIONS — do NOT revert). TFTP provisioning for Cisco SPA502G phones. SSH: `sangoma@192.168.100.2`. Vault: `clients/dataforth/pbx.sops.yaml`. [WARNING] Re-apply `PJSip.class.php` line-504 patch after any `fwconsole ma updateall`. |
|
||||
|
||||
**Neptune Exchange (ACG infrastructure, physically at Dataforth D2):**
|
||||
- `neptune.acghosting.com` | internal `172.16.3.11` | external inbound `67.206.163.124` / outbound `67.206.163.122`
|
||||
@@ -142,11 +171,14 @@ Signal conditioning / data acquisition manufacturer in Tucson, AZ. Long-standing
|
||||
- **M365 licenses:** 50x Business Premium (39 used), 19x Exchange Online Plan 1 (5 used), 5x SPB (4 used)
|
||||
- **SMTP settings:** smtp.office365.com, port 587, STARTTLS — use `sysadmin@dataforth.com`
|
||||
- **SMTP AUTH status:** Tenant-level not disabled; per-mailbox varies. `calibration@dataforth.com` had SmtpClientAuthentication=true re-enabled 2026-04-23. `sysadmin@dataforth.com` SMTP AUTH is blocked by Exchange Online default — testdatadb uses Graph API for email (Mail.Send permission granted to Claude-Code-M365 app 2026-05-12).
|
||||
- **Mail security stack (layered):**
|
||||
1. **INKY PhishFence** — active transport rule `B859327F-3FBD-4BE7-A47A-97D02F1558A7` fires first (StopProcessingRules=true). Use inbox rules for per-user mail routing, NOT transport rules.
|
||||
2. **Mailprotector CloudFilter** — outbound delivery gateway (`dataforth-com.outbound.emailservice.io`, 52.3.213.180). Active outbound connector "Outbound-Mailprotector" (recipientDomains `*`). Mail may be held here. If a message shows "Delivered" in Dataforth outbound trace but never arrives, check Mailprotector (/mailprotector skill). Discovered 2026-06-05 when ghaubner email was held by "INKY - Annotation - Recipient Not Group Member" transport rule.
|
||||
- **DKIM:** Both selector1 and selector2 published. Rotated 2026-05-12; cutover to selector2 on 2026-05-16.
|
||||
- `selector1._domainkey.dataforth.com` → selector1-dataforth-com._domainkey.dataforthcom.onmicrosoft.com
|
||||
- `selector2._domainkey.dataforth.com` → selector2-dataforth-com._domainkey.dataforthcom.onmicrosoft.com
|
||||
- **DNS Host:** ntirety.com — Dataforth's public DNS zone managed through ntirety's portal (not a standard registrar). DNS change requests go to ntirety, not a domain control panel. Joel Lohr's account retained to receive ntirety.com infrastructure notifications (inbox rule → mike@azcomputerguru.com).
|
||||
- **INKY PhishFence:** Active transport rule `B859327F-3FBD-4BE7-A47A-97D02F1558A7` fires first and calls StopProcessingRules=true — blocks all subsequent custom transport rules. Use inbox rules for per-user mail routing.
|
||||
- **AutoForwarding blocked by default** (tenant outbound spam policy). If per-user forwarding needed, create scoped HostedOutboundSpamFilterPolicy for that sender with AutoForwardingMode=On.
|
||||
- **MFA:** 3 Conditional Access policies created 2026-03-27 (initially report-only; enforced 2026-04-04):
|
||||
- "ACG - Require MFA for All Users" — skip from office IP 67.206.163.122
|
||||
- "ACG - Block Foreign Sign-Ins" — US-only; MFA-Travel-Bypass group for exceptions
|
||||
@@ -159,17 +191,17 @@ Signal conditioning / data acquisition manufacturer in Tucson, AZ. Long-standing
|
||||
|
||||
- **Domain:** intranet.dataforth.com | Forest/Domain Level: Windows Server 2016
|
||||
- **ISP:** fdtnet.net | Public IP: 67.206.163.122 (outbound), 67.206.163.124 (Neptune inbound)
|
||||
- **Firewall/Router:** UniFi Dream Machine at 192.168.0.254 (also 192.168.0.1)
|
||||
- **Firewall/Router:** UniFi Dream Machine Pro at 192.168.0.254 (also 192.168.0.1), UniFi OS 5.1.15
|
||||
- **Network:** Flat (no VLANs on main LAN — 192.168.0.0/24). Voice/PBX VLAN: 192.168.100.0/24 — production phones live here. **VLAN 2 "mydata" (192.168.1.0/24)** = SMT production-line network (gateway 192.168.1.1); members on the *D2-SMT Switch* (USW Enterprise 8) + *D2-Breakroom* port 12. Supersedes the earlier note that 192.168.1.0/24 was an unused UDM default voice VLAN — it is in active use by SMT. Inter-VLAN routing from mydata → main LAN is currently OPEN.
|
||||
- **mydata members (2026-06-01):** WinXPBE-724667 (AOI XP, .175), goldstar19, DESKTOP-FT0T4MK, My9-PC, + 3 unnamed industrial/SMT devices (MAC 00:90:fb:80:f0:c6, 00:80:79:05:23:f2, 00:80:79:04:47:e7).
|
||||
- **VPN:** FortiClient required for remote access to 192.168.0.x. VPN can drop mid-session — save work frequently.
|
||||
- **VPN:** OpenVPN for ACG remote access. Client subnet 192.168.6.x (GURU-5070 gets 192.168.6.2). [WARNING] GURU-5070 OpenVPN adapter "Local Area Connection" (ifIndex 12) MTU must be set to 1400 — default 1500 causes PMTU blackhole (tunnel path MTU ~1424; bulk SSH/SCP silently drops). Verify/re-apply: `Set-NetIPInterface -InterfaceIndex 12 -AddressFamily IPv4 -NlMtuBytes 1400`. Permanent fix: add `mssfix 1360` server-side on the Dataforth OpenVPN server.
|
||||
- **Drive mappings (GPO):** B: (\\ad1\itsvc), Q: (\\ad2\c-drive), S: (\\SAGE-SQL\sage), T: (\\ad2\e-drive), W: (\\files-d1\sales), X: (\\ad2\webshare), Y: (\\files-d1\archive). DOS test stations: T: (\\D2TESTNAS\test), X: (\\D2TESTNAS\datasheets)
|
||||
|
||||
### GuruRMM Enrollment
|
||||
|
||||
- **Site name:** Dataforth D1 | Site ID: `3a2f6866-26cd-452c-9806-a8df21475c3c`
|
||||
- **Site API key:** vault `clients/dataforth/...` [check vault for current entry]
|
||||
- **Fleet size:** 45 agents total (40 online) as of 2026-06-04 — grew from 13 enrolled agents
|
||||
- **Fleet size:** 45 agents enrolled as of 2026-06-04; Syncro managed count 50 as of 2026-06-19
|
||||
- **[WARNING] GuruRMM enrollment workaround:** WebSocket auth in `ws/mod.rs` does not validate `enrolled_agents.agent_key_hash`. New agent installs must overwrite registry AgentKey with the site API key (not the enrollment AgentKey) and restart service. See Gitea issue #8.
|
||||
|
||||
**Known enrolled agents:**
|
||||
@@ -184,33 +216,34 @@ Signal conditioning / data acquisition manufacturer in Tucson, AZ. Long-standing
|
||||
| SAGE-SQL | `120ba7bf-8544-48a0-98a1-40ed5cdd3e1f` | Enrolled 2026-06-04 |
|
||||
| DF-HYPERV-B | (see RMM dashboard) | Enrolled 2026-06-04 |
|
||||
| DF-SVR-D2-Sync | (see RMM dashboard) | Enrolled 2026-06-04 |
|
||||
| eng-dev-server | (see RMM dashboard) | Enrolled 2026-06-04 |
|
||||
| ENG-DEV-SERVER | (see RMM dashboard) | Enrolled 2026-06-04 |
|
||||
| (37 additional agents) | — | Mix of workstations; full list in GuruRMM dashboard |
|
||||
|
||||
### Backup Architecture
|
||||
|
||||
- **MSP360 ("ACG-Online Backup", `cbb.exe`):** Backup provider. Storage account: `ACG-Dataforth` (account ID `0b49ca5e-...`).
|
||||
- **AD2:** Two plans — `AD2 Image` (image plan, bunch `35a5c3d2`, running daily), `Files` plan (180-day retention, NBF, daily 2 AM, covers `C:\Shares` tree; GFS off, synthetic full, compression, fast-NTFS). No shadow copies on AD2.
|
||||
- **AD1:** Only `Image2025` image plan. **Files plan PENDING** — command prepared (`addBackupPlan -n "Files" -a "ACG-Dataforth" -nbf ... -d "C:\Engineering" -d "C:\Shares\ITSvc" ... -purge "180d"`); awaiting Mike's "run AD1" signal.
|
||||
- **AD1:** `Image2025` image plan + **Files plan created 2026-06-05** (NBF, daily 2 AM, 180-day retention, `ACG-Dataforth`, covers `C:\Engineering` + `C:\Shares\ITSvc`; initial run at 2:00 AM, not manually triggered). Both image and file plans now in place, matching AD2.
|
||||
- **Pre-attack backup (offline, not MSP360):** HGHAUBNER `D:` drive holds a full pre-attack snapshot of all 7 mapped DF shares, captured before the 2025 ransomware event. This is the only recovery source predating the attack. Accessible via GuruRMM `user_session` on HGHAUBNER. Cross-machine writes use existing GPO-mapped drives only (fresh UNC blocked by WTS-impersonation — see Patterns).
|
||||
- **Historical file-level backup:** NBF bunch `faad5a67` ("Backup plan on 8/29/2025") in `ACG-Dataforth` storage contains restore points 8/29–9/29/2025, archived at old physical path `D:\c-drive\...` (pre-migration layout). Used successfully 2026-06-04 to confirm SP1366 file contents (HGHAUBNER backup chosen for actual restore — no B2 egress).
|
||||
- **WizTree backup CSV (2026-06-04):** Full-drive WizTree export of HGHAUBNER's `D:` stored at AD2 `C:\ClaudeTools\clients\dataforth\WizTree_20260604184904.zip` (sensitive — kept OFF shares). ~8.7M files / 5.7 TB across 7 shares documented. Working copy also at GURU-5070 `C:\Users\guru\AppData\Local\Temp\wiztree.zip` (delete after diff).
|
||||
- **Historical file-level backup:** NBF bunch `faad5a67` ("Backup plan on 8/29/2025") in `ACG-Dataforth` storage contains restore points 8/29–9/29/2025, archived at old physical path `D:\c-drive\...` (pre-migration layout). Used successfully 2026-06-04 to confirm SP1366 file contents.
|
||||
- **WizTree backup CSV (2026-06-04):** Full-drive WizTree export of HGHAUBNER's `D:` stored at AD2 `C:\ClaudeTools\clients\dataforth\WizTree_20260604184904.zip` (sensitive — kept OFF shares). ~8.7M files / 5.7 TB across 7 shares documented.
|
||||
|
||||
### Key Applications
|
||||
|
||||
| Application | Host | URL/Port | Notes |
|
||||
|---|---|---|---|
|
||||
| TestDataDB | AD2 | http://192.168.0.6:3000 | Node.js + Express, PostgreSQL 18, 469K records. Internal LAN only. |
|
||||
| TestDataDB | AD2 | http://192.168.0.6:3000 | Node.js + Express, PostgreSQL 18, 469K records. Internal LAN only. Redesigned UI deployed 2026-06-18 (cert-fit, publish chips, push toasts, full-screen results). |
|
||||
| Sage ERP | SAGE-SQL | \\SAGE-SQL\sage (S:) | RDS-served RemoteApp |
|
||||
| GageTrak | DF-GAGETRAK (192.168.0.102) | — | Calibration tracking. Sends email via calibration@dataforth.com (SMTP). GuruRMM enrolled. |
|
||||
| Dataforth Product API | Hoffman's servers | https://www.dataforth.com/api/v1/TestReportDataFiles | OAuth2 client_credentials. Vault: `clients/dataforth/api-oauth.sops.yaml` |
|
||||
| Dataforth Product API | Hoffman's servers | https://www.dataforth.com/api/v1/TestReportDataFiles | OAuth2 client_credentials. Vault: `clients/dataforth/api-oauth.sops.yaml`. Used actively to recover DSCA33/45 and 8B/5B/SCM spec templates. |
|
||||
| QuickBASIC 4.5 ATE | 64 DOS stations | T:\ (\\D2TESTNAS\test) | Automated test equipment programs. 1,470+ product model specs. |
|
||||
| Power Monitor SPA | Georg's dev / TBD | — | Vanilla-JS SPA for Dataforth power meters (built by Georg/Antigravity AI). Demo at PWM.dataforth.com proposed; gateway architecture designed. Parked pending Mike↔Georg conversation. `clients/dataforth/power-monitor-demo/` |
|
||||
|
||||
---
|
||||
|
||||
## Syncro Asset Inventory (2026-06-02 Reconciliation)
|
||||
|
||||
Pulled full Syncro asset list for customer_id `578095`: **78 assets** across 2 pages.
|
||||
Pulled full Syncro asset list for customer_id `578095`: **78 assets** across 2 pages. Syncro currently shows 50 managed assets (2026-06-19 live data); reconciliation/cleanup ongoing.
|
||||
|
||||
### Reconciliation Result
|
||||
|
||||
@@ -241,7 +274,7 @@ Syncro asset IDs: 23845, 149614, 9708445, 9357407, 9276901, 9212922, 9078651, 88
|
||||
|
||||
### Root Cause — Fleet-wide Syncro Agent Break ~2025-10-06
|
||||
|
||||
57 of 78 assets show `updated_at` frozen at or before 2025-10-06, while the remaining 21 show recent check-ins. This is a hard cutoff, not gradual attrition — indicating a fleet-wide Syncro agent failure around that date. The machines stayed online (visible in ScreenConnect); only the Syncro agent stopped reporting. Root cause not yet investigated. Flag for Dan Center / Winter when replying.
|
||||
57 of 78 assets show `updated_at` frozen at or before 2025-10-06, while the remaining 21 show recent check-ins. This is a hard cutoff, not gradual attrition — indicating a fleet-wide Syncro agent failure around that date. The machines stayed online (visible in ScreenConnect); only the Syncro agent stopped reporting. Root cause not yet investigated.
|
||||
|
||||
### Pending Actions (Coord todo tree, parent `103c48ad-7b31-4967-9388-065a91888e7c`, assigned to Howard)
|
||||
|
||||
@@ -276,11 +309,11 @@ Syncro asset IDs: 23845, 149614, 9708445, 9357407, 9276901, 9212922, 9078651, 88
|
||||
## Access
|
||||
|
||||
### Domain / Server Access
|
||||
- **AD2 SSH:** `ssh sysadmin@192.168.0.6` (port 22) — vault: `clients/dataforth/ad2.sops.yaml` → `credentials.password` — NOTE: stale backslash escape in vault entry; strip with `sed 's/\\//g'`
|
||||
- **AD2 SSH:** `ssh sysadmin@192.168.0.6` (port 22) — vault: `clients/dataforth/ad2.sops.yaml` → `credentials.password` — NOTE: stale backslash escape in vault entry; strip with `sed 's/\\//g'`. MTU-sensitive: GURU-5070 OpenVPN adapter ifIndex 12 must be MTU 1400 for reliable bulk transfers.
|
||||
- **AD1 SSH:** `ssh sysadmin@192.168.0.27` — vault: `clients/dataforth/ad1.sops.yaml`
|
||||
- **D2TESTNAS SSH:** `ssh root@192.168.0.9` — vault: `clients/dataforth/d2testnas.sops.yaml`. Use root, NOT sysadmin (sysadmin SSH fails on D2TESTNAS). SSH key from acg-guru-5070 authorized. (Password auth works for root; UDM does NOT — UDM is publickey/keyboard-interactive only, 2FA push, key `id_ed25519_udm`.)
|
||||
- **D2TESTNAS `aoibackup` share (AOI XP backup):** `\\192.168.0.9\aoibackup` — Samba user `admin` (password matches the XP's local login), `hosts allow = 192.168.1.175` only, `browseable = no`. Other NAS shares (`test`/`datasheets`/`snapshots`) explicitly deny 192.168.1.175. Creds in vault: `clients/dataforth/d2testnas.sops.yaml → credentials.smb.aoi-user` / `.aoi-password` / `.aoi-share`.
|
||||
- **UDM SSH:** `ssh root@192.168.0.254` — SSH key `~/.ssh/id_ed25519_udm` (generated 2026-03-27)
|
||||
- **D2TESTNAS SSH:** `ssh root@192.168.0.9` — vault: `clients/dataforth/d2testnas.sops.yaml`. Use root, NOT sysadmin (sysadmin SSH fails on D2TESTNAS). SSH key from acg-guru-5070 authorized.
|
||||
- **D2TESTNAS `aoibackup` share (AOI XP backup):** `\\192.168.0.9\aoibackup` — Samba user `admin` (password matches the XP's local login), `hosts allow = 192.168.1.175` only, `browseable = no`. Other NAS shares explicitly deny 192.168.1.175. Creds in vault: `clients/dataforth/d2testnas.sops.yaml → credentials.smb.aoi-user` / `.aoi-password` / `.aoi-share`.
|
||||
- **UDM SSH:** `ssh azcomputerguru@192.168.0.254` (2FA push) or `ssh root@192.168.0.254` (root SSH key installed 2026-06-08). Jump via D2TESTNAS: paramiko `direct-tcpip` channel or ProxyJump. Vault: `clients/dataforth/udm.sops.yaml` (corrected 2026-06-09).
|
||||
- **SAGE-SQL SSH:** `ssh sysadmin@192.168.0.153` — SSH key (`C:\ProgramData\ssh\administrators_authorized_keys` on SAGE-SQL)
|
||||
- **All server passwords:** vault (individual vault entries per server — `clients/dataforth/<host>.sops.yaml`)
|
||||
- **WinRM (AD2/AD1):** port 5985 — pywinrm with NTLM, user `INTRANET\sysadmin`
|
||||
@@ -304,6 +337,7 @@ Syncro asset IDs: 23845, 149614, 9708445, 9357407, 9276901, 9212922, 9078651, 88
|
||||
- Grant: `client_credentials`, Client ID: `dataforth.onprem.sync`, Scope: `dataforth.web`
|
||||
- Token TTL: 1 hour
|
||||
- Swagger: `https://www.dataforth.com/swagger/index.html`
|
||||
- Endpoints: `GET /api/v1/TestReportDataFiles/{serial}` (per-model cert), `/bulk`, `/stats`
|
||||
|
||||
### ESXi / Hypervisors
|
||||
- ESXi-122: 192.168.0.122 — vault: `clients/dataforth/esxi-122.sops.yaml`
|
||||
@@ -311,6 +345,7 @@ Syncro asset IDs: 23845, 149614, 9708445, 9357407, 9276901, 9212922, 9078651, 88
|
||||
|
||||
### PBX
|
||||
- Vault: `clients/dataforth/pbx.sops.yaml`
|
||||
- SSH: `sangoma@192.168.100.2`
|
||||
|
||||
---
|
||||
|
||||
@@ -330,13 +365,18 @@ Syncro asset IDs: 23845, 149614, 9708445, 9357407, 9276901, 9212922, 9078651, 88
|
||||
- **GPO cert distribution:** Not completed (AD2 SYSVOL write blocked from non-domain workstation). Pending.
|
||||
- **Bitdefender GravityZone:** Managed AV on SAGE-SQL. Can block PowerShell execution — may need temporary disable for admin work.
|
||||
|
||||
### Voice / Phones
|
||||
### Voice / Phones / FreePBX
|
||||
- **Production phones VLAN:** 192.168.100.0/24. PBX at .196 / .2. All production phones live here.
|
||||
- **Unifi default voice VLAN (192.168.1.0/24):** NOT used for production — phones landing here cannot reach PBX. Switch port misconfiguration symptom: phone shows wrong date/time (NTP failure) and no dial tone.
|
||||
- **D1-Server-Room port 1:** Controls lobby drop → must stay on VLAN 100. Reverted to default once before (2026-05-04 incident).
|
||||
- **FirstDigital trunk — `qualify_frequency=0`:** FD's Sonus SBC ignores SIP OPTIONS keepalives. Setting `qualify=0` in the `pjsip` DB (id=1) prevents trunk from going Unavailable. **Do NOT revert to a non-zero qualify.** (Total phone outage 2026-06-08 was caused by FD SBC not answering OPTIONS, making trunk go Unavailable and blocking all INVITEs.)
|
||||
- **PJSip.class.php line 504 patch must be re-applied** after any `fwconsole ma updateall`. It is wiped by FreePBX updates. Backup before each update (`PJSip.class.php.bak.<timestamp>`).
|
||||
- **Do NOT port-forward the RTP range (10000-20000)** on the UDM for this trunk. A static RTP DNAT creates a conntrack collision with the PBX's outbound RTP — inbound works but outbound audio dies. SIP 5060 forward only (source-locked to 66.7.123.0/24). Current on_boot.d script (`30-freepbx-sip-forward.sh`) is SIP-only, correct.
|
||||
- **Inbound SIP relies on `/data/on_boot.d/30-freepbx-sip-forward.sh`** — not a persistent UniFi UI rule. Must survive UDM reboot via the script. Recommend Mike add a UI port-forward as a belt-and-suspenders measure.
|
||||
|
||||
### Exchange Online / Email
|
||||
- **INKY PhishFence StopProcessingRules:** Kills all subsequent transport rules. Use inbox rules for per-mailbox forwarding, NOT transport rules.
|
||||
- **Mailprotector CloudFilter:** Outbound delivery goes through Mailprotector. If a message is "Delivered" per Dataforth's outbound trace but never arrives, check Mailprotector (`/mailprotector skill`, `py mp.py messages ...`) — it may be held. The INKY "Annotation - Recipient Not Group Member" transport rule can route mail to Mailprotector's hold queue.
|
||||
- **AutoForwarding blocked by default** (tenant outbound spam policy). If per-user forwarding needed, create scoped HostedOutboundSpamFilterPolicy for that sender with AutoForwardingMode=On.
|
||||
- **Get-MessageTrace deprecated Sept 2025:** Use Get-MessageTraceV2 and Get-MessageTraceDetailV2 in Exchange PowerShell.
|
||||
|
||||
@@ -349,12 +389,28 @@ Syncro asset IDs: 23845, 149614, 9708445, 9357407, 9276901, 9212922, 9078651, 88
|
||||
- **Workaround that works:** Run on the SOURCE machine in `user_session` and write to an **existing GPO-mapped drive** (e.g. Q: → `\\ad2\c-drive`). The existing mapping survives impersonation; fresh UNC does not.
|
||||
- **Proven 2026-06-04 on HGHAUBNER:** local `D:\DF C-Drive` read + `Q:` write succeeded; AD2-side `user_session` copy and SSH-from-AD2 both failed.
|
||||
|
||||
### AD2 SSH / VPN MTU
|
||||
- **PMTU blackhole on GURU-5070 → AD2 SSH:** GURU-5070's OpenVPN adapter "Local Area Connection" (ifIndex 12, IP 192.168.6.2) defaults to MTU 1500. Tunnel path MTU is ~1424 (FD ping confirms). Over-MTU bulk TCP segments (SSH transfers, SCP) are silently dropped. Small interactive commands pass, creating a false appearance of "flaky VPN" or "SSH ban."
|
||||
- **Fix (applied 2026-06-18):** `Set-NetIPInterface -InterfaceIndex 12 -AddressFamily IPv4 -NlMtuBytes 1400` on GURU-5070 via SYSTEM RMM agent. Registry-persistent but may reset on OpenVPN reconnect — verify with `Get-NetIPInterface -InterfaceIndex 12`.
|
||||
- **Durable fix:** server-side `mssfix 1360` on the Dataforth OpenVPN server (or `push "tun-mtu 1400"`) — would auto-clamp all fleet clients, not just GURU-5070.
|
||||
- **AD2 is NOT the target for SSH diagnosis** when SSH is the failing channel — use RMM instead.
|
||||
|
||||
### AD2 Branch / Coordination
|
||||
- **AD2 operates on the `ad2` git branch.** Fork is rebased from main + thin Dataforth-specific commits. Do NOT edit shared fleet files on `ad2` — conflicts on every sync. Dataforth context lives in `clients/dataforth/CLAUDE.dataforth.md`.
|
||||
- **AD2 is coord-API isolated:** 172.16.3.30 is unreachable from Dataforth LAN. Coord messages, locks, and todos NEVER reach AD2. All inter-session coordination goes through git sync: committed handoff docs + `## Note for <user>` blocks. Do NOT use the coord skill for AD2.
|
||||
- **sync.sh on AD2:** not fork-aware on the push step (always tries `main`); force-push manually: `git push --force-with-lease origin ad2` after rebasing.
|
||||
|
||||
### Post-Ransomware Recovery Restore (2025) — Incomplete File Migration
|
||||
- **The 10/1/2025 recovery restore was incomplete.** The `Restore plan 10/1/2025` (~3.4M files) migrated each share from the old `D:\<share>` layout to the current `C:\Shares\...` layout on AD2 and dropped files in the process. Proven case: SP1366 MAQ20 Communications Module — each `PRINTOUTS FOR MANUFACTURING` folder for revisions E–H received only one file (the drill panel) when the backup contained ~6 files per revision. The 9/29/2025 file-level backup confirms the files existed before the restore.
|
||||
- **Scope unknown.** Other folders across the 7 shares may have similar gaps. A full migration-gap audit is underway (WizTree both sides — see Active Work). The audit is **review-only** — no automatic restore, because some deletions were intentional and the HGHAUBNER backup is additive-only (includes Georg's personal files alongside corporate data).
|
||||
- **Backup-side CSV** for diffing stored at AD2 `C:\ClaudeTools\clients\dataforth\WizTree_20260604184904.zip` (sensitive file list — keep off shares and off any publicly accessible directory).
|
||||
- **AD2 D: drive is gone.** The old `D:\c-drive` data volume was repurposed as a mounted Windows install ISO during the rebuild. All share data now lives under `C:\Shares`. The historical file-level backup (bunch `faad5a67`) archived the data under `D:\c-drive\...` (pre-migration path) — reconcile paths accordingly.
|
||||
|
||||
### Shares ACL State — All Open to All Staff
|
||||
- **All 8 business shares grant access to every employee** via `Everyone`/`Domain Users` (FullControl on 4 shares, Modify on 3). No department-based security groups exist. Sensitive data — Payroll, OSHA records, Purchase Orders, Accounting/QuickBooks, Sage financials — is fully readable and writable by all domain users.
|
||||
- **Remediation project in progress** (Shares & Permissions, started 2026-06-10). Phase 0 (discovery) complete. Phase 1 (client input/department matrix) pending email to Dan Center. Do not apply ACL changes until after client sign-off on the target model. Details: `clients/dataforth/docs/projects/shares-permissions/`.
|
||||
- **Special shares excluded from remediation:** `test` (DOS/SMB1 guest — leave open); `webshare` (preserve `svc_testdatadb:Full`); `ITSvc` (Domain Computers needs Read).
|
||||
|
||||
### Security
|
||||
- **C2 IP blocks are iptables only** — do not survive UDM reboot. Must add to permanent UniFi block list via UI. C2 IPs: 80.76.49.18, 45.88.91.99 (AS399486 Virtuo, Montreal).
|
||||
- **AD1 disk 90% full** — C:\Engineering = 787 GB of 1023 GB. Risk of replication failures.
|
||||
@@ -376,26 +432,40 @@ Syncro asset IDs: 23845, 149614, 9708445, 9357407, 9276901, 9212922, 9078651, 88
|
||||
|
||||
## Active Work
|
||||
|
||||
As of 2026-06-04:
|
||||
As of 2026-06-19 (no open Syncro tickets):
|
||||
|
||||
- **Migration-gap audit (in progress):** WizTree CSV of HGHAUBNER's pre-attack backup captured (AD2 `C:\ClaudeTools\clients\dataforth\WizTree_20260604184904.zip`). Next: WizTree runs on live servers (AD2, FILES-D1, SAGE-SQL, AD1) tomorrow (2026-06-05); diff CSV-to-CSV per share → `clients/dataforth/migration-gap-catalog-2026-06-04.md`. Full plan in `clients/dataforth/migration-gap-diff-RESUME.md`. RMM agent IDs for the 4 servers are documented there. No auto-restore — review-only catalog.
|
||||
- **Shares & Permissions project (Phase 1 — BLOCKING, pending client input):** Phase 0 (discovery) completed 2026-06-10 — read-only ACL audit confirmed all 8 business shares open to all employees; Domain Users has FullControl on 4 shares. Discovery email to Dan Center drafted (`clients/dataforth/docs/projects/shares-permissions/discovery-email-draft.md`); not yet sent. Phase 1 blocked on client responses: department list, access matrix, sensitive-data rules, staff rosters. Full roadmap: `clients/dataforth/docs/projects/shares-permissions/roadmap.md`.
|
||||
|
||||
- **AD1 Files backup (command ready, not run):** `addBackupPlan` command prepared for AD1 (NBF, daily 2 AM, 180-day retention, `ACG-Dataforth`, covers `C:\Engineering` + `C:\Shares\ITSvc`). Awaiting Mike's explicit "run AD1" approval — production DC. Full command in `clients/dataforth/migration-gap-diff-RESUME.md`.
|
||||
- **8B/5B/SCM render completion (parked with AD2):** Root-caused a `parseRawData` bug (PASS/FAIL line consumed as step-response for families that omit `"0","0",v` line). 136 8B/5B/SCM templates mined from Hoffman API (2026-06-18). Completion — wiring templates into the live renderer with correct slotmaps, QB rounding, and frequency/AAC accuracy — handed to AD2 (its now-proven machinery from DSCA33/45 work). Sync handoff at `projects/dataforth-dos/8B5BSCM-RENDER-VERIFY-2026-06-18.md`. ~9,624 records remain unpublished; this is a render-coverage gap (null renders correctly skipped), not a backlog.
|
||||
|
||||
- **SP1366 MAQ20 file recovery (RESOLVED 2026-06-04):** 19/20 missing manufacturing print PDFs restored for revisions E–H to AD2 `C:\Shares\c-drive\DOCUMENT\DESIGN\SP\SP1366 MAQ20 Communications Module\{E,F,G,H}\PCB1366 REV <rev> PRINTOUTS FOR MANUFACTURING`. Syncro ticket #32385 billed 1.0 hr remote (prepaid, $0), resolved + invoiced. REV F `TOP PASTE LAYER` confirmed absent from both independent backups — not restored.
|
||||
- **Migration-gap audit (parked):** WizTree CSV of HGHAUBNER's pre-attack backup captured (AD2 `C:\ClaudeTools\clients\dataforth\WizTree_20260604184904.zip`). WizTree runs on live servers deferred — no diff yet. Plan: run WizTree on AD2, FILES-D1, SAGE-SQL, AD1 → diff CSV-to-CSV per share → `clients/dataforth/migration-gap-catalog-2026-06-04.md`. Full plan in `clients/dataforth/migration-gap-diff-RESUME.md`. No auto-restore — review-only catalog.
|
||||
|
||||
- **Syncro asset cleanup (2026-06-02):** 78-asset reconciliation complete. 28 confirmed-dead assets pending GUI deletion; 21 alive-but-broken machines need Syncro agent reinstall; 9 servers in VERIFY bucket. Move to metered billing once clean. Reply to Winter pending. Coord todo tree assigned to Howard (parent `103c48ad-7b31-4967-9388-065a91888e7c`). See [Syncro Asset Inventory](#syncro-asset-inventory-2026-06-02-reconciliation) above.
|
||||
- **Syncro asset cleanup (with Howard):** 78-asset reconciliation complete. 28 confirmed-dead assets pending GUI deletion; 21 alive-but-broken machines need Syncro agent reinstall; 9 servers in VERIFY bucket. Move to metered billing once clean. Coord todo tree assigned to Howard (parent `103c48ad-7b31-4967-9388-065a91888e7c`). See [Syncro Asset Inventory](#syncro-asset-inventory-2026-06-02-reconciliation) above.
|
||||
|
||||
- **AOI XP backup + isolation (2026-06-01):** AOI optical-inspection XP PC moved to VLAN 2 (mydata/SMT) @ 192.168.1.175; locked-down SMB1 share `aoibackup` on D2TESTNAS (XP-only, user `admin`). Other NAS shares now deny the XP. Mike OK'd full SMT visibility ("it's part of SMT"). **Optional EOL hardening pending:** block XP → company LAN (except NAS 192.168.0.9) + Internet on the UDM, scoped to .175 (won't affect other SMT devices). Todo `37543f7f`.
|
||||
- **AOI XP backup + isolation (ongoing):** AOI optical-inspection XP PC on VLAN 2 (mydata/SMT) @ 192.168.1.175; locked-down SMB1 share `aoibackup` on D2TESTNAS (XP-only, user `admin`). Other NAS shares now deny the XP. **Optional EOL hardening pending:** block XP → company LAN (except NAS 192.168.0.9) + Internet on the UDM, scoped to .175. Todo `37543f7f`.
|
||||
|
||||
- **AD2 Claude capability updates (parked):** AD2 runs its own Claude from `C:\ClaudeTools`. Needs: (a) syncro + coord commands, (b) DF wiki read-write, (c) Dataforth client data access. Determine if remote is shared Gitea (git pull sufficient) or diverged clone. See resume doc.
|
||||
- **AD2 Claude capability updates (parked):** AD2 runs its own Claude from `C:\ClaudeTools` on the `ad2` branch. Needs: (a) syncro + coord commands, (b) DF wiki read-write, (c) Dataforth client data access. Python 3.12.8 and identity.json installed 2026-06-17. Coord API unreachable from Dataforth LAN — comms via git sync only.
|
||||
|
||||
- **Power Monitor SPA demo (parked):** Georg Haubner developed a vanilla-JS power-meter SPA (AI-built, `clients/dataforth/ExternalCodeReview.zip`). ACG designed a gateway architecture for a gated demo at `PWM.dataforth.com` (inbound tunnel, no meter publicly exposed, magic-link auth). Spec at `clients/dataforth/power-monitor-demo/GATEWAY-SPEC.md`. Parked pending Mike↔Georg conversation.
|
||||
|
||||
- **Test Datasheet Pipeline:**
|
||||
- Production pipeline healthy. 469K records, DSCA33/45 recovery complete (1,452 new certs published 2026-06-18 via Hoffman API). Daily task runs 02:30 AM.
|
||||
- Email notifications deployed (Graph API via `sysadmin@dataforth.com`).
|
||||
- 8B/5B/SCM render gap — parked with AD2 (see above).
|
||||
- 2 niche DSCA models (DSCA33-1948, DSCA45-1746) and their 8B equivalents have no Hoffman original — no template, cannot auto-publish.
|
||||
- DKIM: cutover to selector2 on 2026-05-16 — no action needed; verify signing after that date.
|
||||
|
||||
- **GAGEtrak email (ticket #32142):** calibration@ SMTP re-enabled 2026-04-23. GAGEtrak configured (smtp.office365.com:587, calibration@dataforth.com). Kevin Wackerly verifying schedule — expected Monday run appears to run Tuesday.
|
||||
|
||||
- **Test Datasheet Pipeline:** Production pipeline healthy. 469K records, 458.5K live on website. Daily task runs 02:30 AM. Email notification deployed but pending SMTP AUTH fix — sysadmin SMTP AUTH disabled in Exchange Online. See `projects/dataforth-dos/CONTEXT.md`.
|
||||
- **GAGEtrak email (ticket #32142):** calibration@ SMTP re-enabled 2026-04-23. GAGEtrak configured (smtp.office365.com:587, calibration@dataforth.com). Kevin Wackerly verifying schedule on DF-GAGETRAK — expected Monday run appears to run Tuesday.
|
||||
- **DKIM rotation:** Automatic cutover to selector2 on 2026-05-16 — no action needed; verify signing after that date.
|
||||
- **jlohr forwarding:** ntirety.com inbox rule active as of 2026-05-12; confirmed delivering to mike@azcomputerguru.com. Defunct transport rule pending cleanup.
|
||||
|
||||
- **RDS / SAGE-SQL:** RDS grace period reset. GPO cert distribution pending. RDS CALs purchase needed long-term.
|
||||
- **MFA enforcement ongoing** — 19 users were still not enrolled as of April 4 enforcement date; current count unverified.
|
||||
|
||||
- **MFA enforcement ongoing** — 19 users were not enrolled as of April 4 enforcement date; current enrollment count unverified.
|
||||
|
||||
- **C2 IP blocks need permanence:** Iptables rules on UDM (80.76.49.18, 45.88.91.99) need to be added to permanent UniFi UI block list.
|
||||
|
||||
- **UDM inbound SIP port-forward:** Recommended to add matching rule in UniFi UI (current on_boot.d script covers reboots; UI rule is belt-and-suspenders).
|
||||
|
||||
---
|
||||
|
||||
@@ -424,10 +494,17 @@ As of 2026-06-04:
|
||||
| 2026-05-04 | Howard onsite — lobby phone offline (VLAN misconfiguration on D1-Server-Room port 1 → fixed to VLAN 100). |
|
||||
| 2026-05-06 | SAGE-SQL RDS issues resolved — grace period reset, SSL cert replaced, TSGateway disabled, RemoteApp permission prompts fixed. |
|
||||
| 2026-05-12 | Pipeline audit + email notifications implemented (Graph API). jlohr forwarding configured (ntirety.com → mike@). DKIM keys rotated. |
|
||||
| 2026-06-01 | AOI optical-inspection XP PC isolated onto VLAN 2 (mydata/SMT) @ 192.168.1.175; `aoibackup` SMB1 share created on D2TESTNAS locked to the XP only; other NAS shares set to deny the XP. D2TESTNAS confirmed Debian 13 / Samba 4.22.6 (repurposed Netgear ReadyNAS); vault + wiki OS corrected. Mike: AOI may see all of SMT; optional company-LAN/Internet block for the XP still pending. |
|
||||
| 2026-06-01 | Chauncey Bell (cbell) M365 verified — active mailbox, licensed Microsoft 365 Business Standard (full Office + Exchange); AD password reset on AD2 (synced user, OU=Azure_Users), signed into Office. Bobbi's Outlook printing fixed by switching to Outlook (Classic). Ticket #32364 (0.5 hr onsite). |
|
||||
| 2026-06-02 | Syncro asset reconciliation (78 assets): 20 keep / 21 save+flag / 28 remove / 9 verify. Root cause identified: fleet-wide Syncro agent break ~2025-10-06 silenced ~half the fleet while boxes stayed online (visible in ScreenConnect). Dataforth confirmed phasing off Bitdefender (only 4 of 57 GravityZone endpoints actively managed; 53 in Deleted folder). GUI delete list and 5-step todo tree handed to Howard. Move to metered billing pending cleanup. ScreenConnect API auth pattern documented (CTRLAuthHeader raw secret + Origin). |
|
||||
| 2026-06-04 | SP1366 MAQ20 manufacturing print recovery — 19/20 PDFs for revisions E–H restored to AD2 from HGHAUBNER's pre-attack backup (D:\DF C-Drive) via GuruRMM user_session + GPO-mapped Q: drive. Root cause of loss: incomplete 10/1/2025 recovery restore. MSP360 file backup (`faad5a67`) independently cross-validated (both sources agree: 19/20 present). Syncro #32385, 1.0 hr remote, prepaid $0, resolved. GuruRMM fleet grew 13 → 45 agents (AD1, FILES-D1, SAGE-SQL, DF-HYPERV-B, DF-SVR-D2-Sync, eng-dev-server, + many workstations enrolled). WizTree backup-side CSV captured for migration-gap diff; diff deferred to 2026-06-05. AD1 Files backup command prepared (not run). |
|
||||
| 2026-06-01 | AOI optical-inspection XP PC isolated onto VLAN 2 (mydata/SMT) @ 192.168.1.175; `aoibackup` SMB1 share created on D2TESTNAS locked to the XP only; other NAS shares set to deny the XP. D2TESTNAS confirmed Debian 13 / Samba 4.22.6 (repurposed Netgear ReadyNAS); vault + wiki OS corrected. |
|
||||
| 2026-06-01 | Chauncey Bell (cbell) M365 verified — active mailbox, licensed M365 Business Standard; AD password reset on AD2 (synced user, OU=Azure_Users), signed into Office. Bobbi's Outlook printing fixed. Ticket #32364 (0.5 hr onsite). |
|
||||
| 2026-06-02 | Syncro asset reconciliation (78 assets): 20 keep / 21 save+flag / 28 remove / 9 verify. Root cause identified: fleet-wide Syncro agent break ~2025-10-06 silenced ~half the fleet while boxes stayed online (visible in ScreenConnect). Dataforth confirmed phasing off Bitdefender. Cleanup list handed to Howard. |
|
||||
| 2026-06-04 | SP1366 MAQ20 manufacturing print recovery — 19/20 PDFs for revisions E–H restored to AD2 from HGHAUBNER's pre-attack backup via GuruRMM user_session + GPO-mapped Q: drive. Root cause of loss: incomplete 10/1/2025 recovery restore. Syncro #32385, 1.0 hr remote, prepaid $0, resolved. GuruRMM fleet grew 13 → 45 agents. WizTree backup-side CSV captured for migration-gap diff (deferred). |
|
||||
| 2026-06-05 | AD1 Files backup plan created via GuruRMM remote command (cbb.exe, NBF, 180-day retention, daily 2 AM, covers C:\Engineering + C:\Shares\ITSvc). AD1 now has both image and file plans matching AD2. |
|
||||
| 2026-06-05 | **Mailprotector CloudFilter discovered** as Dataforth's outbound delivery layer (atop INKY + Exchange Online). Email from Georg Haubner was held by Mailprotector due to INKY "Annotation" transport rule. Released manually. New `/mailprotector` skill built and committed. |
|
||||
| 2026-06-05 | Georg Haubner's Power Monitor SPA analyzed (vanilla-JS, AI-built). Gateway architecture designed for PWM.dataforth.com demo. Parked pending Mike↔Georg conversation. |
|
||||
| 2026-06-08–09 | **Total Dataforth phone outage.** Outbound failed (FirstDigital SBC ignoring OPTIONS → trunk Unavailable); inbound never worked (no SIP port-forward existed). Fixed: `qualify_frequency=0` in pjsip DB; `PJSip.class.php` line 504 re-patched; `/data/on_boot.d/30-freepbx-sip-forward.sh` added (SIP-only DNAT, source-locked 66.7.123.0/24). Two-way audio verified. UDM vault corrected. Syncro #32392, 1.0 hr emergency (×1.5 rate) remote, prepaid. |
|
||||
| 2026-06-10 | **Shares & Permissions Phase 0 complete.** Read-only ACL audit of all 8 business shares: all grant Domain Users/Everyone Full or Modify; no department security groups exist; Payroll/OSHA/PO/accounting data open to all employees. Phase 1 (client input) pending discovery email to Dan Center. |
|
||||
| 2026-06-17 | AD2 identity.json + Python 3.12.8 installed. `CLAUDE.dataforth.md` created for AD2 context file (relocated from in-line `.claude/CLAUDE.md` edits to maintain clean fork). |
|
||||
| 2026-06-18 | **DSCA33/45 certs recovered via Hoffman API** — 56 model templates mined, 1,452 new DSCA33/45 certs published on AD2 (0 overwrites). Root-caused `parseRawData` bug affecting 8B/5B/SCM families. 136 8B/5B/SCM templates mined from Hoffman and handed to AD2 for wiring. TestDataDB UI redesigned and deployed on AD2 (cert-fit, publish chips, push toasts, full-screen inspector). AD2 SSH PMTU blackhole diagnosed (GURU-5070 adapter MTU 1500 vs tunnel ~1424) and fixed (MTU 1400). Syncro #32441. |
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user