Mike Swanson
fe3b5b0382
- Session manager: self-service RDP session reset for Dataforth users (Default.aspx + web.config) - WORKITEMS.md: shared task board for Mike/Howard with @tagging, syncs via Gitea - Session log: deployment deferred due to VPN connectivity issues Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
264 lines
13 KiB
Markdown
264 lines
13 KiB
Markdown
# Session Log — 2026-04-17
|
||
|
||
## User
|
||
- **User:** Mike Swanson (mike)
|
||
- **Machine:** DESKTOP-0O8A1RL
|
||
- **Role:** admin
|
||
- **Mode:** client/infra (mixed)
|
||
|
||
## Session Summary
|
||
|
||
Full day of client security work + infrastructure + tooling. Major items: Jupiter OwnCloud migration confirmed complete, Glaztech phishing incident (32 messages purged, MX/DMARC/EFC hardened), MVAN DMARC added, Syncro PSA integration built, GoDaddy API onboarded, jparkinson DNS fixes, Neptune access issues.
|
||
|
||
## Work Completed
|
||
|
||
### 1. Jupiter OwnCloud migration — confirmed complete
|
||
- rsync finished at 22:59 MST (2h49m total for ~750G uncompressed)
|
||
- Cache dropped from 82% (756G) to 34% (311G)
|
||
- MariaDB-Official + Discourse running healthy 7+ hours post-migration
|
||
- OwnCloud VM running, share config changed to `shareUseCache="no"`
|
||
|
||
### 2. Glaztech phishing incident — full remediation
|
||
**Two phishing campaigns bypassing MailProtector via exposed M365 MX record:**
|
||
|
||
Campaign 1: "ATTN: MaiIbox Password Login Expire" (spoofed alexander@, from 23.94.30.18 ColoCrossing)
|
||
Campaign 2: "HR Paperwork – Awaiting Completion Approval" (spoofed enrique@, from 86.38.225.18)
|
||
|
||
Both: SPF FAIL, DKIM none, DMARC FAIL (p=none), SCL 1 (M365 didn't flag), connected directly to MX 10 bypassing MailProtector.
|
||
|
||
**Actions taken:**
|
||
- Removed MX 10 (glaztech-com.mail.protection.outlook.com) from DNS on IX
|
||
- Updated DMARC from p=none to p=reject
|
||
- Enabled Enhanced Filtering for Connectors (EFSkipIPs: MailProtector IPs)
|
||
- Purged 32 messages across 8 mailboxes (alexander, seastman, dominic, jack, bryce, cesar, daryld, holly)
|
||
- Saved forensic .eml + .json samples
|
||
- Onboarded Glaztech to remediation tool (admin consent + Exchange Admin role)
|
||
- Syncro ticket #32165 created + billed
|
||
|
||
**Glaztech tenant:** 82931e3c-de7a-4f74-87f7-fe714be1f160
|
||
**Remediation tool roles:** Exchange Administrator assigned to ComputerGuru - AI Remediation SP
|
||
|
||
### 3. MVAN phishing — DMARC added
|
||
- mvaninc.com had NO DMARC, NO MailProtector, direct M365 MX only
|
||
- Added DMARC p=reject via GoDaddy web GUI (delegate access from MVAN)
|
||
- Syncro ticket #32166 created with notes to client about MailProtector add-on option and other domains needing protection
|
||
- MVAN tenant: 5affaf1e-de89-416b-a655-1b2cf615d5b1 (already consented for remediation tool)
|
||
|
||
### 4. /syncro command — Syncro PSA integration
|
||
Built `/syncro` slash command for ticket management via Syncro REST API.
|
||
|
||
**Key discovery:** Time is added as part of the comment, NOT via separate timer endpoint.
|
||
- `POST /tickets/{id}/comment` with `product_id`, `minutes_spent`, `bill_time_now` fields
|
||
- Timer entries (`/tickets/{id}/timer_entry`) exist but rarely used
|
||
- Invoice creation: `POST /invoices` with `ticket_id` + `customer_id`
|
||
- Invoice line items: `POST /invoices/{id}/line_items`
|
||
|
||
**Labor product IDs:**
|
||
- 1190473 — Labor - Remote Business
|
||
- 26118 — Labor - Onsite Business
|
||
- 26184 — Labor - Emergency or After Hours Business
|
||
- 9269129 — Labor - Prepaid Project Labor
|
||
- 9269124 — Labor - Internal Labor
|
||
- 26117 — Fee - Travel Time
|
||
- 68055 — Labor - Website Labor
|
||
|
||
**Glaztech billing:** Prepaid Hours - Block (product 46303) at $130/hr, 40hr blocks
|
||
|
||
### 5. GoDaddy API — onboarded
|
||
- Created Production API key "RemediationTools"
|
||
- Vaulted at `services/godaddy-api.sops.yaml`
|
||
- Can manage DNS for ACG-owned domains programmatically
|
||
- Delegate domains (client-managed) only accessible via web GUI, NOT API
|
||
- MVAN delegated access accepted but API still returns 403 (known GoDaddy limitation)
|
||
|
||
### 6. jparkinsonaz.com DNS fixes
|
||
- Added DMARC: `p=reject; sp=reject`
|
||
- Added autodiscover: CNAME → mail.acghosting.com
|
||
- Changed A record: 72.194.62.7 (IX) → 67.206.163.124 (Neptune) — mail-only domain, no website
|
||
- Required `pdns_control reload` after zone file edits (regular PowerDNS restart not sufficient)
|
||
- Required `/usr/local/cpanel/scripts/dnscluster synczone` for cluster propagation
|
||
- Serial format: epoch-based (NOT YYYYMMDDNN) — use incrementing epoch or zone check fails
|
||
- Neptune certbot for autodiscover failing — likely DNS propagation delay (14400s TTL on old A)
|
||
|
||
### 7. desertrat.com DNS audit
|
||
- MX: mail.desertrat.com → 162.248.93.81 (ACG WebSvr/NFOservers VDS, NOT MailProtector)
|
||
- SPF: includes spf.wdsolutions.com (WD Solutions/SmarterMail), uses ~all (softfail)
|
||
- DMARC: MISSING
|
||
- DNS: AWS Route 53 (not IX or GoDaddy)
|
||
- Needs: DMARC p=reject, SPF ~all → -all, eventual migration to IX + MailProtector
|
||
- Recommended SPF with MailProtector added: `v=spf1 +a +mx +ip4:162.248.93.233 +ip4:162.248.93.81 +include:spf.wdsolutions.com +include:spf.us.emailservice.io -all`
|
||
|
||
### 8. Neptune password reset — failed
|
||
- Attempted to set jparkinson password to `jP$48504850` on Neptune (jparkinsonaz.com domain)
|
||
- Neptune at 67.206.163.124 (public) / 172.16.3.50 (internal)
|
||
- WinRM from AD2 failed (Kerberos cross-domain), direct WinRM from workstation failed (Negotiate auth error)
|
||
- Internal IP 172.16.3.50 has RDP + WinRM open but auth failed
|
||
- May have caused account lockout — user handling via separate Claude session on Neptune directly
|
||
- ACG\administrator creds: `Gptf*77ttb##`
|
||
|
||
## Credentials
|
||
|
||
### GoDaddy API (Production)
|
||
- Key: `2wXWWFcuYk_2RGxdvpe1WZV2yPMvNLGEe`
|
||
- Secret: `5pQZs7H9WY7dwh59XsJMNr`
|
||
- Auth header: `Authorization: sso-key 2wXWWFcuYk_2RGxdvpe1WZV2yPMvNLGEe:5pQZs7H9WY7dwh59XsJMNr`
|
||
- Vault: `services/godaddy-api.sops.yaml`
|
||
|
||
### Syncro PSA
|
||
- API Key: `T259810e5c9917386b-52c2aeea7cdb5ff41c6685a73cebbeb3`
|
||
- Base: `https://computerguru.syncromsp.com/api/v1`
|
||
- Vault: `msp-tools/syncro.sops.yaml`
|
||
|
||
### Glaztech M365
|
||
- Tenant ID: `82931e3c-de7a-4f74-87f7-fe714be1f160`
|
||
- Remediation tool consented + Exchange Admin role assigned
|
||
|
||
### MVAN M365
|
||
- Tenant ID: `5affaf1e-de89-416b-a655-1b2cf615d5b1`
|
||
- Already consented for remediation tool
|
||
|
||
### Neptune
|
||
- Public: 67.206.163.124
|
||
- Internal: 172.16.3.50
|
||
- Creds: `ACG\administrator` / `Gptf*77ttb##`
|
||
- jparkinson target password: `jP$48504850`
|
||
|
||
### IX server
|
||
- 172.16.3.10, root, `Gptf*77ttb!@#!@#`
|
||
- PowerDNS, cPanel, zone files at `/var/named/`
|
||
- Cluster sync: `/usr/local/cpanel/scripts/dnscluster synczone <domain>`
|
||
|
||
## DNS Changes Made Today
|
||
|
||
| Domain | Record | Before | After | Server |
|
||
|---|---|---|---|---|
|
||
| glaztech.com | MX 10 | glaztech-com.mail.protection.outlook.com | REMOVED | IX |
|
||
| glaztech.com | _dmarc TXT | p=none | p=reject; sp=reject | IX |
|
||
| mvaninc.com | _dmarc TXT | (missing) | p=reject; sp=reject | GoDaddy (web GUI) |
|
||
| jparkinsonaz.com | _dmarc TXT | (missing) | p=reject; sp=reject | IX |
|
||
| jparkinsonaz.com | autodiscover | (missing) | CNAME mail.acghosting.com | IX |
|
||
| jparkinsonaz.com | A (root) | 72.194.62.7 (IX) | 67.206.163.124 (Neptune) | IX |
|
||
|
||
## IX DNS gotchas (learned today)
|
||
|
||
1. **`pdns_control reload <zone>`** needed after zone file edits — full PowerDNS restart doesn't always pick up changes
|
||
2. **Serial format varies** — some zones use epoch (1776xxxxxx), some use YYYYMMDDNN. New serial must be HIGHER than old or changes are ignored.
|
||
3. **DNS cluster sync** required: `/usr/local/cpanel/scripts/dnscluster synczone <domain>` — editing zone files directly doesn't trigger cluster propagation
|
||
4. **Zone file backups** at `/var/named/<domain>.db.bak-YYYYMMDD`
|
||
|
||
## Syncro tickets created
|
||
|
||
| # | Customer | Subject | Time | Status |
|
||
|---|---|---|---|---|
|
||
| 32165 | Glaz-Tech Industries | Email Security - Phishing remediation + MX/DMARC hardening | 1hr (timer, not comment — needs fix) | Invoiced |
|
||
| 32166 | MVAN Enterprises Inc | Email Security - DMARC protection added for mvaninc.com | 30 min Remote Business | Resolved |
|
||
|
||
## Files created/modified
|
||
|
||
- `clients/glaztech/reports/2026-04-17-phishing-incident-report.md`
|
||
- `clients/glaztech/reports/2026-04-17-phishing-ATTN-mailbox-password.eml`
|
||
- `clients/glaztech/reports/2026-04-17-phishing-ATTN-mailbox-password.json`
|
||
- `clients/glaztech/reports/2026-04-17-phishing-HR-paperwork.eml`
|
||
- `clients/glaztech/reports/2026-04-17-hr-paperwork-*.json`
|
||
- `.claude/commands/syncro.md` (new)
|
||
- `D:\vault\services\godaddy-api.sops.yaml` (new)
|
||
|
||
## Pending
|
||
|
||
1. **Neptune jparkinson password** — being handled in separate Claude session on Neptune
|
||
2. **desertrat.com** — needs DMARC + SPF hardening on Route 53 (need AWS access)
|
||
3. **desertrat.com** — long-term migration from WebSvr to IX + MailProtector
|
||
4. **Glaztech ticket #32165** — timer entry created wrong (should be comment+time); fix or rebill in Syncro GUI
|
||
5. **jparkinsonaz.com certbot** — retry once A record propagates (14400s TTL from old IP)
|
||
6. **MVAN other domains** — only mvaninc.com has DMARC; client has other domains needing protection
|
||
7. **GoDaddy delegate API limitation** — can't manage delegate domains via API; need client's own API key for programmatic DNS
|
||
8. **All carry-over items from 2026-04-16** (Howard onboarding, GuruRMM migration drift, Len's deployment, etc.)
|
||
|
||
---
|
||
|
||
## Update: 13:00 — vault fix, Ollama Tailscale, Howard review
|
||
|
||
### Cascades pfSense vault fix
|
||
- Deleted stale `clients/dataforth/cascades-router.sops.yaml` (wrong password `a6A6c6fe`, misfiled under dataforth)
|
||
- Created `clients/cascades-tucson/pfsense-firewall.sops.yaml` with correct password `Th1nk3r^99`
|
||
- Howard caught the discrepancy during Cascades onsite work
|
||
|
||
### Ollama shared via Tailscale
|
||
- Set `OLLAMA_HOST=0.0.0.0:11434` (User env var, persists)
|
||
- Added Windows Firewall rule: port 11434 inbound, restricted to 100.0.0.0/8 (Tailscale subnet only)
|
||
- Verified: `http://100.92.127.64:11434/` → "Ollama is running" via Tailscale IP
|
||
- All 3 models accessible remotely (qwen3:14b, codestral:22b, nomic-embed-text)
|
||
- CLAUDE.md updated: per-machine URL detection (localhost for DESKTOP-0O8A1RL, Tailscale IP for all others)
|
||
- ONBOARDING.md updated: Howard doesn't need local Ollama install
|
||
|
||
### Howard's session reviewed
|
||
- Cascades: folder redirection (primary computer GPO issue) + WiFi (TP-Link USB driver + UniFi roaming)
|
||
- EVS: Win11 right-click menu fix (was actually Mike's session, miscategorized)
|
||
- Vault hygiene: caught wrong Cascades pfSense password — fixed above
|
||
- Ollama: his ARM64 laptop can't run models locally — resolved via Tailscale sharing
|
||
|
||
### jparkinsonaz.com DNS (continued)
|
||
- IX DNS cluster sync required after zone edits: `/usr/local/cpanel/scripts/dnscluster synczone jparkinsonaz.com`
|
||
- `pdns_control reload` needed on top of PowerDNS restart for zone changes to take effect
|
||
- Certbot for autodiscover should work once root A record TTL (14400s) expires and propagates to 67.206.163.124
|
||
|
||
### Credentials (this update)
|
||
|
||
#### Cascades pfSense
|
||
- Host: 192.168.0.1
|
||
- Username: admin
|
||
- Password: `Th1nk3r^99`
|
||
- Vault: `clients/cascades-tucson/pfsense-firewall.sops.yaml`
|
||
|
||
#### Ollama Tailscale access
|
||
- Mike's Tailscale IP: 100.92.127.64
|
||
- Ollama URL: `http://100.92.127.64:11434`
|
||
- Firewall: inbound TCP 11434 from 100.0.0.0/8 only
|
||
- Env var: `OLLAMA_HOST=0.0.0.0:11434` (User scope on DESKTOP-0O8A1RL)
|
||
|
||
---
|
||
|
||
## Update: 20:00 — SAGE-SQL session manager, shared work items
|
||
|
||
### Dataforth SAGE-SQL session manager — built, not yet deployed
|
||
Built self-service session reset web app for Dataforth users on SAGE-SQL (192.168.0.153, Windows Server 2016).
|
||
|
||
**Problem:** Users connect via RemoteApps to SAGE. Sessions hang/disconnect and require IT to remote in and logoff sessions manually.
|
||
|
||
**Solution:** Single-file ASP.NET WebForms app (`Default.aspx` + `web.config`) that:
|
||
- Uses Windows Authentication (auto-identifies domain user, no login needed)
|
||
- Shows only the authenticated user's own RDP/RemoteApp sessions
|
||
- Only allows resetting disconnected ("Disc") sessions, not active ones
|
||
- Confirmation prompt before reset
|
||
- Logs all reset actions to monthly log files at `~/logs/YYYY-MM.log`
|
||
- Dark themed UI
|
||
|
||
**Files:**
|
||
- `clients/dataforth/session-manager/Default.aspx` — full app (server-side C# + HTML/CSS)
|
||
- `clients/dataforth/session-manager/web.config` — IIS config (Windows Auth on, Anonymous off)
|
||
|
||
**Deployment blocked:** VPN connectivity issues — SSH to AD2 times out (ICMP works, TCP blocked), WinRM to SAGE-SQL blocked, RMM API at 172.16.3.30:3001 unreachable. Deferred to next session.
|
||
|
||
**Deployment steps (for tomorrow):**
|
||
1. Create `C:\inetpub\sessions\` on SAGE-SQL
|
||
2. Copy Default.aspx + web.config to that directory
|
||
3. Create IIS application: `New-WebApplication -Name "sessions" -Site "Default Web Site" -PhysicalPath "C:\inetpub\sessions" -ApplicationPool "DefaultAppPool"`
|
||
4. Verify Windows Auth enabled, Anonymous Auth disabled
|
||
5. Test at `http://sage-sql/sessions/`
|
||
6. App pool identity (NetworkService) should have permission to run `logoff` command
|
||
|
||
**WinRM TrustedHosts updated:** Added `192.168.0.153,SAGE-SQL` to local TrustedHosts for future NTLM auth (workstation not domain-joined).
|
||
|
||
### Shared work items board — created
|
||
Created `WORKITEMS.md` at repo root — shared task list that syncs via Gitea.
|
||
- Both Mike and Howard can add/claim/complete items
|
||
- Uses `@mike`/`@howard`/`@unassigned` tagging
|
||
- Populated with all carry-over items from this session and previous days
|
||
- Claude can read/update it on request ("show work items", "add work item: ...")
|
||
|
||
### Network issues (end of day)
|
||
- AD2 (192.168.0.6): ICMP ping works (23-46ms), SSH port 22 times out
|
||
- SAGE-SQL (192.168.0.153): WinRM port 5985 unreachable from workstation
|
||
- RMM server (172.16.3.30:3001): connection times out
|
||
- Likely VPN/firewall filtering TCP but passing ICMP
|