sync: auto-sync from GURU-5070 at 2026-06-03 11:52:45
Author: Mike Swanson Machine: GURU-5070 Timestamp: 2026-06-03 11:52:45
This commit is contained in:
@@ -0,0 +1,90 @@
|
||||
# Session: 2026-06-03 - Remove SBS from mspbackups for Glaztech
|
||||
|
||||
**User:** Mike Swanson (mike)
|
||||
**Machine:** GURU-5070
|
||||
**Role:** admin
|
||||
|
||||
## Session Summary
|
||||
|
||||
Removed the SBS computer entry from MSP360/mspbackups for Glaztech Industries. This is the computer with MSP360 UserID d425fbbe-43f6-4fb7-8695-a9296b762a3b, ComputerName SBS, backup destination ACG-GLAZTECH (233GB used).
|
||||
|
||||
Followed locate-confirm-execute: used GrepAI, MSP360 API (as documented in 2026-06-01 session log), vault, B2 for verification (read-only).
|
||||
|
||||
Disabled the user via MSP360 API PUT /api/Users Enabled=false, then DELETE /api/Users/{id} (both 200).
|
||||
|
||||
This should trigger chained B2 data deletion for prefix MBS-d425fbbe-43f6-4fb7-8695-a9296b762a3b/CBB_SBS/ in ACG-GLAZTECH bucket. Did not touch B2 data directly.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- Removal only via MSP360 API (computer entry), per user clarification that license is expired, and removal triggers B2.
|
||||
|
||||
- Used MSP360 API with fresh token from /api/Provider/Login using vault creds.
|
||||
|
||||
- Claimed coord lock on clients/glaztech mspbackups/SBS.
|
||||
|
||||
- Snapshot: MSP360 user state (before/after Enabled), B2 files under prefix (still present), Monitoring still showed plan.
|
||||
|
||||
- No RMM agent found for SBS in glaztech (searches in code and API queries showed other GTI- machines).
|
||||
|
||||
- Documented in clients/glaztech/session-logs/
|
||||
|
||||
## Problems Encountered
|
||||
|
||||
- No active LicenseID found for SBS in current /api/Licenses (consistent with expired trial from report).
|
||||
|
||||
- MSP360 DELETE /api/Users/{id} succeeded 200 but entry still queryable with Enabled=false (soft delete/queue like vland example in docs).
|
||||
|
||||
- B2 deletion is asynchronous via MSP360 portal processing.
|
||||
|
||||
## Configuration Changes
|
||||
|
||||
- MSP360: User d425fbbe-43f6-4fb7-8695-a9296b762a3b set Enabled=false via PUT, then DELETE called.
|
||||
|
||||
## Commands & Outputs
|
||||
|
||||
(See tool calls in transcript for full API queries, vault, b2 lists, coord lock/release.)
|
||||
|
||||
Key:
|
||||
|
||||
- Vault get-field msp-tools/msp360-api.sops.yaml credentials -> login kY9PvDdWki password p9wzJFRT8nC6VfFz6UDZ
|
||||
|
||||
- Login POST -> token
|
||||
|
||||
- GET /api/Users -> found SBS entry
|
||||
|
||||
- GET /api/Monitoring -> showed plan for SBS, last backup 2026-05-08
|
||||
|
||||
- PUT /api/Users {"ID": "d425fbbe-43f6-4fb7-8695-a9296b762a3b", "Enabled": false} -> 200
|
||||
|
||||
- Re-GET user -> Enabled: false
|
||||
|
||||
- DELETE /api/Users/d425fbbe-43f6-4fb7-8695-a9296b762a3b -> 200
|
||||
|
||||
- B2: py ... b2.py files ACG-GLAZTECH --prefix MBS-d425fbbe-43f6-4fb7-8695-a9296b762a3b/CBB_SBS --limit 10 -> listed files (still there)
|
||||
|
||||
- Coord: POST lock, later DELETE release.
|
||||
|
||||
## Pending
|
||||
|
||||
- Monitor MSP360 portal / B2 for deletion progress of the SBS computer and its backup data in ACG-GLAZTECH.
|
||||
|
||||
- If RMM has mapping for glaztech SBS backup, reconcile (no current agent found).
|
||||
|
||||
- Update glaztech wiki/clients/glaztech.md if infrastructure changes documented.
|
||||
|
||||
## References
|
||||
|
||||
- MSP360 API usage: session-logs/2026-06-01-session.md (detailed commands, endpoints, vault, cross-ref)
|
||||
|
||||
- B2 mspbackups: .claude/skills/b2/SKILL.md
|
||||
|
||||
- Client context: wiki/clients/glaztech.md , clients/glaztech/
|
||||
|
||||
- Coord lock: released dcf3c54c-840a-4131-a653-69857bdeac45
|
||||
|
||||
- SPEC-023 for future automation: projects/msp-tools/guru-rmm/docs/specs/SPEC-023-msp360-license-release-on-decommission.md
|
||||
|
||||
## User
|
||||
- **User:** Mike Swanson (mike)
|
||||
- **Machine:** GURU-5070
|
||||
- **Role:** admin
|
||||
106
clients/glaztech/session-logs/2026-06-03-session.md
Normal file
106
clients/glaztech/session-logs/2026-06-03-session.md
Normal file
@@ -0,0 +1,106 @@
|
||||
# Session Log — 2026-06-03 — Glaz-Tech Industries
|
||||
|
||||
## User
|
||||
- **User:** Mike Swanson (mike)
|
||||
- **Machine:** GURU-5070
|
||||
- **Role:** admin
|
||||
|
||||
---
|
||||
|
||||
## Session Summary
|
||||
|
||||
Worked a series of Glaztech web/payment issues on server `WWW` (192.168.8.72 / public 65.113.52.88), all via the GuruRMM agent, ending in a full security assessment of the website and its databases.
|
||||
|
||||
First, diagnosed why `shannon@glaztech.com` was not receiving MailProtector quarantine digests (sent as `noreply@azcomputerguru.com`). A `Get-MessageTraceV2` (Exchange Operator app, tenant 82931e3c) over the last 10 days showed 629 digests delivered to ~60 recipients and **zero to Shannon**. Confirmed Shannon is a real, active mailbox receiving normal mail. Conclusion: not an Exchange/EOP problem (the morning SCL-bypass rule is irrelevant to her) — MailProtector is not generating a digest for her; the fix is on the MailProtector side (provision her / enable the Spam Summary). Vault has no MailProtector credentials, so it is a manual partner-portal task.
|
||||
|
||||
Handled an emergency apex 404: `glaztech.com` returned 404 (HTTP.sys) while `www.glaztech.com` returned 200, because IIS site `glaztech_new` had a host-header binding only for `www`. Added apex bindings (http:80 + https:443, reusing the existing SAN cert that already covers both names). Then added a URL Rewrite HTTP→HTTPS 301 redirect with a `/.well-known/acme-challenge/` exclusion so Let's Encrypt (Certify The Web, HTTP-01) renewal is unaffected. Created and invoiced ticket #32376 (1h remote, prepaid).
|
||||
|
||||
Diagnosed and fixed the CyberSource credit-card payment outage ("Could not create SSL/TLS secure channel," 20 attempts / 0 success across all offices). Root cause: CyberSource (PNC processor) disabled TLS 1.0/1.1; the .NET payment library on Server 2019 negotiated old TLS. Applied `SchUseStrongCrypto=1` + `SystemDefaultTlsVersions=1` to both `.NETFramework\v4.0.30319` hives and recycled the `glaztech_new` app pool. Verified via the payments DB that card payments resumed (a real credit-card approval at 09:36 after the fix; eCheck path was never affected). Created and invoiced ticket #32377 (1.5h emergency remote, prepaid).
|
||||
|
||||
Performed a deep security assessment (read-only; no card numbers or passwords retrieved). Findings are severe: full PANs and CVV stored in plaintext, ~9,000+ plaintext passwords, SQL injection via a fake `quo()` escaper in payment pages, reflected XSS, debug/error disclosure, the production server doubling as a dev workstation, remote-access sprawl (incl. RealVNC 4.2.8), and a listener accepting TLS 1.0/1.1. Produced two reports and a high-priority client ticket (#32378) with a plain-language, point-by-point public writeup for Steve/Tom. Also traced why cards are stored (card-on-file invoice auto-pay via `gt_auto_process_2020.dll`) and confirmed the exposure is contained to the web app's databases (not Sage 100). Billed 1h remote to #32378 and left it open for remediation.
|
||||
|
||||
---
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **MailProtector digest = MailProtector-side issue.** Message trace proved Shannon gets no digest at all; the EOP SCL-bypass rule (created earlier) only helps users whose digests were being spam-filtered. No Exchange change made for Shannon.
|
||||
- **Apex fix = add binding, not redirect.** Restored apex by adding host-header bindings to the existing site (apex serves same content as www) — fastest restore; cert already SAN-covered apex so no cert change.
|
||||
- **HTTP→HTTPS redirect with ACME exclusion.** Used IIS URL Rewrite (installed) with a `/.well-known/acme-challenge/` negate condition so Certify/Let's Encrypt HTTP-01 renewal is not broken. 301 permanent.
|
||||
- **Payment fix applied server-side first.** Set the .NET TLS keys + app-pool recycle (the standard fix). Verified success via the payments DB rather than waiting on Tom; the app-code `ServicePointManager.SecurityProtocol = Tls12` fallback was staged but not needed (DB confirmed live card approvals post-fix).
|
||||
- **Security findings classified, never exfiltrated.** All card/password determinations used aggregate `CASE`/`COUNT` only — no PAN or password value was selected or displayed.
|
||||
- **#32378 left OPEN (In Progress)** at Mike's direction for likely remediation work; billed but not resolved/closed.
|
||||
- **Tokenization is the card remediation** — preserves the auto-pay feature while removing all stored PAN/CVV.
|
||||
|
||||
---
|
||||
|
||||
## Problems Encountered
|
||||
|
||||
- **`Get-MessageTraceV2` not exposed via `adminapi/InvokeCommand` initially** — investigator-exo token 401'd; the exchange-op token works and the cmdlet runs via InvokeCommand with a ≤10-day window constraint. The ExchangeOnlineManagement PowerShell module is NOT installed on GURU-5070 (used the REST InvokeCommand path instead).
|
||||
- **PowerShell quoting/parse errors** in several RMM probes (escaped quotes inside `$(...)`, `foreach | Format-Table` empty-pipe, DBNull comparison). Resolved by collecting into variables, moving counts out of inline subexpressions, and DBNull-safe formatters.
|
||||
- **Local apex test 404'd** because the binding is IP-scoped to 192.168.8.72 and `localhost` is 127.0.0.1 — verified externally instead (301/200/acme-404 all correct).
|
||||
- **Syncro long-comment heredoc broke bash** (apostrophes inside `$(cat <<EOF)`). Resolved by writing the body to a file with a direct heredoc redirect, collapsing newlines to `<br>`, and posting via `jq -n --rawfile`.
|
||||
|
||||
---
|
||||
|
||||
## Configuration Changes
|
||||
|
||||
**On server WWW (192.168.8.72) — via GuruRMM:**
|
||||
- IIS site `glaztech_new`: added bindings `http/192.168.8.72:80:glaztech.com` and `https/192.168.8.72:443:glaztech.com` (apex; reuses cert thumbprint `2684b127f039be5cb5adea6f3f616c111824e4dc`).
|
||||
- `D:\web\glaztech_4\web.config`: inserted URL Rewrite rule "HTTP to HTTPS" (301, host-preserving, with `/.well-known/acme-challenge/` exclusion) as the first rule. Backup at `D:\web\glaztech_4\web.config.bak-20260603-090701`.
|
||||
- Registry: `SchUseStrongCrypto=1` + `SystemDefaultTlsVersions=1` (DWORD) added to BOTH `HKLM\SOFTWARE\Microsoft\.NETFramework\v4.0.30319` and `HKLM\SOFTWARE\WOW6432Node\Microsoft\.NETFramework\v4.0.30319`; recycled app pool `glaztech_new`.
|
||||
|
||||
**Repo (ClaudeTools):**
|
||||
- Created `clients/glaztech/reports/2026-06-03-pci-cardholder-data-finding.md`
|
||||
- Created `clients/glaztech/reports/2026-06-03-website-security-assessment.md` (incl. attack-path + card data-flow sections)
|
||||
|
||||
**Syncro tickets:** #32376 (apex 404 + redirect), #32377 (payment TLS), #32378 (security assessment — left open).
|
||||
|
||||
---
|
||||
|
||||
## Credentials & Secrets
|
||||
|
||||
- **MailProtector / CloudFilter partner portal** — NO credentials in the vault. ACG manages it via manual partner-portal login. (Action: consider vaulting for future automation.)
|
||||
- **MSP360 API** (unrelated, used 6/2): `msp-tools/msp360-api.sops.yaml`.
|
||||
- **Exchange Operator app** (message trace): vault `msp-tools/computerguru-exchange-operator.sops.yaml`, app `b43e7342-5b4b-492f-890f-bb5a4f7f40e9`, tenant `82931e3c-de7a-4f74-87f7-fe714be1f160`; token via `get-token.sh <tenant> exchange-op`.
|
||||
- **Glaztech SQL** (observed in site Web.config, NOT vaulted): server `192.168.8.62,3436`, login `tom` (password in `D:\web\glaztech_4\Web.config` connectionStrings), 15 per-office databases (`glaz_prod`, `glaz_prod_phx`, …, `glaz_prod_corp`) + `mas_gti` (Sage 100) + `qqest` (TimeForce).
|
||||
|
||||
---
|
||||
|
||||
## Infrastructure & Servers
|
||||
|
||||
- **WWW** — Windows Server 2019 Standard, build 10.0.17763.8755 (patched through May 2026). Local IP 192.168.8.72, public 65.113.52.88. IIS 10.0, .NET 4.8. Site `glaztech_new` → `D:\web\glaztech_4` (full VB.NET source on disk, not precompiled). LE cert `CN=www.glaztech.com`, SAN glaztech.com + www, expires 2026-08-19, via Certify The Web (HTTP-01). GuruRMM agent id `455a1bc7-1c29-42bc-b597-fa1e64f08eec`.
|
||||
- **SQL** — `192.168.8.62,3436`, 15 office DBs + Sage 100 (`mas_gti`) + TimeForce (`qqest`).
|
||||
- **Payment gateway** — CyberSource REST (`api.cybersource.com/pts/v2/payments`) as PNC merchant processor (card path); CyberSource SOAP toolkit for eCheck/ACH (`cybs.log`).
|
||||
- **Email/spam** — MailProtector inbound filter `glaztech-com.inbound.emailservice.io`; digests from `noreply@azcomputerguru.com`.
|
||||
|
||||
---
|
||||
|
||||
## Commands & Outputs
|
||||
|
||||
- Apex 404: `curl -I https://glaztech.com` → 404 `Microsoft-HTTPAPI/2.0`; post-fix → 200; `/.well-known/acme-challenge/probe` over http → 404 plain (renewal safe).
|
||||
- Payment TLS test (WWW → api.cybersource.com): TLS1.0/1.1 FAIL (SSPI), **TLS1.2 OK**; post-fix DB shows credit-card approval id 84455 at 09:36 (`web_payment_header`, glaz_prod_corp).
|
||||
- Auth proc `get_web_accesslevel`: `where acct_no=@username and web_password=@passwd` (plaintext password compare).
|
||||
- `quo()`: `Return "'" + stext + "'"` (no escaping → SQLi).
|
||||
- `get_cc_data`: `select * from cc_file where acct_no=@acctno` (returns full PAN+CVV; IDOR-shaped).
|
||||
- Card storage classification: `cc_file` ~780 rows total (plaintext PAN, 0 encrypted); `cc_file.cc_code` 50/54 CVV in Tucson; `cof_payments_header` Phoenix 14,496 rows / 11,794 plaintext PAN; `web_security` ~9,000+ plaintext passwords (corp 6,017 + tuc 3,012), 0 hash-like.
|
||||
- Message trace (Shannon): 629 noreply digests in 10 days to ~60 recipients, 0 to shannon@glaztech.com.
|
||||
|
||||
---
|
||||
|
||||
## Pending / Incomplete Tasks
|
||||
|
||||
- **MailProtector:** provision/enable Shannon's Spam Summary in the MailProtector partner portal (manual). The earlier `gtimail@glaztech.com` daily digest failure (Priority-1 reject rule) is still open with Steve.
|
||||
- **#32378 remediation (open ticket):** quick wins ready on Mike's go — purge stored CVV (`cc_file.cc_code`, backup-first, needs explicit sign-off), set `debug="false"` + `customErrors="On"`, remove RealVNC 4.2.8 + stale ScreenConnect, disable TLS 1.0/1.1. Then: hash passwords + stop emailing them, parameterize payment-page SQL (kill `quo()`), tokenize card-on-file (CyberSource vault), separate dev from prod, least-privilege DB.
|
||||
- **Payment fix:** monitor that card successes continue; Tom code fallback (`SecurityProtocol = Tls12`) staged if needed.
|
||||
- **Backups contain plaintext cards** — cleanup must address DB backups, not just live data.
|
||||
|
||||
---
|
||||
|
||||
## Reference Information
|
||||
|
||||
- Tickets: #32376 (id 112107224, apex 404+redirect, Resolved, 1h remote), #32377 (id 112109440, CyberSource TLS, Resolved, 1.5h emergency remote), #32378 (id 112111185, security assessment, **In Progress**, 1h remote).
|
||||
- Glaztech Syncro customer id 143932; **prepaid block 26.5 → 22.25 hrs** today.
|
||||
- Reports: `clients/glaztech/reports/2026-06-03-pci-cardholder-data-finding.md`, `…-website-security-assessment.md`.
|
||||
- GuruRMM client id `d857708c-5713-4ee5-a314-679f86d2f9f9`; WWW agent `455a1bc7-1c29-42bc-b597-fa1e64f08eec`.
|
||||
- TLS fix pattern (legacy .NET → modern gateway): `SchUseStrongCrypto=1` + `SystemDefaultTlsVersions=1` both `.NETFramework\v4.0.30319` hives + app-pool recycle.
|
||||
- LE-safe HTTP→HTTPS redirect: URL Rewrite rule with `{REQUEST_URI}` `^/\.well-known/acme-challenge/` negate condition.
|
||||
Reference in New Issue
Block a user