From b93c9d9e9425dff433b8b0067610a8b8a3ae8823 Mon Sep 17 00:00:00 2001 From: Mike Swanson Date: Thu, 4 Jun 2026 19:29:28 -0700 Subject: [PATCH] sync: auto-sync from GURU-5070 at 2026-06-04 19:29:23 Author: Mike Swanson Machine: GURU-5070 Timestamp: 2026-06-04 19:29:23 --- .../session-logs/2026-06-04-session.md | 36 +++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/clients/glaztech/session-logs/2026-06-04-session.md b/clients/glaztech/session-logs/2026-06-04-session.md index 746ebbd..d4f3ad5 100644 --- a/clients/glaztech/session-logs/2026-06-04-session.md +++ b/clients/glaztech/session-logs/2026-06-04-session.md @@ -144,3 +144,39 @@ Executed a large batch of Glaztech remediation through GuruRMM. WWW was already - On-WWW logs: `C:\temp\{vs_uninstall, devtools_groupB, groupC, acl_fix, acl_fix2, sa_via_tom, webconfig_apply}.log`; on DCs/SQL: `C:\temp\timefix_*.log`. - Local scripts (this machine): `C:\Users\guru\AppData\Local\Temp\grok_glaztech\*.ps1`. - Coord locks held: `clients/glaztech:glaztech/domain-time` (61cd25f2), `clients/glaztech:WWW/devtools-removal` (c4226bac). + +--- + +## Update: 19:28 PT — msdb plaintext-cred removal (Credential+Proxy) — PAUSED mid-flight (stopping for the night) + +### Summary +Began removing the plaintext `glaztech\administrator` password from the **11 msdb backup-copy job steps** via the user-approved **SQL Credential + Agent CmdExec proxy** method. Phase 1 (create credential + proxy, test, snapshot, generate proposed commands) ran — but the **proxy authentication failed**. Decision on how to finish is deferred to next session. **The 11 production backup steps were NOT modified; backups run exactly as before.** User chose to leave the staged objects in place tonight (not cleaned up). + +### EXACT STATE on GTI-INV-SQL (resume point) +- **11/11 backup-copy steps UNCHANGED** (still inline `net use ... /user:glaztech\administrator `); **0 converted**. Backups functional, untouched. +- **3 inert objects created in Phase 1 (NOT wired to any job, no effect on backups) — LEFT IN PLACE:** + - Credential **`glaztech-backup-share`** (IDENTITY=`glaztech\administrator`, pw encrypted) — UNUSED. + - Proxy **`ACG-BackupShareProxy`** (granted to CmdExec) — UNUSED. + - Rollback table **`msdb.dbo.acg_jobstep_backup_20260604`** (11 rows = original step defs, **contains the plaintext pw**). + - Temp job `ACG-ProxyTest` already deleted. +- **NEXT SESSION FIRST STEP:** decide method (below), then either reuse these or **DROP all 3** (`DROP CREDENTIAL [glaztech-backup-share]`; `sp_delete_proxy 'ACG-BackupShareProxy'`; `DROP TABLE msdb.dbo.acg_jobstep_backup_20260604`) to return to pristine. The snapshot table holds plaintext — drop it once we no longer need rollback. + +### Key Findings +- **Embedded backup password is LIVE, not stale** — `net use \\192.168.8.52\sql_backup /user:glaztech\administrator ` returned "command completed successfully"; share reachable. Current copies work. +- **Proxy auth fails with system error 1326 ("user name or password is incorrect")** despite the password being correct → a **logon-type mismatch**: `net use` = network logon (validated by the remote file server); a SQL Agent proxy does a **batch logon locally on GTI-INV-SQL**, resolving `glaztech\administrator` differently. NOT stale-pw, NOT a logon-right error (that'd be 1385). +- **The backup steps already "Executed as user: GLAZTECH\administrator"** — the **SQL Agent service account IS the domain admin**. So the simplest fix needs no stored credential at all (see decision). +- **Pre-existing bug (separate):** job **"Glaz Prod Differential (Hourly) to 8.62"** — **steps 13 & 14 FAIL** with `Incorrect syntax near 'EXEC [192.168.8.212,3436].msdb.dbo.sp_start_job N'` / `[192.168.8.52,3436]...` (broken linked-server job trigger). Backup-copy steps 11/12 succeed, but the job reports failure on 13/14. Worth investigating. + +### DECISION PENDING (resume here) — how to remove the plaintext +- **RECOMMENDED: CmdExec-as-Agent.** Convert the 11 steps to CmdExec running the `copy` only (no `net use`, no creds). **Removes plaintext, stores NO password anywhere.** Works because the Agent = `GLAZTECH\administrator` (domain admin w/ share access). Generated commands already validated (copy cmds extract cleanly; **disabled step 1 → `rem` no-op**). Caveat: leans on the Agent's domain-admin privilege — but backups already depend on that today (zero regression); redo with a dedicated account + credential when the Agent is de-privileged. +- **ALT: troubleshoot the proxy** (try IDENTITY `administrator@glaztech.com` UPN, or batch-logon nuance) to keep the decoupled credential/proxy design. +- The generated new per-step commands (no secrets) are in the Phase-1 output / `C:\temp\msdb_cred_phase1.log` on WWW. + +### Broader pending (Glaztech remediation backlog) +- **Rotate `glaztech\administrator`** — Mike coordinating with **Steve** (deferred). After rotation: `ALTER CREDENTIAL`/cmdkey or re-do CmdExec auth as needed. +- After plaintext removal → **disable `xp_cmdshell`** (was gated on these jobs). +- **Disable TLS 1.0/1.1 on WWW** (needs reboot); **full web-root `Everyone` sweep** (static, low pri); **seize/clean Schema-master FSMO off dead NS4**; **de-privilege the SQL Agent account**; investigate the hourly-job steps 13/14 bug. + +### Reference +- Scripts (this machine): `C:\Users\guru\AppData\Local\Temp\grok_glaztech\{fix_msdb_phase1, diag_netuse, fix_www_acl2, sa_via_tom, fix_pdc_time, fix_dc1_time, fix_member_time, sched_webconfig, fix_apply}.ps1`. +- On GTI-INV-SQL (reach via WWW `tom` connection from `D:\web\glaztech_4\Web.config`, server `192.168.8.62`): cred `glaztech-backup-share`, proxy `ACG-BackupShareProxy`, table `msdb.dbo.acg_jobstep_backup_20260604`.