From 1c0df9b1bd076a275913a86ad8781b9dd4e90b6a Mon Sep 17 00:00:00 2001 From: Mike Swanson Date: Sun, 10 May 2026 19:52:40 -0700 Subject: [PATCH] sync: auto-sync from DESKTOP-0O8A1RL at 2026-05-10 19:52:39 Author: Mike Swanson Machine: DESKTOP-0O8A1RL Timestamp: 2026-05-10 19:52:39 --- .../session-logs/2026-05-10-session.md | 197 ++++++++++++++++++ 1 file changed, 197 insertions(+) create mode 100644 clients/peaceful-spirit/session-logs/2026-05-10-session.md diff --git a/clients/peaceful-spirit/session-logs/2026-05-10-session.md b/clients/peaceful-spirit/session-logs/2026-05-10-session.md new file mode 100644 index 0000000..f7fbdd6 --- /dev/null +++ b/clients/peaceful-spirit/session-logs/2026-05-10-session.md @@ -0,0 +1,197 @@ +# Peaceful Spirit — VPN Pre-Login Setup + RMM Enrollment + +**Date:** 2026-05-10 +**Client:** Peaceful Spirit (Country Club site) +**Ticket scope:** Pre-login IKEv2 VPN for Mara + domain connectivity from remote machines + +## User +- **User:** Mike Swanson (mike) +- **Machine:** DESKTOP-0O8A1RL +- **Role:** admin +- **Session span:** ~3 hours prior (unlogged, crashed) + recovery session + +--- + +## Session Summary + +Reconstructed session context from vault, git log, Windows event log, and RMM after a previous session crash with no log saved. Identified that the previous session had installed the RMM agent on PST-SERVER, reconfigured the Unifi Cloud Gateway (UCG-PST-CC) for pre-login IKEv2, and created multiple IKEv2 and L2TP connections on DESKTOP-0O8A1RL. PST-SERVER was confirmed online in GuruRMM with a valid agent and Windows Server 2016 Essentials. + +Diagnosed IKEv2 error 812 (NPS policy denial) by querying NPS IAS logs via RMM. Logs showed PEACEFULSPIRIT\apst-admin being rejected — this user does not exist in AD (only pst-admin does). The typo in the credential caused the NPS order-1 policy (conditioned on WseRemoteAccessUsers group membership) to fail evaluation, falling through to the default RRAS deny policy (order 999998). The IKEv2 IPSec layer itself was confirmed functional — UCG port-forwards UDP 500/4500 to PST-SERVER, and PST-SERVER's RRAS is the actual IKEv2 endpoint. + +Also diagnosed L2TP error 788 (IPSec negotiation failure). L2TP via PST-CC had connected successfully at 12:18 PM local time, but broke after the previous session's UCG VPN reconfiguration. NAT-T registry fix was already in place (AssumeUDPEncapsulationContextOnSendRule=2). UCG SSH on the WAN IP (98.190.129.150:22) was not accessible, so the exact UCG config state couldn't be inspected. + +Applied two fixes: updated Windows Credential Manager on DESKTOP-0O8A1RL to correct the credential from apst-admin to pst-admin, and added a broad NPS test policy (PST-VPN-Test, order 0) on PST-SERVER via RMM command. Manual IKEv2 connection test via Windows VPN Settings is pending. Pre-login VPN configuration for Mara on three machines was not reached this session. + +--- + +## Key Decisions + +- **Added NPS policy PST-VPN-Test at order 0** — broad time-of-day condition, Allow-Dial-In=TRUE. Ensures auth proceeds even if the existing order-1 group condition fails evaluation. Intentionally permissive for testing; will be tightened or removed once IKEv2 is verified working. +- **Updated Credential Manager rather than recreating VPN connections** — the IKEv2 connections (PST-CC-IKEv2, PST-CC-IKEv2-TEST) were structurally correct; only the stored credential was wrong. Fixing in-place avoided having to rebuild EAP config XML. +- **Did not attempt to recreate UCG VPN config** — UCG SSH inaccessible from WAN, and the IKEv2 IPSec layer is working (tunnel establishes). UCG fix deferred to UniFi cloud portal access or on-site visit. +- **Deferred pre-login VPN setup for Mara** — pre-login VPN (AllUser + UseWinlogonCredential=true) requires IKEv2 end-to-end verification first. Setup can't be meaningfully pushed to the 3 machines until the NPS auth chain is confirmed working. + +--- + +## Problems Encountered + +- **Previous session crashed with no log saved (~3 hours of work lost).** Reconstructed context from: vault (PST-SERVER credentials, UCG details), Windows event log (VPN connection attempts at 6:01 PM and 6:23 PM local), RMM (PST-SERVER online, NPS IAS log, AD user/group queries). +- **IKEv2 error 812 — NPS policy denial.** Root cause: VPN credential stored as `PEACEFULSPIRIT\apst-admin` (nonexistent user). NPS order-1 policy condition (WseRemoteAccessUsers group SID) can't evaluate for a nonexistent user, so it falls through to the default deny policy. Fixed by correcting credential to `pst-admin` and adding order-0 policy. +- **L2TP error 788 — IPSec negotiation failure.** Was working earlier today, broke after UCG IKEv2 reconfiguration. UCG WAN SSH not accessible, so direct inspection wasn't possible. Likely cause: UCG IKEv2 config change altered IPSec proposals, breaking L2TP SA negotiation parameters. Not resolved this session. +- **rasdial cannot test IKEv2/EAP non-interactively (error 703).** IKEv2 only supports EAP or machine certificate auth; `Set-VpnConnectionUsernamePassword` not available in PS5.1; EAP credential dialog requires interactive context. Manual test via Windows VPN Settings required. +- **RMM API at 172.16.3.30 unreachable** — DESKTOP-0O8A1RL is on Wi-Fi (10.2.36.218/16) with no route to 172.16.3.x. Used public URL (rmm.azcomputerguru.com via Cloudflare) for all RMM API calls. + +--- + +## Configuration Changes + +### NPS on PST-SERVER (via RMM) +- Added policy: `PST-VPN-Test` — order 0, enabled, time-of-day=all, Allow-Dial-In=TRUE +- Existing policies untouched: + - `{502F03DC-...}` order 1: WseRemoteAccessUsers group, PEAP+TLS, Allow=TRUE (was not matching due to apst-admin) + - `Connections to Microsoft Routing and Remote Access server` order 999998: Allow=FALSE (default RRAS) + - `Connections to other access servers` order 999999: Allow=FALSE (default) + +### Windows Credential Manager on DESKTOP-0O8A1RL +- Deleted: `PST-CC-IKEv2-TEST`, `PST-CC-IKEv2`, `98.190.129.150` (stale apst-admin entries) +- Added: `PST-CC-IKEv2` → `PEACEFULSPIRIT\pst-admin` +- Added: `98.190.129.150` → `PEACEFULSPIRIT\pst-admin` + +### VPN Connections on DESKTOP-0O8A1RL (created in prior session, confirmed present) +| Name | Type | Auth | AllUser | Status | +|------|------|------|---------|--------| +| PST-CC | L2TP/IPSec | MS-CHAPv2 + PSK | No | Disconnected (error 788) | +| PST-CC-IKEv2-TEST | IKEv2 | PEAP-MSCHAPv2 | No | Disconnected (error 812, now fixed) | +| PST-CC-IKEv2 | IKEv2 | PEAP-MSCHAPv2 | No | Disconnected (error 812, now fixed) | + +--- + +## Credentials & Secrets + +| Item | Value | +|------|-------| +| PST-SERVER SSH | sysadmin / r3tr0gradE99! | +| UCG SSH key | ~/.ssh/pst-cc-ucg / password: Gptf*77ttb123!@# | +| VPN credential (L2TP + IKEv2) | PEACEFULSPIRIT\pst-admin / 24Hearts$ | +| VPN PSK | z5zkNBds2V9eIkdey09Zm6Khil3DAZs8 | +| NPS RADIUS shared secret (UCG client) | PST-RADIUS-UCG-2026!@# | +| UCG VPN user (alternate) | sysadmin / Paper123!@# | +| pst-admin (domain admin) | 24Hearts$ | +| Mara (domain user, VPN eligible) | (not captured — needs reset if pre-login VPN uses UseWinlogonCredential) | + +Vault paths: +- `clients/peaceful-spirit/server.sops.yaml` — PST-SERVER, UCG details +- `clients/peaceful-spirit/vpn.sops.yaml` — VPN credentials, PSK, network + +--- + +## Infrastructure & Servers + +| Component | Value | +|-----------|-------| +| PST-SERVER IP (LAN) | 192.168.0.2 | +| PST-SERVER OS | Windows Server 2016 Essentials (build 14393) | +| PST-SERVER domain | PEACEFULSPIRIT.local | +| PST-SERVER roles | AD DS, DNS, RRAS (VPN server), NPS | +| UCG-PST-CC LAN IP | 192.168.0.10 | +| UCG-PST-CC WAN IP | 98.190.129.150 | +| UCG VPN endpoint | UDP 500/4500 → forwarded to 192.168.0.2 (PST-SERVER RRAS) | +| PST network | 192.168.0.0/24 | +| DNS server | 192.168.0.2 | +| GuruRMM client | Peaceful Spirit (00015eae-50e5-4102-93fa-ab0fdb135c08) | +| GuruRMM site | Country Club (7b32983d-982a-4a5c-af07-45a23453f589) | +| PST-SERVER agent ID | 6b6106a7-8515-4b6b-857d-0dc6ede53f35 | +| PST-SERVER agent enrolled | 2026-05-10 23:19 UTC | +| PST-SERVER last seen | 2026-05-11 01:29 UTC (active) | + +### AD Users in WseRemoteAccessUsers (VPN eligible) +- Domain Admins (group) +- PSTAdmin +- pst-admin +- LMT +- Mara + +--- + +## Commands & Outputs + +### RMM JWT generation (bash) +```bash +py /tmp/jwt.py # generates HS256 token for admin@azcomputerguru.com +# Secret: ZNzGxghru2XUdBVlaf2G2L1YUBVcl5xH0lr/Gpf/QmE= (UTF-8 bytes, not base64-decoded) +``` + +### Send command to PST-SERVER via RMM +```bash +AGENT_ID="6b6106a7-8515-4b6b-857d-0dc6ede53f35" +py -c "import json; print(json.dumps({'command': '', 'command_type': 'powershell'}))" > /tmp/cmd.json +curl -s -X POST "https://rmm.azcomputerguru.com/api/agents/$AGENT_ID/command" \ + -H "Authorization: Bearer $TOKEN" \ + -H "Content-Type: application/json" \ + -d @/tmp/cmd.json +``` + +### NPS config check (PST-SERVER) +``` +netsh nps show client +netsh nps show np +``` +Result: UCG-PST-CC at 192.168.0.10, secret PST-RADIUS-UCG-2026!@#. 3 policies; order-1 is WseRemoteAccessUsers. + +### NPS IAS log tail (PST-SERVER) +```powershell +Get-ChildItem "C:\Windows\System32\LogFiles\IN*.log" | Sort LastWriteTime -Desc | Select -First 1 | ForEach-Object { Get-Content $_.FullName -Tail 10 } +``` +Key finding: all auth attempts arriving as `PEACEFULSPIRIT\apst-admin`, rejected by "Microsoft Routing and Remote Access Service Policy" with reason code 8. + +### Add NPS policy (PST-SERVER) +``` +netsh nps add np name="PST-VPN-Test" state=enable processingorder=0 policysource=0 conditionid=0x1006 conditiondata="0 00:00-24:00; 1 00:00-24:00; 2 00:00-24:00; 3 00:00-24:00; 4 00:00-24:00; 5 00:00-24:00; 6 00:00-24:00" profileid=0x100f profiledata=TRUE +``` +Result: `Ok.` — policy at order 0 confirmed present. + +### Credential Manager fix (DESKTOP-0O8A1RL) +``` +cmdkey /delete:"PST-CC-IKEv2" +cmdkey /delete:"PST-CC-IKEv2-TEST" +cmdkey /delete:"98.190.129.150" +cmdkey /add:"98.190.129.150" /user:"PEACEFULSPIRIT\pst-admin" /pass:"24Hearts$" +cmdkey /add:"PST-CC-IKEv2" /user:"PEACEFULSPIRIT\pst-admin" /pass:"24Hearts$" +``` + +### VPN test (error at time of session) +``` +rasdial "PST-CC" "sysadmin" "Paper123!@#" +→ Error 788: L2TP security layer could not negotiate compatible parameters + +rasdial "PST-CC-IKEv2" +→ Error 703: needs information (EAP cannot run non-interactively) +``` + +--- + +## Pending / Incomplete Tasks + +| Task | Status | Notes | +|------|--------|-------| +| IKEv2 VPN connection test from DESKTOP-0O8A1RL | **PENDING** | Connect PST-CC-IKEv2 via Windows VPN Settings. Credential is now pst-admin. NPS order-0 policy should allow it. | +| Fix L2TP error 788 | **PENDING** | UCG config likely broke L2TP IPSec proposals. Need UCG access (unifi.ui.com cloud portal or on-site). Check if L2TP VPN type is still enabled on UCG. | +| Pre-login IKEv2 VPN for Mara on 3 machines | **NOT STARTED** | Requires IKEv2 working first. Then: Add-VpnConnection -AllUserConnection -AuthenticationMethod Eap, EAP XML with UseWinlogonCredentials=true, deploy to 3 machines. | +| Identify Mara's 3 machines | **NOT STARTED** | Need to confirm which 3 computers need pre-login VPN. | +| Tighten/remove PST-VPN-Test NPS policy | **PENDING** | Remove order-0 test policy once IKEv2 end-to-end is verified. The order-1 WseRemoteAccessUsers policy should be the access gate. | +| RMM agent on Mara's 3 machines | **UNKNOWN** | Unknown if already enrolled. Check RMM for Peaceful Spirit / Country Club site. | +| Create Peaceful Spirit client directory in ClaudeTools | **DONE** | `clients/peaceful-spirit/` created this session. | + +--- + +## Reference Information + +- GuruRMM API: `https://rmm.azcomputerguru.com/api/` +- PST-SERVER agent: `https://rmm.azcomputerguru.com/api/agents/6b6106a7-8515-4b6b-857d-0dc6ede53f35` +- Peaceful Spirit client in RMM: ID `00015eae-50e5-4102-93fa-ab0fdb135c08` +- Country Club site in RMM: ID `7b32983d-982a-4a5c-af07-45a23453f589` +- Vault: `clients/peaceful-spirit/server.sops.yaml`, `clients/peaceful-spirit/vpn.sops.yaml` +- NPS reason code 8 in IAS logs = "Authentication type not permitted" (policy did not match) +- Windows event IDs for VPN: 20221 (dial start), 20222 (device connected), 20223 (link established), 20224 (link established), 20227 (failure) +- IKEv2 EAP XML for UseWinlogonCredentials: set `true` in the MSCHAPv2 inner EAP block +- AllUser VPN (pre-login): `Add-VpnConnection -AllUserConnection $true` — requires admin rights, connection is available at Windows login screen