## User
- **User:** Howard Enos (howard)
- **Machine:** Howard-Home
- **Role:** tech

# CS-SERVER: full Datto stack removal + SMB error-67 root-cause reframe

## Session Summary

Continuation of the Cascades NAS->CS-SERVER migration / CS-SERVER SMB "error 67"
investigation. Two outcomes this session: (1) **completely removed the Datto software
stack from CS-SERVER**, and (2) **disproved the working theory** that the AV was causing
the SMB failure and narrowed the real cause.

The AV on CS-SERVER was misidentified in the prior session as GravityZone Bitdefender.
Investigation showed it is actually **DattoAV** — the "Endpoint Protection SDK"
(v1.0.2510.6851) under `C:\Program Files\infocyte\agent\dattoav`, managed by the **Datto
RMM agent (CagService/CentraStage)** and **Datto EDR Agent (HUNTAgent / Infocyte HUNT,
v3.17.1.5552)**. DattoAV uses the Bitdefender engine + Avira "Sentry" driver, which is why
the kernel minifilters were named `BdSentry` ("Avira Sentry Driver"), `rtp1`, `rtp2`. This
is why removing the box from the GravityZone console did nothing — GravityZone never
managed it.

Removed all three Datto components in order: `endpointprotection.exe uninstallSdk` (cleared
`rtp1`/`rtp2`/`BdSentry` registry keys + unloaded the minifilters with no reboot), Datto RMM
via `CentraStage\uninst.exe /VERYSILENT` (clean, exit 0), then the Datto EDR agent. The EDR
agent's GUI uninstaller crashed headless (wgpu surface panic) and `agent.exe --uninstall
--no-gui` was blocked by uninstall-protection ("token required"). Stopped + disabled
HUNTAgent (held across reboot — no tamper watchdog), rebooted to clear the in-memory state,
then — since the tamper drivers were already gone — **force-removed** the orphaned EDR
agent: killed processes, `sc delete HUNTAgent`, removed `C:\Program Files\infocyte`, deleted
the uninstall registry entries. CS-SERVER is now 100% Datto-free (no services, dirs,
registry entries, or drivers).

Critically, **SMB error 67 persisted unchanged after the full AV removal + reboot**, so the
AV was NOT the cause. Deep diagnosis then revealed the SMB server is actually **healthy for
domain-joined (Kerberos) clients** — `Get-SmbSession` shows 7+ live SMB 3.1.1 sessions with
open files (lauren.hasselman, Sharon.Edwards, Crystal.Rodriguez, Megan.Hiatt, Ashley.Jensen,
chris.knight) established after the reboot. The failure is confined to **workgroup/NTLM and
loopback** connections: Karen's workgroup PC (DESKTOP-LPOPV30) gets error 67 to every share
(IPC$, C$, a brand-new test share) even in her real user session with correct credentials,
and `Get-SmbConnection` shows the SMB session never even negotiates (no SMBServer or
SMBClient event is logged on either side — the failure is pre-SMB-negotiate, BAD_NET_NAME).
NTLM restriction is NOT set on the server (checked `RestrictReceivingNTLMTraffic`,
`LmCompatibilityLevel`, NTLM Operational log — all clear). The Datto EDR uninstall token
Howard found is moot for CS-SERVER (already removed) but useful for other Cascades agents.

## Key Decisions
- Removed the entire Datto stack (AV + RMM + EDR) per Howard's "uninstall all the datto
  software, we can install again if needed" — GuruRMM is a separate product, so removing
  Datto RMM did not sever our control channel.
- Force-removed the orphaned EDR agent rather than chase the uninstall-protection token:
  the kernel tamper drivers (BdSentry/rtp*) were already removed by `uninstallSdk`, so
  nothing was actively protecting the files; `sc delete` + dir/reg removal was clean and
  reboot-safe.
- Stopped treating the AV as the SMB root cause once error 67 reproduced identically after
  full removal + reboot, AND once 7 live domain SMB 3.1.1 sessions proved the server engine
  is healthy.
- Did NOT make further SMB-server config changes (e.g., disabling multichannel) unilaterally
  on the production DC — paused to get Howard's direction.

## Problems Encountered
- **AV misidentified as GravityZone Bitdefender** -> actually DattoAV (Endpoint Protection
  SDK) via the Datto/Infocyte agent. Logged as a `--correction` in errorlog.
- **Datto EDR GUI uninstaller crashed headless** (wgpu "Invalid surface" panic) under SYSTEM
  -> used `--no-gui`, which then hit uninstall-protection ("token required").
- **EDR uninstall-protection token unavailable** -> CS-SERVER is not enrolled in the Datto
  EDR tenant (azcomp4587) anymore (tenant-wide hostname search returns empty), so no token
  can be issued for it. Force-removal used instead.
- **rtp1/rtp2 reverted Start=4->1 and reloaded after the first reboot** -> the EDR agent
  re-armed them on boot; resolved by removing the managing agents, not just the drivers.
- **`shutdown /a`/RPC-dependent ops returned 1717/1702** during the SMB-broken window ->
  switched to local-only checks.
- **Disproved my own NTLM-restriction theory and the prior CSC-ENT-vs-CSCNet theory** with
  evidence (NTLM policy unset; domain clients on the same 10.0.20.x subnet as Karen work).

## Configuration Changes
- CS-SERVER (192.168.2.248): **removed** DattoAV / Endpoint Protection SDK, Datto RMM
  (CentraStage, `CagService`), Datto EDR Agent (`HUNTAgent`). Deleted services, removed
  `C:\Program Files\infocyte` and `C:\Program Files (x86)\CentraStage`, removed all related
  Uninstall registry keys, kernel drivers `rtp1`/`rtp2`/`BdSentry` gone. Rebooted ~06:43 MST.
- Prior-session NullSessionShares REG type repair (REG_SZ->REG_MULTI_SZ) confirmed still
  good (`Get-SmbServerConfiguration` works).
- Created + removed a throwaway test share `ztest$` (C:\ztest) during diagnosis.
- No NTFS/share ACL changes. No multichannel change (deferred for Howard's call).

## Credentials & Secrets
- Datto EDR (azcomp4587) org-level uninstall-protection token: viewable in console at
  `https://azcomp4587.infocyte.com/organizations/2d5ea96e-3228-461b-9c60-13ae464b61d8`
  (unmask button). NOT YET VAULTED (value not captured) — vault into
  `msp-tools/datto-edr.sops.yaml` if Howard pastes it; useful for removing other Cascades
  EDR agents.
- CS-SERVER orphaned EDR agent config (now removed): api-url
  `https://azcomp4587.infocyte.com:443`, server-key `b7cmnghlgh`.
- Karen domain pwd still vaulted: `clients/cascades-tucson/karen-rossini.sops.yaml`.

## Infrastructure & Servers
- CS-SERVER: DC `cascades.local`/`CASCADES`, IPs 192.168.2.248 (Ethernet) + 192.168.2.254
  (Hyper-V vSwitch). GuruRMM agent `c39f1de7-d5b6-45ae-b132-e06977ab1713`. SMB 3.1.1, signing
  required, RejectUnencryptedAccess=True, EnableMultiChannel=True (advertises .248, .254, IPv6
  ULAs fde4::/fd8f::). TCP 445 listens on `::` (dual-stack) owned by PID 4 (kernel). Booted
  06:43 MST 2026-06-26.
- Live SMB clients on CS-SERVER (post-reboot, proof server is healthy): lauren.hasselman
  (10.0.20.235), Sharon.Edwards (192.168.3.133), Crystal.Rodriguez (10.0.20.205, 7 opens),
  Megan.Hiatt (10.0.20.202), Ashley.Jensen (192.168.3.37), chris.knight + DESKTOP-N5G1ROO$
  (10.0.20.183).
- Karen's PC: DESKTOP-LPOPV30 (WORKGROUP), agent `ad725bb2-d8cb-4a83-8203-6f7e9c906b29`,
  logged on locally as `desktop-lpopv30\karen rossini`. Multi-homed mess: only 10.0.20.100
  (Wi-Fi 2) has a real IP; 4 other adapters on APIPA 169.254.x (Ethernet, Wi-Fi, 2x LAC*).
  cmdkey has `Domain:CS-SERVER -> CASCADES\karen.rossini` and `Domain:CASCADESDS`.
- Meredith: ASSISTMAN-PC `cf86fa5e-96a2-494d-9cb1-8be22a518ad0` (on 192.168.2.x / CSC ENT,
  on-link with CS-SERVER; reportedly can access shares).
- Datto EDR tenant azcomp4587: Cascades org `2d5ea96e-3228-461b-9c60-13ae464b61d8`, site
  `1dbd2b02-f7df-45d0-a7f2-18667f48447f` (33 agents, NO server enrolled).

## Commands & Outputs
- `endpointprotection.exe uninstallSdk` -> rtp1/rtp2/BdSentry reg keys GONE, filters unloaded
  (no reboot). exit 2 (worked despite nonzero).
- `CentraStage\uninst.exe /VERYSILENT /SUPPRESSMSGBOXES /NORESTART` -> exit 0, clean.
- `agent.exe --uninstall --no-gui` -> "Uninstall protection is enabled: token required".
- Force removal: kill agent/system-tray, `sc.exe delete HUNTAgent` (SUCCESS), `Remove-Item
  C:\Program Files\infocyte -Recurse -Force` (clean), reg Uninstall keys removed.
- Post-removal verify: NO datto/edr uninstall entries; HUNTAgent/CagService deleted; infocyte
  + CentraStage dirs False; rtp1/rtp2/BdSentry keys False; no filters.
- SMB after full removal+reboot: `net use \\localhost\IPC$` -> error 67; `New-SmbMapping
  \\localhost\{IPC$,C$,ztest$}` -> "network name cannot be found"; NO SMBServer/SMBClient
  events for the attempts. `Get-SmbSession` -> 7 live domain SMB3.1.1 sessions.
- Karen user-session: `whoami`=desktop-lpopv30\karen rossini; `net use \\CS-SERVER\Server
  /user:CASCADES\karen.rossini <correct pw>` -> error 67; `Get-SmbConnection` -> none.
- NTLM policy: RestrictReceivingNTLMTraffic unset, LmCompatibilityLevel unset, NTLM
  Operational log empty. NoLmHash=1, RestrictNullSessAccess=1, restrictanonymous=1 (normal).

## Pending / Incomplete Tasks
- **SMB error 67 root cause (UNRESOLVED):** server healthy for domain/Kerberos clients;
  fails for Karen's workgroup PC + loopback with BAD_NET_NAME *before* SMB negotiate (no
  events). Next discriminators: (a) fresh connect from Meredith (workgroup, on-link
  192.168.2.x) — works? -> isolates Karen-machine vs workgroup-general; (b) Karen's APIPA
  adapter cleanup; (c) test whether SMB multichannel advertising unreachable interfaces
  (.254 vSwitch + IPv6 ULAs) breaks routed/workgroup new-connection setup -> candidate fix:
  unbind File&Printer from .254 / disable multichannel (prod DC — needs care, domain clients
  currently fine); (d) consider domain-joining Karen (then Kerberos like working clients).
- Vault the Datto EDR org uninstall token if Howard captures it.
- Resume Karen ALDocs repoint (`\\CS-SERVER\Server\ALDocs`) via drive-map once access works.
- Correct memory `project_cascades_network_segments.md` (AV was DattoAV; SMB cause is NOT the
  AV; server healthy for domain clients).

## Reference Information
- DattoAV = Datto EDR's Endpoint Protection SDK (Bitdefender engine + Avira Sentry). NOT
  GravityZone. Managed by Datto RMM (CentraStage/CagService) + Datto EDR (HUNTAgent/Infocyte).
- Datto EDR tenant: azcomp4587.infocyte.com; skill `/datto-edr`; vault
  `msp-tools/datto-edr.sops.yaml`. Cascades org `2d5ea96e-3228-461b-9c60-13ae464b61d8`.
- GuruRMM agent CS-SERVER `c39f1de7-d5b6-45ae-b132-e06977ab1713`.
- drive-map skill: `.claude/skills/drive-map/`.
