sync: auto-sync from GURU-5070 at 2026-06-23 16:38:25

Author: Mike Swanson
Machine: GURU-5070
Timestamp: 2026-06-23 16:38:25

clients/four-paws/docs/avimark-recovery-playbook.md

@@ -0,0 +1,100 @@
+# Four Paws — Avimark Recovery Playbook (post OS-reinstall)
+
+**Scenario:** Four Paws' Avimark server had its **OS reinstalled** (same box — NOT a hardware migration). Avimark
+must be re-stood-up on the freshly-installed OS. Source: Covetrus "Moving Avimark to a New Server" (2024).
+Digested 2026-06-23 (Mike) for Howard, who is doing the hands-on recovery.
+
+## Client context
+- **Customer:** Four Paws (Syncro customer_id `33050383`). Josh Fender — fourpawsarizona@gmail.com — 520-321-0277.
+  4750 East Grant Road, Tucson AZ 85712.
+- **Avimark mode: CLIENT/SERVER** (confirmed from the 2026-05-18 incident — `AvimarkServer.exe` had vanished from
+  Services; fixed via AvimarkGuardian "Install AVImark as a Hidden Service"). So this recovery follows the doc's
+  **"When Using Client Server"** path, not the simple file-copy path.
+- **Avimark/Covetrus Tech Support: (877) 838-9273 Option 1** (sub-options below). 877-999-2838 is sales, not support.
+
+## The OS-reinstall reality (how this differs from the Covetrus doc)
+The doc assumes old + new server both live on the network so you can copy the Avimark folder across. Here there is
+**no old server** — same machine, wiped OS. So the #1 question is **where is the Avimark data now:**
+1. **Surviving partition** — if Avimark lived on a non-OS drive (e.g. `D:\Avimark`) and only `C:` was wiped, the
+   data folder is still there. Best case: re-point to it.
+2. **Backup restore** — if Avimark was on `C:`, restore the **entire `\Avimark` folder** from backup (image / file
+   backup / Data Vault). Get the most recent good copy.
+- **Do NOT start a fresh/empty Avimark install over the top** — you'd lose the practice's data. Recover the existing
+  Avimark folder first.
+- If the **hostname changed** during reinstall, client-workstation UNC shortcuts (`\\OLDNAME\Avimark\...`) break and
+  must be recreated to the new name. Keeping the **same hostname** avoids reworking every workstation.
+
+## CRITICAL safety callouts (data-loss risks — read first)
+- **Rebuild the Attachment index on the FIRST Avimark launch, immediately.** If you get ANY rebuild error — STOP, do
+  not proceed, call support. Skipping/ignoring = inability to access attachments.
+- **Attachments stored OUTSIDE the Avimark folder** are a known footgun. If any were ever stored outside `\Avimark`
+  (or you're unsure) — call support before/while recovering, or those attachments won't resolve.
+- Verify the **Documents / Forms / Photos / File Attachments** paths point at real, restored locations
+  (Avimark: Work with > Hospital Setup > Advanced tab > Advanced Options). Ideally all inside `\Avimark`.
+
+## Recovery steps (Client/Server)
+1. **Locate/restore the Avimark folder** on the (reinstalled) server per "OS-reinstall reality" above. Confirm the
+   sub-dirs (Attachments, Documents, Forms, Photos) are present and ideally inside `\Avimark`.
+2. **Share the Avimark folder** — full access (users can change files). Share *just* the Avimark folder (recommended)
+   or the whole drive if the sub-dirs live elsewhere on it. **Disable the Windows Sharing Wizard** first
+   (see Avimark "Recommended Settings" doc).
+3. **Users must run as power user / administrator** on the server.
+4. **Avimark.ini** holds the server location for client/server. If carried over from the old OS it may point wrong —
+   if the server can't start clean, delete `Avimark.ini` from the `\Avimark` folder and let Guardian recreate it.
+5. **AvimarkGuardian** (create a UNC shortcut to `AvimarkGuardian.exe`, Run as Administrator):
+   - Open it → "Open AvimarkServer" checked → **Run**.
+   - In the blue **Avimark Server** window: **File > Options** → set **port 9191**, **password 123** → OK.
+   - Back in Guardian: **Install Avimark Server as a hidden Service** → Run. Wait until status shows the server is
+     running as a service, then close.
+   - (This is the exact thing that fixed the May incident — the hidden service must exist + be running.)
+6. **First client connect to seed the link:** make a UNC shortcut to `Avimark.exe` (Run as Administrator). Copy its
+   Target path, append ` /connect` (e.g. `C:\Avimark\Avimark.exe /connect`), run it →
+   choose **"Connect to Application server on the network or Internet"** → enter the **server's computer name** +
+   the **same port 9191 / password 123** → OK.
+7. **Rebuild the Attachment index immediately** (see CRITICAL callout). Errors → STOP + call support.
+8. **Remove the ` /connect`** from the Target path once connected.
+9. **Workstations:** delete old Avimark shortcuts; create **new UNC shortcuts** (via Network > the server's
+   `\Avimark` share → right-click `Avimark.exe` > Send to > Desktop), Run as Administrator. **Do NOT map a drive.**
+10. **Oplocks/Caching:** on the server, Avimark **Help > About Avimark** → if a **System Settings** tab exists,
+    click **Correct** on the Oplocks and/or Caching messages → **reboot the server** for it to take effect.
+11. **Print test** from a workstation inside Avimark — errors/failure → call support.
+12. **Recommended Settings doc** (Avimark.net > Support > Support Documents): apply caching, power management, and
+    firewall settings for the OS version.
+
+## Ancillary services — each needs a Covetrus call AFTER recovery (don't skip; some are silent failures)
+Confirm with Four Paws which they use, then call the matching number:
+- **Data Vault (cloud backup):** call **(888) 538-8348** after the move or backups silently stop.
+- **Avimark Payment Solutions (APS)** credit-card processing: (877) 838-9273 opt 1 sub 3.
+- **Dragonfly** (if on the Avimark server): (877) 838-9273 opt 1 sub 8.
+- **VetXML / Sync Service / Intelligent Inventory / VetRx / IVLS / Time Clock / Rapport-PetWise-eMinders:** all need
+  support steps (opt 1, or opt 2 for Rapport/PetWise/eMinders, opt 4 sub 2 for IVLS lab). See the source doc p.3-4.
+
+## RESOLVED + VERIFIED — 2026-06-23 (via GuruRMM, agent ccb55043)
+Avimark is back up. Verified end-state on the server:
+- **Hostname renamed to `SERVER`** (was DESKTOP-18P49QQ) — now matches the client INI `IP=server`, and workstation `\\SERVER\AVImark` UNC shortcuts resolve.
+- **`AVImarkServer` service = Running** (Automatic start), **listening on port 1095** (pid AVImarkServer).
+- **`C:\AVImark` shared** as `AVImark`.
+- Client INI: `IP=server`, `Port=1095`, `StandAlone=0`; client password blob regenerated = freshly re-connected.
+- Outstanding (confirm with Josh if used): Data Vault backup re-register (888-538-8348) + other integrations.
+
+## LIVE ASSESSMENT — 2026-06-23 (via GuruRMM, agent ccb55043 / DESKTOP-18P49QQ → renamed SERVER)
+- **OS reinstalled today** (Win 11 Pro build 26100, installed 6/23/2026 15:10). localadmin logged in at console.
+- **Data INTACT:** `C:\AVImark` = **45.4 GB**, last activity 6/15 (restored from backup). Binaries present:
+  `AVImark.exe`, `AVImarkServer.exe`, `AVImarkGuardian.exe` (in `C:\AVImark`).
+- **`AVImark.ini` [Client]:** `IP=server`, **`Port=1095`**, `StandAlone=0` (client/server), encrypted password blob.
+- **THREE breakages (root causes):**
+  1. **No Avimark Server service** (`Get-Service *avimark*` = none) — same failure mode as the May 2026 incident.
+  2. **`C:\AVImark` not shared** (no non-admin SMB shares) — workstations can't reach it.
+  3. **Hostname = `DESKTOP-18P49QQ`** but clients expect host **`server`** — old box was almost certainly named `SERVER`.
+- **Fix order:** (A) rename box to old name (likely SERVER) + reboot → fixes `IP=server` + workstation UNCs; (B) share
+  `C:\AVImark` full-access (disable Sharing Wizard); (C) AVImarkGuardian → Open AvimarkServer → File>Options **Port 1095**
+  + password → Install as hidden Service → run; (D) launch AVImark, rebuild attachment index (stop on error); (E)
+  Help>About>System Settings → Correct Oplocks/Caching → reboot; (F) workstations; (G) Data Vault re-register + integrations.
+- **Use existing Port 1095** (NOT the doc's 9191) so existing client INIs keep matching.
+- Coord to Howard: msgs 10f4fd4f (playbook) + 6ecf8bcd (live findings). Howard is hands-on at the console.
+
+## References
+- Source PDF: `Instructions-for-Moving-Avimark-to-a-New-Server-2024-1.pdf` (Covetrus 2024).
+- Client/Server setup doc: https://software.covetrus.com/wp-content/uploads/dlm_uploads/2021/03/Setting-up-an-AVImark-client-server.pdf
+- Recommended Settings: avimark.net > Support > Support Documents.
+- Prior incident: `session-logs/2026-05-18-session.md` (Syncro #110507180 — AvimarkServer service reinstall).

clients/four-paws/session-logs/2026-06/2026-06-23-mike-avimark-server-os-reinstall-recovery.md

@@ -0,0 +1,66 @@
+## User
+- **User:** Mike Swanson (mike)
+- **Machine:** GURU-5070
+- **Role:** admin
+
+## Session Summary
+
+Four Paws' Avimark server had its OS reinstalled (same hardware, fresh Windows 11 Pro) and Avimark needed to be brought back online. Mike provided the Covetrus "Moving Avimark to a New Server" PDF to digest and asked me to coordinate the info to Howard (hands-on) and help recover. Digested the PDF into a recovery playbook tailored to the OS-reinstall scenario (no old server to copy from — data must come from a surviving partition or backup), and noted Four Paws runs Avimark in CLIENT/SERVER mode (confirmed from the 2026-05-18 incident where AvimarkServer.exe had vanished from Services).
+
+Discovered via prior context that the Avimark server also has a GuruRMM agent, so I assessed the live state directly: agent DESKTOP-18P49QQ (Four Paws / Main, id ccb55043). Read-only assessment found the data intact (C:\AVImark, 45.4 GB, restored from a 6/15 backup) with all binaries present, but three concrete breakages: (1) no AvimarkServer service installed, (2) C:\AVImark not shared, (3) hostname DESKTOP-18P49QQ while the client INI expects host "server" on port 1095. Sent Howard two coord messages — the full playbook and the concrete live findings with an exact fix order — and posted RMM alerts.
+
+On Mike's instruction, rebooted the server via RMM (shutdown /r /t 15), then set the local user "staff" password to never-expire (Set-LocalUser -PasswordNeverExpires $true; verified PasswordExpires now blank).
+
+Howard completed the recovery (rename + share + service + reconnect). Final verification via RMM confirmed all three breakages resolved: hostname now SERVER, AVImarkServer service Running (Automatic) and listening on 1095, C:\AVImark shared, and the client INI re-connected (password blob regenerated). Stamped the playbook doc RESOLVED + VERIFIED. Avimark is back up.
+
+## Key Decisions
+
+- **Reused the existing port 1095, not the doc's example 9191** — the client INIs already point at 1095; switching ports would have broken every workstation.
+- **Renamed the box to SERVER (the old name)** as the first fix — it simultaneously satisfies the client INI `IP=server` and keeps the workstations' `\\SERVER\AVImark` UNC shortcuts working, avoiding per-workstation rework.
+- **Did the assessment + safe ops via RMM but left rename + Guardian GUI service-install to Howard at the console** — those need an interactive desktop session and the (old) server password.
+- **Tailored the generic Covetrus migration doc to the OS-reinstall reality** — the doc assumes old+new servers both online; here the operative concern was restoring/locating the C:\AVImark folder and re-establishing the client/server service, share, and name.
+
+## Problems Encountered
+
+- **Avimark down — root causes (3):** no AvimarkServer service, C:\AVImark not shared, hostname != "server". All three identified via a single RMM assessment and resolved (service installed + running on 1095, folder shared, box renamed to SERVER).
+- **Assessment command returned exit 1** but full stdout — the non-zero came from `query user`, not a real failure.
+- **Agent re-resolution:** after the rename, the live command reports HOST=SERVER while the agent record/id (ccb55043) stayed stable — resolved by id, with a Four Paws client-name fallback.
+
+## Configuration Changes
+
+- **Created** `clients/four-paws/docs/avimark-recovery-playbook.md` (digested playbook + live assessment + RESOLVED/VERIFIED stamp).
+- **Four Paws server (SERVER / DESKTOP-18P49QQ, agent ccb55043):** rebooted (RMM); local user `staff` set PasswordNeverExpires=$true. (Rename to SERVER, C:\AVImark share, and AvimarkServer hidden-service install were done by Howard at the console.)
+- **Coord:** msgs to Howard-Home 10f4fd4f (playbook), 6ecf8bcd (live findings). RMM #dev-alerts posted for assessment, reboot, staff change, verify.
+- **Session log:** this file.
+
+## Credentials & Secrets
+
+- No new credentials created or vaulted this session. Avimark client/server password not captured (the client INI stores an encrypted blob; the server-side plaintext was set by Howard during the Guardian service install). Avimark/Covetrus Tech Support: (877) 838-9273 Option 1.
+
+## Infrastructure & Servers
+
+- **Four Paws Avimark server:** hostname **SERVER** (renamed from DESKTOP-18P49QQ on 2026-06-23), Windows 11 Pro build 26100 (OS reinstalled 2026-06-23). GuruRMM agent id `ccb55043-b310-47df-afe3-2671c8ff113c` (Four Paws / Main).
+- **Avimark:** Client/Server mode. Data at `C:\AVImark` (~45 GB). `AVImarkServer` service Running (Automatic), listening TCP **1095**. Share `AVImark` -> `C:\AVImark` (full access). Binaries: AVImark.exe/AVImarkServer.exe/AVImarkGuardian.exe (2013-2015 build) in C:\AVImark. Client INI `C:\AVImark\AVImark.ini` [Client]: IP=server, Port=1095, StandAlone=0.
+- **Drives:** C: 373 GB free; E: ~9 GB (small); D:/F: empty.
+- **Four Paws contact:** Josh Fender — fourpawsarizona@gmail.com — 520-321-0277. 4750 East Grant Road, Tucson AZ 85712. Syncro customer_id 33050383.
+
+## Commands & Outputs
+
+- RMM assessment (read-only): found C:\AVImark 45.4GB, no *avimark* service, no non-admin shares, INI IP=server/Port=1095.
+- Reboot: `shutdown.exe /r /t 15 /c "ACG remote reboot - Avimark recovery"` (cmd ccef3cf2, exit 0).
+- Staff never-expire: `Set-LocalUser -Name "staff" -PasswordNeverExpires $true` -> verified `PasswordExpires=` blank (cmd 7956a35d).
+- Verify: HOST=SERVER; `AVImarkServer [Running] Automatic`; `LISTEN 1095 AVImarkServer`; `SHARE: AVImark -> C:\AVImark` (cmd 0f6c2a22).
+
+## Pending / Incomplete Tasks
+
+- **Integrations to re-register** if Four Paws uses them (confirm with Josh): Data Vault cloud backup (888-538-8348) — silent failure if skipped; APS credit cards (877-838-9273 opt 1 sub 3); plus VetXML/Sync/Intelligent Inventory/Rapport-PetWise-eMinders/IVLS per the playbook.
+- **Recommended Settings doc** (caching/power/firewall) and Oplocks/Caching correction — verify applied (Help > About > System Settings) if not already done during recovery.
+- Workstations: confirm all can open Avimark + print (covered by the rename keeping \\SERVER\AVImark shortcuts valid).
+
+## Reference Information
+
+- Playbook: `clients/four-paws/docs/avimark-recovery-playbook.md`.
+- Source PDF: Covetrus "Moving Avimark to a New Server" (2024).
+- Client/Server setup doc: https://software.covetrus.com/wp-content/uploads/dlm_uploads/2021/03/Setting-up-an-AVImark-client-server.pdf
+- Prior incident: `session-logs/2026-05-18-session.md` (Syncro #110507180 — AvimarkServer service reinstall).
+- RMM agent id: ccb55043-b310-47df-afe3-2671c8ff113c. Coord msgs: 10f4fd4f, 6ecf8bcd.

errorlog.md

@@ -27,6 +27,8 @@ Categories (the `[type]` tag): _(none)_ = skill/command execution failure ·
 
 2026-06-23 | GURU-BEAST-ROG | syncro/billing | Syncro API has no delete or update endpoint for line items — add_line_item is the only confirmed write method. Cannot correct a line item price via API; requires manual UI deletion. [ctx: ticket=32447 line_item_id=42988553]
 
+2026-06-23 | GURU-5070 | rmm/vwp-qbs-firewall | [correction] treated VWP-QBS firewall-disabled as an oversight to re-enable promptly; correct: it's intentionally OFF until VWP testing completes - leave it, do not re-flag [ctx: client=valleywide host=VWP-QBS]
+
 2026-06-23 | Howard-Home | unifi-wifi/pfsense-ssh | SSH connect/auth failed (rc=255) [ctx: host=192.168.0.1:22 slug=cascades-tucson act=run]
 
 2026-06-23 | Howard-Home | bash/json-test-data | [friction] Git-Bash heredoc (even quoted <<'EOF') wrote C: as single backslash -> invalid JSON -> PS engine threw 'Unrecognized escape sequence' exit 3; fix: build JSON test files via PowerShell ConvertTo-Json, not bash heredocs [ctx: ref=feedback_tmp_path_windows]