# 2026-05-01 — Howard — Syncro billing batch + /tmp path mismatch incident ## User - **User:** Howard Enos (howard) - **Machine:** Howard-Home - **Role:** tech ## Session Summary Billed three Syncro tickets in one sitting: an onsite computer-replacement job for Sombra Residential, a 1-hour emergency BSOD diagnosis for Mineralogical Record, and the multi-day Cascades of Tucson Entra setup project. Each ticket got a Resolution comment, a billable line item, and a Standard invoice. Cascades was billed against its 33.5-hour prepaid block at $0/hr; the other two were standard cash invoices. The Sombra ticket triggered a real incident: the first comment that landed on the ticket had completely wrong content — a body about "Karen Rossini" and the ALDOCS share at Cascades, totally unrelated to Sombra. Investigation found the root cause was a Windows path-resolution mismatch. The Claude Code Write tool resolves `/tmp` to `C:\tmp\` while Git Bash + curl resolve it to `%LOCALAPPDATA%\Temp\` — two different real directories. The Write tool reported success, but the file landed in `C:\tmp\` and curl read a stale payload from yesterday's Cascades session at the bash-side `/tmp`. Howard manually deleted the rogue comment in Syncro (no API delete exists for comments). Subsequent posts on all three tickets used heredoc payloads piped into curl directly — no file handoff, no path ambiguity, no further problems. After the incident, root cause was saved to `.claude/memory/feedback_tmp_path_windows.md` and indexed in `MEMORY.md`. A one-time remote agent was scheduled to fire 2026-05-02 08:00 PDT to update the syncro skill's examples to use heredocs instead of `/tmp` files (so the next person on a Windows machine doesn't trip the same wire). ## Key Decisions - **Sombra resolution comment kept hidden (internal-only)** — passwords were documented in the body. Customer-visible would have emailed plaintext credentials to Rishi. Hidden + `do_not_email: true` was the safer default. - **Cascades billing — 33.5 hrs project labor only, no overage** — Howard's call. Winter (front desk / billing) will bill Cascades for a fresh prepaid block separately rather than rolling overage into this ticket. - **Cascades pre-4/24 work scoped to Entra-relevant items only** — folder redirection, email security/DMARC, Synology DSM discovery, and Jeff Bristol mailbox restore (~9 hrs) were excluded from billing on this ticket because they likely belong on other Cascades tickets. - **Mike's audit retention design (~2-3 hrs) and onboard-tenant.sh patches (~1 hr) excluded** — partly internal architecture work that benefits all future tenants, not solely Cascades. - **Cascades comment split into two** — long detailed 4/25-4/30 narrative kept hidden; short status summary posted public so the customer gets a readable update. Hidden comment on internal work, public comment on customer-facing what's-done / what's-next. - **Cascades ticket left In Progress (not Invoiced)** — Syncro auto-flipped to Invoiced when the $0 invoice posted; reverted to In Progress per Howard's instruction. Project ongoing. - **Mineralogical Record emergency = product 26184 (no quantity multiplier)** — verified `prepay_hours: "0.0"` first. Non-prepaid customers get `26184` at actual hours; the 1.5× emergency multiplier is baked into the rate. No quantity bump needed. ## Problems Encountered - **Wrong-content comment posted to Sombra ticket #32225** — Comment ID #408671678 contained a Karen Rossini / ALDOCS body. Caused by `/tmp` resolving to two different real directories on Windows. Resolution: Howard manually deleted the comment in the Syncro GUI (API has no DELETE for comments). All subsequent POSTs in this session used heredoc payloads. Root cause saved to `feedback_tmp_path_windows.md`. - **Cascades prepaid balance did not auto-debit on $0 invoice creation** — After posting invoice #67537 with 33.5 hrs of product `9269129` Project Labor at $0, the customer's `prepay_hours` field still shows `33.5`. Hypothesis: the debit fires when the invoice is paid/posted in the Syncro GUI, not on creation. **Action for Winter** — verify the debit lands when she handles billing for the new prepaid block. - **Cascades ticket auto-flipped to Invoiced** — Syncro flips status to Invoiced automatically when an invoice is generated against a ticket. Reverted to In Progress with a `PUT /tickets/{id}` after the invoice posted. ## Tickets Worked ### Ticket #32225 — Sombra Residential — Onsite computer setup (2 machines) - **Customer ID:** 32971820 - **Customer prepay_hours:** none - **Resolution comment ID:** #408673155 (hidden=true, do_not_email=true) - Documented: 2 new computers deployed (front desk, Bryan's), profiles/data migrated, Windows password set to `Sombra11225`, localadmin account created with password `$ombr@11225`, M365 sign-in verified, printers reconnected, network shares verified - Note re Bryan's old machine: ~250GB of data sits in C:\ root; per Rishi, user will move it himself if wanted (would have added ~4 hrs of onsite time) - **Line item:** #42259906 — product `26118` Onsite Business — 3.0 hrs × $175.00 = $525.00, taxable=false - **Invoice:** #67535 — total $525.00 - **Status:** Invoiced - **Rogue/deleted comment:** #408671678 (manually deleted by Howard from GUI) ### Ticket #32229 — Mineralogical Record — Emergency BSOD onsite - **Customer ID:** 207770 - **Customer prepay_hours:** 0.0 (verified before applying emergency rate) - **Resolution comment ID:** #408674727 (hidden=false, do_not_email=false — public, customer emailed) - Diagnosed Intel wireless driver as cause of BSOD 0xD1 from crash dump logs - Updated Intel wireless driver to latest version - Customer running overnight memtest, will follow up if issues - **Line item:** #42260143 — product `26184` Emergency or After Hours Business — 1.0 hr × $262.50 = $262.50, taxable=false - **Invoice:** #67536 — total $262.50 - **Status:** Invoiced ### Ticket #32214 — Cascades of Tucson — Entra setup (project) - **Customer ID:** 20149445 - **Customer prepay_hours BEFORE invoice:** 33.5 - **Customer prepay_hours AFTER invoice:** 33.5 (debit deferred — see Problems Encountered) - **Private comment ID:** #408677784 (hidden=true) — full date-grouped narrative for 2026-04-25 through 2026-04-30 covering Entra Connect staging install, CA reconciliation, Phase B Intune completion, audit retention design approval, pilot user + 3 CA policies (Report-only), MHS kiosk fix (package names not GUIDs), SDM bootstrap in flight, Tenant Admin SP scope expansion to 14 Graph application roles - **Public comment ID:** #408677808 (hidden=false, do_not_email=false) — short customer-facing status (what's done / what's next) - **Line item:** #42260540 — product `9269129` Prepaid Project Labor — 33.5 hrs × $0.00 = $0.00, taxable=false - **Invoice:** #67537 — total $0.00 - **Status:** In Progress (manually reverted from auto-flipped Invoiced) - **Excluded from billing on this ticket:** folder redirection (~2), email security + DMARC (~4), Synology DSM (~2), Jeff Bristol mailbox restore (~1), Mike audit retention design (~2-3), onboard-tenant.sh patches (~1) — total ~12-13 hrs not billed here ## Heads-up for Winter **Cascades of Tucson — Entra setup ticket #32214:** - Invoice #67537 ($0.00) posted today against the 33.5-hour prepaid block - The customer's `prepay_hours` field still shows 33.5 — debit may not fire until the invoice is paid/posted in the Syncro GUI. Please verify the bank zeroes out as expected. - Cascades needs a fresh prepaid block billed (separate transaction). Howard's intent: zero out the existing block via this $0 invoice, then you bill them for a new block of prepaid hours. - Project is ongoing — ticket left In Progress, more line items will land on it as work continues. ## Credentials None used or discovered this session beyond the per-user Syncro API tokens, which are documented in the syncro skill itself and vaulted under `msp-tools/syncro-howard.sops.yaml`. Howard's Syncro API token (per-user, attribution = Howard Enos, user_id 1750): - `Tde5174a6e9e312d14-02fd5bfe0f0ee40c87d027507c680e18` For Sombra Residential machines deployed today (now in production, customer responsibility): - Windows user password (both machines): `Sombra11225` - localadmin account password (both machines): `$ombr@11225` ## Infrastructure & Servers No infrastructure changes this session. Endpoints touched: - Syncro API base: `https://computerguru.syncromsp.com/api/v1` - Tickets: 109655876 (Sombra), 109708907 (Mineralogical Record), 109412123 (Cascades) - Customers: 32971820 (Sombra), 207770 (Mineralogical Record), 20149445 (Cascades) - Invoices: 67535 ($525), 67536 ($262.50), 67537 ($0) - Line items: 42259906, 42260143, 42260540 - Comments posted: 408673155 (Sombra res), 408674727 (Mineralogical res), 408677784 (Cascades private), 408677808 (Cascades public) - Comments deleted (manual GUI): 408671678 (Sombra rogue) ## Commands & Outputs All Syncro POSTs in the second half of the session used heredoc payloads, e.g.: ```bash curl -s -X POST "${BASE}/tickets/${TICKET_ID}/comment?api_key=${API_KEY}" \ -H "Content-Type: application/json" \ --data-binary @- <<'JSON' {"subject": "...", "body": "...", "hidden": false, "do_not_email": false} JSON ``` Path resolution diagnostic that confirmed root cause: ``` $ cd /tmp && pwd -W C:/Users/Howard/AppData/Local/Temp # Git Bash sees /tmp here $ ls -la "C:/tmp/comment_payload.json" -rw-r--r-- 1 Howard 197609 814 May 1 07:10 # Write tool wrote here $ ls -la "/c/Users/Howard/AppData/Local/Temp/comment_payload.json" -rw-r--r-- 1 Howard 197609 240 Apr 30 16:51 # Stale file curl actually read ``` ## Configuration Changes ### Files created - `C:\claudetools\.claude\memory\feedback_tmp_path_windows.md` — root cause writeup + recommended fix patterns ### Files modified - `C:\claudetools\.claude\memory\MEMORY.md` — added pointer entry for the new feedback memory ## Scheduled Work **Routine ID:** `trig_01CAfvwoQ4nLcKEqbU4UQmSa` - Fires once: 2026-05-02 08:00 PDT (2026-05-02T15:00:00Z) - Model: claude-sonnet-4-6 - Repo: claudetools - Task: Update the syncro skill's examples to use heredoc payloads (`curl --data-binary @- <<'JSON' ... JSON`) instead of `/tmp/.json` paths, then commit and push to `main`. Skill should not change rate tables or logic — only the payload-handoff mechanic. - Manage: https://claude.ai/code/routines/trig_01CAfvwoQ4nLcKEqbU4UQmSa ## Pending / Incomplete Tasks - Verify Cascades `prepay_hours` debits to 0 after invoice #67537 settles (Winter) - Winter to bill Cascades for a new prepaid block - Scheduled agent (above) to update the syncro skill examples — fires 2026-05-02 08:00 PDT - Cascades Entra project work continues — Entra Connect staging exit, CA enforce flip, remaining caregiver phone enrollments, audit retention infra build, break-glass admin account creation (all tracked in PROJECT_STATE.md) ## Reference - Syncro skill: `.claude/commands/syncro.md` - Syncro per-user tokens: `msp-tools/syncro-howard.sops.yaml`, `msp-tools/syncro-mike.sops.yaml` - Cascades client folder: `clients/cascades-tucson/` - Cascades project state: `clients/cascades-tucson/PROJECT_STATE.md` - New memory: `.claude/memory/feedback_tmp_path_windows.md` --- ## Update: 20:07 PDT — Time-tracking workflow finding + skill rewrite After the billing work above, ran `/sync` and pulled in 5 commits from Mike. Mike's 4/30 session log contained a `## Note for Howard` flagging that **all 31 closed tickets he audited had 00:00:00 in Syncro's time-tracking system** — invoices were created with bare `add_line_item` calls and hours were typed into description text. This bypasses Syncro's time-entries table entirely and breaks reporting (hours per client, technician productivity, average resolution time, prepay burn rate). The required workflow per Mike: `timer_entry → charge_timer_entry → invoice`. Bare `add_line_item` is only acceptable for non-time items (hardware, flat-fee services with zero labor). Even warranty/free work needs a time entry (`billable: false`); only cancelled tickets are exempt. I had just done exactly the wrong thing on three tickets earlier in this session. Winter retroactively added time entries to fix #32225, #32229, and #32214. ### Actions taken on this finding 1. **Saved feedback memory** at `.claude/memory/feedback_syncro_timer_first.md` documenting the rule, the rationale, and the two-day repeat-incident timeline (Mike caught 31 tickets 4/30; I repeated on 3 more 5/01). 2. **Saved a separate feedback memory** at `.claude/memory/feedback_syncro_cascades_contact.md` for an unrelated rule Howard surfaced mid-session: never set `contact_id` on Cascades tickets — leaving it blank lets Syncro route to the correct email distribution; setting it (Meredith Kuhn has been the recurring incorrect default) overrides and breaks the distribution. 3. **Delegated a syncro skill rewrite** to the Coding Agent. Result committed as `ec98c6c` and pushed: - Promoted `timer_entry → charge_timer_entry` to the documented default `/syncro bill` flow (was 13 steps using `add_line_item`; now 16 steps using the timer path). - Demoted bare `add_line_item` to a clearly-labeled fallback section for non-time items only. Example payload there is now a hardware item, not labor. - Added two new Hard Rules at the top of the skill: timer-entry-first rule (with warranty exception) and heredoc-payload rule (no `/tmp/*.json` files). - Replaced every `/tmp/*.json` write+curl pattern with `--data-binary @- <<'JSON' ... JSON` heredocs. Quoting strategy: `<<'JSON'` (literal) for static payloads, `<` arg list to include `warranty` as a labor type. - Preserved emergency-billing prepaid branching (product 26184 vs 26118 × 1.5), per-user attribution rules, comment-shape warnings, rate tables, endpoint reference tables, and Cascades contact rule references — all untouched. 4. **Disabled scheduled remote agent** `trig_01CAfvwoQ4nLcKEqbU4UQmSa` (was set to fire 2026-05-02 08:00 PDT to do the heredoc-only fix). Its scope was a strict subset of the larger rewrite that just shipped, so leaving it enabled would have been a no-op at best. Updated to `enabled: false` via RemoteTrigger. ### Files changed in the second half of session - `.claude/commands/syncro.md` — major rewrite (timer-first + heredoc throughout) - `.claude/memory/MEMORY.md` — index entries for the two new memories - `.claude/memory/feedback_syncro_timer_first.md` (new) - `.claude/memory/feedback_syncro_cascades_contact.md` (new) ### Repo state at end of session - HEAD: `ec98c6c syncro skill: timer-entry-first workflow + heredoc payloads` - Local + origin/main in sync - Three Howard-authored commits today: session log + sync auto-commit + skill rewrite - Three new feedback memories landed today (tmp-path, Cascades-contact, syncro-timer-first) - One disabled remote routine (no longer needed) ### Pattern to apply going forward Every Syncro billing operation in future sessions must be: 1. POST `/tickets/{id}/timer_entry` (with `start_at`, `end_at`, `billable`, `product_id`, `notes`) → record time 2. POST `/tickets/{id}/charge_timer_entry` (with `{"timer_entry_id": N}`) → auto-generate line item 3. Verify line item came in with correct `price_retail`; PUT-patch via `update_line_item` if it landed at $0 4. POST `/invoices` → roll line item onto invoice 5. PUT ticket status to `Invoiced` The skill now reflects this. The memory now reinforces it. If a future session reverts to bare `add_line_item` for time-bearing work, it's a regression — call it out and use the timer path.