238 lines
16 KiB
Markdown
238 lines
16 KiB
Markdown
# Session — security.azcomputerguru.com: live deploy + smoke test (found/fixed prod auth bug)
|
|
|
|
## User
|
|
- **User:** Howard Enos (howard)
|
|
- **Machine:** Howard-Home
|
|
- **Role:** tech
|
|
|
|
## Session Summary
|
|
|
|
Continued the security-assessment work (same HOWARD-HOME session; see the two companion logs from
|
|
today). Howard requested: make UI elements slightly larger, test every field for usable info, and a
|
|
big new multi-tenant client portal + ACG sales view. Did the unambiguous pieces, deferred the portal
|
|
to a feature request, then (on Howard's "b" greenlight) deployed to the live IX host and functionally
|
|
tested — which surfaced a production-breaking bug.
|
|
|
|
Bumped the wizard UI sizing ~10-12% (text, inputs, spacing, wider sheet, bigger buttons) and committed
|
|
it. Built `app/fieldcheck.cjs` and audited every questions.json field: 59 fields / 25 scorable, all 25
|
|
emit a usable finding on worst-case answers, no orphan/missing score keys, all findings complete, all 13
|
|
requiredControls resolve — every field yields usable info. Captured the portal vision as FR-1 in
|
|
`FEATURES.md` (personas, no-auto-sync guardrail, quote→active workflow, 3 auth options) and asked the
|
|
foundational auth decision; Howard chose to defer it to a feature request and focus on testing. Added an
|
|
executive summary + top-3 prioritized actions + ACG footer to the export (client = recommendations,
|
|
internal = + ACG service).
|
|
|
|
Deployed to IX (172.16.3.10, cPanel account `azcomputerguru`, docroot
|
|
`/home/azcomputerguru/public_html/security`, PHP 8.1.34) over SSH: backed up the live files, uploaded to
|
|
`.new`, lint-gated (php -l + JSON validate), then atomic-swapped into place. `config.php` (live secrets)
|
|
was never touched. The public URL was already live behind Cloudflare Access (a public hit returns CF
|
|
Access 403), so this was an update, not first-time setup.
|
|
|
|
Smoke-testing on the server found a **production-breaking bug**: `api.php`'s allow-list check did a
|
|
single `strcasecmp($email, ALLOWED_EMAIL)` against the WHOLE comma-separated list, so once a 2nd address
|
|
(howard@) was added on Jun 19, every API call (lookup/save/load/list/export) returned `forbidden` — the
|
|
live backend had been unusable. Fixed it to split + membership-check (matching index.php), redeployed,
|
|
and re-verified: `action=list` reads the DB; internal export renders posture/exec-summary/findings/ACG-
|
|
service/REQUIRED/captured-intake; client export renders posture+recommendations with zero upsell leaks
|
|
and no raw intake. Inserted a weak-answer test row, exported both views, then deleted it — prod DB back
|
|
to its 2 real rows. Committed everything to submodule `main` and advanced the claudetools pin so git
|
|
matches the live server.
|
|
|
|
## Key Decisions
|
|
- **Deploy via root SSH + base64 file upload, lint-gated atomic swap.** Backup → upload `.new` →
|
|
`php -l`/JSON-validate → `mv` into place. Never overwrite `config.php`. Safe for a live site.
|
|
- **Functional-test via PHP CLI + a shipped harness, not inline `php -r`.** Inline `ssh "php -r '...'"`
|
|
mangled through nested quoting (printed PHP usage); a base64'd `.php` harness that includes api.php and
|
|
drives `$_GET[action]` is reliable. Bypasses the Apache/Cloudflare vhost routing (origin-direct curl
|
|
404'd — a vhost/SNI artifact, not a broken deploy).
|
|
- **Test row written to the tool's own MySQL then deleted** — acceptable for a smoke test (its own DB,
|
|
not Syncro/RMM), clearly tagged `ZZ-SMOKE-DELETE`, cleaned up.
|
|
- **Portal deferred to FR-1** (Howard's call) rather than building a multi-tenant auth model speculatively.
|
|
- **MSYS `ssh` for password+askpass**, not native Windows OpenSSH (which fails the .sh askpass) — matches
|
|
the working pfsense-ssh.sh pattern.
|
|
|
|
## Problems Encountered
|
|
- **PROD BUG — api.php 403 for everyone** (single strcasecmp vs comma list). Found via live smoke test;
|
|
fixed + redeployed + verified. This is why "test every function" mattered.
|
|
- **Native Windows OpenSSH askpass failed** `CreateProcessW error:193` (can't exec a .sh askpass) →
|
|
used MSYS bare `ssh`. Logged --friction.
|
|
- **Inline `php -r` over ssh mangled** (nested quote stripping) → shipped a base64'd .php harness.
|
|
Logged --friction (ref `feedback_windows_quote_stripping`).
|
|
- **Origin-direct curl 404'd** (vhost/SNI; CF proxies the real path) → tested via PHP CLI instead.
|
|
- **Submodule reset to old scaffold** earlier (stale gitlink + a concurrent submodule-update) → restored
|
|
to origin/main and advanced the claudetools pin to stop the churn.
|
|
|
|
## Configuration Changes
|
|
security-assessment submodule (all on `main`, pushed; claudetools pin advanced each time):
|
|
- `app/index.php` — UI sizing bump (`66eb7cb`).
|
|
- `app/api.php` — export exec-summary/top-3/footer (`3e3a9ab`); **allow-list auth fix** (`f246091`).
|
|
- `FEATURES.md` (new) — FR-1 portal request; `app/fieldcheck.cjs` (new) — field-audit dev tool (`3a2301b`).
|
|
- Local scratch `app/_deploytest.php` + `app/_smoke.php` created for testing, removed after (not committed).
|
|
Live IX server: `index.php`, `api.php`, `questions.json` updated in the docroot (backups `.bak-20260621-181744`); `config.php` untouched.
|
|
ClaudeTools: this session log + `errorlog.md` (2 friction entries).
|
|
|
|
## Credentials & Secrets
|
|
None created/discovered. Used existing vault `infrastructure/ix-server` (root SSH 172.16.3.10:22, password
|
|
field `credentials.password`; also a WHM API token `credentials.whm-api-token` = full-access root). DB creds
|
|
live in the server's `config.php` (vault `msp-tools/security-assessment-db`). No secrets printed/committed.
|
|
|
|
## Infrastructure & Servers
|
|
- IX server 172.16.3.10 (ext 72.194.62.5), Rocky Linux WHM/cPanel, PHP 8.1.34 (ea-php81), root SSH :22.
|
|
- Site docroot: `/home/azcomputerguru/public_html/security` (cPanel acct `azcomputerguru`, sub of azcomputerguru.com, vhost 172.16.3.10:80/:443).
|
|
- security.azcomputerguru.com behind Cloudflare Access (Zero Trust app `8ce5f31c-...`; allow mike@ + howard@). Origin answers via CF; origin-direct curl needs `--resolve`/Host and still 404'd (CF-proxied path is the working one).
|
|
- Live DB (acgsec_assess): 2 real rows (id 1 Darrell Delphen, id 2 empty) after test cleanup.
|
|
|
|
## Commands & Outputs
|
|
```
|
|
# find docroot
|
|
ssh root@172.16.3.10 'grep -i security.azcomputerguru.com /etc/userdatadomains'
|
|
# -> azcomputerguru==root==sub==azcomputerguru.com==/home/azcomputerguru/public_html/security==...ea-php81
|
|
# deploy pattern: backup -> base64 upload to .new -> php -l -> chown user -> mv (atomic)
|
|
base64 local.php | ssh root@ix "base64 -d > docroot/file.new"
|
|
# functional test (bypasses Apache/CF): shipped harness includes api.php
|
|
php _deploytest.php list # {"items":[{id:2...},{id:1,...}]} (after the auth fix)
|
|
php _deploytest.php export <id> internal # posture/exec/findings/ACG service/Captured intake
|
|
php _deploytest.php export <id> client # posture/recommendations, 0 ACG-service, no Captured intake
|
|
# BUG before fix: php _deploytest.php list -> {"error":"forbidden"}
|
|
```
|
|
|
|
## Pending / Incomplete Tasks
|
|
- **Live browser UI click-through** (through Cloudflare Access) — the only thing not tested; needs a human
|
|
logged into https://security.azcomputerguru.com (look up a client, answer fields, watch posture, Export
|
|
both views). Backend fully verified server-side.
|
|
- **#1 GuruRMM endpoint prefill** — deferred (infra: no Syncro→RMM mapping, no reachable RMM API from IX).
|
|
- **FR-1 multi-tenant portal** — filed in FEATURES.md, awaiting the auth decision when Howard wants it.
|
|
|
|
## Reference Information
|
|
- Submodule `main` HEAD `f246091` (claudetools pin `27c1d97` at time of fix).
|
|
- Commits this turn: `66eb7cb` sizing, `3a2301b` FR-1+fieldcheck, `3e3a9ab` export exec-summary, `f246091` api auth fix.
|
|
- Live server backups: `*.bak-20260621-181744` in the docroot (rollback if needed).
|
|
- Export endpoint: `api.php?action=export&id=<id>&view=internal|client` (origin requires the CF Access email header).
|
|
- Companion logs: `2026-06-21-howard-security-assessment-scoring.md`, `...-unifi-pfsense-control-verbs.md`, `...-gururmm-bug-018-019.md`.
|
|
|
|
---
|
|
|
|
## Update: 18:53 PT — site was broken in browser (stale opcache); root-caused + fixed; CONFIRMED loading
|
|
|
|
After the deploy, Howard reported the live page "mostly broken — only the Export and Assessments
|
|
buttons at the top." Diagnosed and resolved.
|
|
|
|
### What was wrong (two compounding issues)
|
|
1. **api.php allow-list bug** (already fixed earlier this session): single `strcasecmp($email, ALLOWED_EMAIL)`
|
|
vs the whole comma string → 403 for every API call. Confirmed live via the access log (`POST /api.php?action=save 403` repeatedly).
|
|
2. **Stale PHP OPcache** — the server kept executing the OLD `index.php` (rendered output 14637 bytes) and OLD
|
|
`api.php` even though the new files were on disk (md5 matched local). So the new wizard never rendered (boot()
|
|
ran the old code / the old api 403'd) and the fix wasn't live.
|
|
|
|
### Diagnosis path
|
|
- Verified deployed files = local (identical md5), perms 644, `.htaccess` benign, questions.json valid on server,
|
|
no PHP error log → not a crash. Ran the wizard JS in node with DOM/fetch stubs → boot() completed fine (so not
|
|
a code bug). The **Apache SSL access log** (`/etc/apache2/logs/domlogs/security.azcomputerguru.com-ssl_log`) was
|
|
the smoking gun: `GET / 200 14637` (old size; new index.php renders ~27.9k) + many `api.php ... 403`.
|
|
- SAPI = Apache (httpd) + opcache On. Origin-direct curl to 127.0.0.1 hits a default vhost that 404s — the app is
|
|
only reachable via the EXTERNAL IP path Cloudflare uses.
|
|
|
|
### Fix
|
|
- Dropped a one-off `_oc.php` (`opcache_reset()`) in the docroot and ran it through the real path:
|
|
`curl -sk --resolve security.azcomputerguru.com:443:72.194.62.5 -H 'Cf-Access-Authenticated-User-Email: mike@azcomputerguru.com' https://.../_oc.php` → `opcache_reset: OK` (validate_timestamps=1, but the cache hadn't revalidated; reset forced it).
|
|
- Verified: `GET /` now 200 **27885 bytes**, served HTML contains `computeScores`/`gradeFor`/"Posture & findings";
|
|
`api.php?action=list` returns JSON (no 403). Removed `_oc.php` + local scratch (`_jsrun.cjs`, `_deploytest.php`, `_smoke.php`).
|
|
- Howard confirmed: **"it is loading now."**
|
|
|
|
### Durable fixes
|
|
- `DEPLOY.md` updated (`31bc786`): "FLUSH OPCACHE after updating" (the external-IP `_oc.php` recipe) + the
|
|
comma-separated `ALLOWED_EMAIL` gotcha. claudetools pin advanced (`5299f3d`). Logged --friction (deploy/cpanel: stale opcache).
|
|
|
|
### Key gotchas for next deploy
|
|
- After uploading PHP changes to this cPanel host, opcache can serve stale bytecode → flush it (recipe in DEPLOY.md).
|
|
- Origin-direct (127.0.0.1) 404s the vhost; test/flush via `--resolve ...:443:72.194.62.5` (external IP).
|
|
- Coord (Mike, GURU-5070): BUG-018 (cea87d4) + Event Log Watch UI merged to gururmm main + deploying; I'm cleared to self-merge gururmm going forward.
|
|
|
|
### Net state
|
|
security.azcomputerguru.com is LIVE and working (new scoring wizard + larger UI + dual export + fixed backend),
|
|
verified server-side and confirmed loading in Howard's browser. Remaining: #1 RMM prefill (deferred, infra), FR-1 portal (deferred, auth decision).
|
|
|
|
---
|
|
|
|
## Update: 19:31 PT — post-deploy live fixes (wrong-customer lookup bug + New/clear button)
|
|
|
|
Two issues Howard found while using the live site, both fixed + deployed + verified.
|
|
|
|
### Bug: phone lookup returned the WRONG customer
|
|
- Symptom: entering `5205851310` pulled up "Marjolaine Deslauriers" (an unrelated record).
|
|
- Root cause: Syncro's `GET /customers?phone=<n>` param does NOT filter — it returns the full customer
|
|
list (5079 across 51 pages); the code took `customers[0]` (Marjolaine, phone 5203310273). Reproduced
|
|
via the Syncro API (vault `msp-tools/syncro-howard` cred field `credentials.credential`): `?phone=` →
|
|
count 100 unfiltered; `?query=5205851310` → count 0 (number not in Syncro); `?query=5208880000`
|
|
(Insty-Prints' real number) → count 1 (so `?query=` DOES search phone by digits).
|
|
- Fix (api.php lookup): use `?query=<digits>` and VERIFY a candidate's phone digit-tail matches the
|
|
entered digits before prefilling (handles leading-1 / 7-vs-10 digits). No match → return a clear
|
|
"No customer matches that phone number" + suggestions, and prefill NOTHING. index.php doLookup updated
|
|
to show the message/suggestions and not prefill on no-match. Format was never the issue (input is
|
|
digit-normalized, so 5205851310 == (520) 585-1310).
|
|
- Verified live (web POST via the external-IP path): `5205851310` → no-match; `5208880000` → Insty-Prints
|
|
full prefill; `(520) 585-1310` → no-match (same as digits).
|
|
|
|
### Feature: "+ New" button (Howard's request)
|
|
- After loading a client, `state.id` persisted, so a fresh lookup + save could overwrite the previous
|
|
client's record. Added a top-bar **+ New** button -> `clearAll()` resets id/data/consent/sec/view +
|
|
URL + topbar, so the next save INSERTs a new record. Confirms before clearing if work is in progress.
|
|
|
|
### Deploy mechanics (each fix)
|
|
- Upload changed file(s) to `.new` -> `php -l` -> atomic `mv` -> flush opcache via a one-off `_oc.php`
|
|
hit through `--resolve ...:443:72.194.62.5` (external IP; 127.0.0.1 vhost 404s) -> verify served HTML.
|
|
Backups `*.bak-<ts>` in the docroot. Browser needs Ctrl+Shift+R to pick up index.php changes.
|
|
|
|
### Commits
|
|
- `ef03b4e` lookup phone-match fix; `a9c85a7` + New button. claudetools pins advanced (`c462f50`, `5b44a3e`).
|
|
|
|
### Net
|
|
Phone search works correctly (verified by Howard), wrong-customer risk eliminated, and the + New button
|
|
prevents cross-record contamination. Site fully functional.
|
|
|
|
---
|
|
|
|
## Update: 21:25 PT — onboarding/upsell questions + opportunity engine + Save-quote + quote delete
|
|
|
|
Expanded the assessment per Howard ("what other questions to better integrate clients + gather upsell
|
|
data") plus quote lifecycle controls. All deployed to IX + verified live.
|
|
|
|
### Save-quote + status
|
|
- Added a top-bar **Save quote** button + QUOTE badge; added a `status` column (default `quote`) to the
|
|
`assessments` table (ALTER on live DB + schema.sql). New saves are `quote` (unverified, never auto-synced);
|
|
UPDATE leaves status (preserves a future promotion). `list` returns status; Assessments list shows it.
|
|
|
|
### New questions (Howard's "go with recommendations")
|
|
- Security-scored: **M365 license tier** (Basic/Standard -> upgrade to Business Premium; it unlocks MFA/CA/Intune/Defender).
|
|
- New **Other Services** section: VoIP (PacketDial), managed print, surveillance, web hosting.
|
|
- New **Onboarding & Sales Context** section (internal-only): current support model, existing agents to remove,
|
|
admin-access readiness, hardware age, cyber-insurance renewal timing, decision-maker, biggest frustration, planned changes.
|
|
- Plus WiFi coverage, internet failover, UPS.
|
|
|
|
### Opportunity engine (the key design)
|
|
- New field-level `opportunity` block (parallel to `risk`): non-security upsells that fire on a `when`
|
|
trigger ({equals|in|includes|notIncludes}) and surface for SALES WITHOUT affecting the security grade.
|
|
- Engine mirrored JS (`computeScores` returns `opportunities`) + PHP (`score_assessment` + `opp_fires`).
|
|
- Export: INTERNAL shows an "Upsell opportunities" section (each -> ACG service); CLIENT view hides them
|
|
entirely. Wizard internal view shows them; client toggle hides. Verified: 6 opportunities fire on a weak
|
|
sample, security findings stay separate (2).
|
|
|
|
### Quote lifecycle
|
|
- api `promote` (quote->active) + `delete` actions (ACG-gated). Assessments list: **Delete** works
|
|
(confirm + clears the loaded record if it's the one deleted); **Activate/make-live button DISABLED** per
|
|
Howard (live import to Syncro/RMM stays manual until wired; promote API is dormant/ready).
|
|
|
|
### Problem hit
|
|
- First deploy shipped api.php + index.php but NOT the new questions.json -> opportunities/new questions didn't
|
|
appear live (OPPS=0). Root-caused via a CLI opp-debug (server questions.json was old). Deployed questions.json
|
|
(lint-gated) -> verified opportunities fire + new sections live. Lesson: deploy ALL changed files, incl. data.
|
|
|
|
### Commits / state
|
|
- Submodule `0f6927b` (questions+engine+delete) on main; earlier this session `66eb7cb`/`3a2301b`/`3e3a9ab`/
|
|
`f246091`/`a9c85a7`/`7a3dfb8`. claudetools pin `0f6927b` (HEAD `8713b46`). All live on IX, opcache flushed.
|
|
- Concurrent screenconnect WIP in the shared tree preserved through the pin rebases.
|
|
|
|
### Still open
|
|
- Activate/promote UI intentionally disabled (wire live import later). FR-1 portal still deferred (auth decision).
|
|
- #1 RMM prefill deferred (infra). Optional: extend fieldcheck.cjs to validate `opportunity` fields.
|