diff --git a/.claude/memory/project_ix_php_fpm_opcache_remediation.md b/.claude/memory/project_ix_php_fpm_opcache_remediation.md index f9bd26d5..86847a62 100644 --- a/.claude/memory/project_ix_php_fpm_opcache_remediation.md +++ b/.claude/memory/project_ix_php_fpm_opcache_remediation.md @@ -1,6 +1,6 @@ --- name: project_ix_php_fpm_opcache_remediation -description: IX fleet-wide PHP is per-request with OPcache cold (server default); fix is per-vhost PHP-FPM. ezfast pilot done. +description: IX fleet-wide PHP WAS per-request/OPcache-cold (server default); fixed 2026-07 — ALL 72 sites moved to PHP-FPM+OPcache, new-account default set to FPM. metadata: type: project --- @@ -28,9 +28,14 @@ ea-php84-php-opcache` first — it was the first ea-php84 FPM site). Raised ea-p (memory_consumption=256, max_accelerated_files=16229, interned_strings_buffer=32); default 4000 files was 99.6% full on a 31-plugin WP. validate_timestamps left ON (2s) so edits go live without manual reset. -**Rollout is PROPOSED, phased, per-site verified with body-check + auto-rollback** (NOT a fleet flip). -Report: `clients/internal-infrastructure/reports/2026-07-20-ix-php-fpm-opcache-remediation.md`. Also file -separately: hightechmortgage.com live 500; sparcoracing/rarengineer/jparkinsonaz/jefffurrier unreachable. +**COMPLETE (2026-07-21):** phased, per-site body-check + auto-rollback rollout finished — pilot(1) + Phase1(10) ++ Phase2(30, incl. all 10 alt-php Selector sites reset native->ea-php84 FPM) + Phase3(31) = **all 72 unsuspended +sites now on FPM+OPcache**, zero rollbacks. `default_accounts_to_fpm=1` set so NEW accounts provision on FPM. +Report: `clients/internal-infrastructure/reports/2026-07-20-ix-php-fpm-opcache-remediation.md`. +Side-fixes: **hightechmortgage.com 500 was a PHP OOM** (memory_limit 128M exhausted by WooCommerce+AccelerateWP); +raised to 512M via `.user.ini`. The **4 "down" sites** (sparcoracing/jefffurrier/rarengineer/jparkinsonaz) are +NOT server issues — IX serves them locally; their public DNS points to registrar-parking / another host / nothing +(business+DNS decision pending; IX public IP 72.194.62.5). Discipline from the same-day SSD-move incident: verify by response BODY not HTTP status; test DB with `mysql --no-defaults -u U -pPASS`; `cagefsctl --remount-all` after socket/path moves. See [[feedback_tmp_path_windows]] for the base64-pipe pattern used to run these remote scripts safely. diff --git a/clients/internal-infrastructure/reports/2026-07-20-ix-php-fpm-opcache-remediation.md b/clients/internal-infrastructure/reports/2026-07-20-ix-php-fpm-opcache-remediation.md index c7ad3bdd..ccf368ee 100644 --- a/clients/internal-infrastructure/reports/2026-07-20-ix-php-fpm-opcache-remediation.md +++ b/clients/internal-infrastructure/reports/2026-07-20-ix-php-fpm-opcache-remediation.md @@ -1,7 +1,7 @@ # Remediation — IX PHP handler / OPcache (fleet-wide) + ezfastautoglass pilot **Date:** 2026-07-20 · **Server:** ix.azcomputerguru.com (CloudLinux 9 + CageFS + cPanel/WHM, Dell R740xd) -**Author:** Mike Swanson (GURU-5070) · **Status:** Pilot + Phase 1 + Phase 2 COMPLETE (41 sites on FPM/OPcache; new-account default set); Phase 3 (remaining ~31 fast-but-cold sites) PROPOSED +**Author:** Mike Swanson (GURU-5070) · **Status:** COMPLETE — all 72 sites on FPM/OPcache (Phases 1-3 done), new-account default set. Fleet-wide per-request PHP eliminated. **Related:** `2026-07-20-ix-mysql-ssd-move-incident-postmortem.md`, `2026-07-20-ix-web-tier-to-ssd-plan.md` --- @@ -132,10 +132,11 @@ not apply under FPM), which cPanel already maintains. 1.70->0.85) — the flat samples were cold-first-hit noise. - **Phase 2b — FPM as default for NEW accounts: DONE (2026-07-21).** `whmapi1 php_set_default_accounts_to_fpm default_accounts_to_fpm=1` (0->1). New cPanel accounts provision on FPM, so the anti-pattern won't silently rebuild. -- **Phase 3 (PROPOSED) — The fast-but-cold remainder** (~31 ea-php suPHP sites already < 1s TTFB; still recompile - per request, so they benefit under load / as plugins grow). Same one-flag `php_fpm=1` per vhost. Lower urgency - since they're not user-visibly slow today; convert opportunistically or in one batch. (New-account default is - already handled — Phase 2b.) +- **Phase 3 — DONE (2026-07-21).** Converted every remaining primary-domain vhost still on suPHP (31 sites) to + FPM, keeping each site's version. 31/31 OK, zero rollbacks, zero skips. Verified against the LOCAL server + (`curl --resolve :443:127.0.0.1`) so DNS-parked/detached domains verified correctly. These were already light + (~0.15s local TTFB), so the gain is persistence under load + consistency, not visible latency. **Result: all 72 + unsuspended sites are now on FPM + OPcache — no per-request PHP remains on the box.** **Per-site procedure (non-negotiable, from the SSD-move incident lessons):** 1. Pre-check bodies clean (abort if already erroring). @@ -178,6 +179,12 @@ privacy-policy HTTP 200, full byte counts; TTFB 0.41-0.48s across repeated warm - **hightechmortgage.com 500: RESOLVED (2026-07-20)** — root cause was PHP OOM (`memory_limit` 128M exhausted by WooCommerce + AccelerateWP + crypto plugins), not the handler. Raised to 512M via `.user.ini`; site returns 200, no new fatals. Will move to FPM in Phase 2 (memory_limit carries over). -- Separately triage the **4 unreachable sites** (HTTP 000: sparcoracing, rarengineer, jparkinsonaz, - jefffurrier) — availability, unrelated to perf. +- **4 "unreachable" sites (HTTP 000) — diagnosed 2026-07-21: NOT server issues.** IX serves all four correctly + when hit locally (`--resolve` to 127.0.0.1 = 200/301); they are unreachable publicly because their **DNS points + away from IX**. Needs a business/DNS decision (server IX public IP = 72.194.62.5): + - **sparcoracing.com** — DNS at registrar parking (3.33/15.197.x). IX serves 301. Repoint to IX or confirm parked. + - **jefffurrier.com** — DNS at registrar parking. IX serves 200. Repoint or confirm parked. + - **rarengineer.com** — DNS -> 184.187.220.69 (a different host). IX serves 200. Site likely migrated away / stale DNS. + - **jparkinsonaz.com** — NO public A record at all (unresolvable; expired domain or DNS not delegated). IX serves 200. + (All four were still converted to FPM in Phase 3, so they're optimized if/when DNS is restored.) - Install `ea-php83-php-opcache` **iff** any site is moved onto ea-php83.