90 lines
9.0 KiB
Markdown
90 lines
9.0 KiB
Markdown
# 2026-06-20 — IX Server: CloudLinux Tuning, WordPress Performance/Security (ezfastautoglass + farwest)
|
||
|
||
## User
|
||
- **User:** Mike Swanson (mike)
|
||
- **Machine:** GURU-5070
|
||
- **Role:** admin
|
||
|
||
## Session Summary
|
||
|
||
Investigated a "white screen" complaint on ezfastautoglass.com (IX-hosted WordPress) and expanded into a CloudLinux/Apache performance + security pass across multiple IX accounts. IX server itself was healthy (load ~2 on 64 cores, 51 GB RAM free, disk 50%/38%). ezfastautoglass.com served HTTP 200 on all tested pages; the white screen was a transient during a plugin/theme update at 13:42 (WP Rocket cached a blank page, self-healed when the cache regenerated at 15:44). Found a latent CloudLinux anti-pattern (VMEM capped at 1 GB) and tuned ezfastautoglass LVE limits (PMEM 2→3 GB, VMEM 1 GB→unlimited, EP 60→80) as a per-account override.
|
||
|
||
Ran a server-wide CloudLinux fault check (`lveinfo`). A handful of accounts were hitting LVE limits: **farwest** (EPf 435, EP+CPU maxed), **fsgtucson** (EPf 229), **rrspc** (EPf 151), **peacefulspirit** (minor), and **ezfastautoglass** (PMemF 154, memory). Right-sized all of them with per-account overrides (CPU/EP/NPROC headroom; left memory alone for the EP accounts since their PMem usage was tiny).
|
||
|
||
Analyzed farwest's traffic (farwestwell.com / reliantpump.services). It was abuse, not organic: a WordPress brute-force (wp-login 882, xmlrpc 259, wp-json user enumeration) from 104.37.191.206 + a distributed scraper of **357 Google Cloud IPs** (rotating fake-Chrome UAs) crawling the expensive uncached /portfolio_* taxonomy pages — exactly the URLs throwing 508s. First attempted .htaccess mitigations, but proved (sanity test: trivial `[F]` rule returned 404 not 403) that .htaccess RewriteRules are not honored on this AccelerateWP/Apache stack. Pivoted to the correct layer: **Imunify360 WAF** — dropped the brute-forcer IP at the network level, and enabled **AI Bot Protection** (balanced) which targets exactly this AI/scraper bot flood (the distributed low-and-slow crawl evades the per-IP DoS limit). Removed the non-working .htaccess block.
|
||
|
||
Disabled two PHP-8-incompatible broken plugins on farwestwell.com (main domain was 500ing): `social-media-icons-widget` (uses removed `create_function()`) and `wordpress-seo`/Yoast (corrupted, querying a malformed table name) — restored the site to 200. Investigated caching: corrected an earlier mis-diagnosis — page caching on the real site reliantpump.services is actually WORKING via WP Rocket (verified 2.68 s uncached → 0.23 s cached); the "inert LiteSpeed cache" applied only to the low-traffic main farwestwell.com. Fixed a real latent bug (stale WPCACHEHOME pointing at the ACG sandbox/template path from a template clone), enabled the free AccelerateWP suite, and removed a redundant WP Super Cache install. Full-page Max Cache via AccelerateWP requires the paid Premium suite (flagged, not purchased).
|
||
|
||
## Key Decisions
|
||
|
||
- **Tuned LVE per-account, not the shared package** — overrides via `lvectl set <uid>` so only the target account changes and package changes don't clobber it. Sized to ~1.5–2× observed peak, not maxed.
|
||
- **VMEM 1 GB → unlimited** on ezfastautoglass — CloudLinux best practice; a VMEM cap causes spurious PHP OOM even with RAM free. Left memory alone for the EP-bound accounts (tiny PMem usage; their issue was CPU/EP).
|
||
- **Did NOT mass-block GCP /16s** — that would be server-wide (affecting every client's GCP traffic). Used Imunify360 AI Bot Protection (targeted, the right tool) instead.
|
||
- **Abandoned .htaccss blocking after proving it's not honored** on this stack (sanity `[F]` test → 404). Moved all blocking to the Imunify360 WAF layer (cache/Apache-independent).
|
||
- **Disabled broken plugins to restore the site** rather than deep-repair Yoast's corrupted tables (priority = site up; reinstall flagged as follow-up).
|
||
- **Removed my redundant WP Super Cache** once WP Rocket was confirmed as the active, working page cache — avoid two-cache conflict.
|
||
|
||
## Problems Encountered
|
||
|
||
- **wp-cli returned empty / fatals at default 128 MB CLI memory** on these plugin-heavy sites. Resolved by running `php -d memory_limit=512M $(command -v wp)` as the account user for all wp-cli calls.
|
||
- **.htaccess mitigations silently ineffective** — UA/IP/xmlrpc blocks returned 200/404 not 403; AllowOverride is All and it's Apache 2.4.68, yet a trivial `[F]` test rule 404'd. Concluded AccelerateWP/vhost intercepts before per-dir rewrite. Pivoted to Imunify360.
|
||
- **Cascading 500s on farwestwell.com** — after disabling social-media-icons-widget, Yoast surfaced as the next fatal (missing/malformed table). Deactivated Yoast too → 200.
|
||
- **Stale WPCACHEHOME** in reliantpump.services wp-config pointed at `/home/acg/public_html/sandbox/wp-template/...` (template-clone artifact), spewing include warnings. Corrected to the real path.
|
||
- **Caching mis-diagnosis** — initially called caching "inert"; it was working (WP Rocket) on the real site. Corrected after a cache-buster vs repeat test.
|
||
|
||
## Configuration Changes
|
||
|
||
CloudLinux LVE per-account overrides (`lvectl set <uid>`, IX server):
|
||
- ezfastautoglass (uid 1084): PMEM 3 GB, VMEM unlimited, EP 80, NPROC 200, SPEED 400.
|
||
- farwest (uid 1019): SPEED 900, EP 150, NPROC 450 (PMEM left unlimited).
|
||
- fsgtucson (uid 1013): SPEED 500, EP 100, NPROC 250.
|
||
- rrspc (uid 1090): SPEED 400, EP 80, NPROC 200.
|
||
- peacefulspirit (uid 1081): SPEED 450, EP 80, NPROC 200.
|
||
|
||
Imunify360 (IX): dropped IP 104.37.191.206 (`ip-list local add --purpose drop`); enabled `WORDPRESS.ai_bot_protection=true` (preset balanced) — server-wide.
|
||
|
||
WordPress (farwest account, /home/farwest/public_html and .../reliantpump.services):
|
||
- Deactivated plugins on farwestwell.com: `social-media-icons-widget`, `wordpress-seo`.
|
||
- reliantpump.services wp-config: WPCACHEHOME corrected to real path (backup `.bak-*`).
|
||
- AccelerateWP free suite enabled (`clwpos-admin enable-feature --users farwest`).
|
||
- Installed then removed WP Super Cache (net no change).
|
||
- Removed the ineffective ACG-MITIGATION .htaccess block from both docroots.
|
||
|
||
No ClaudeTools repo files changed except this session log.
|
||
|
||
## Credentials & Secrets
|
||
|
||
- No new credentials. IX root access via SSH key from GURU-5070 (vault `infrastructure/ix-server.sops.yaml` is the password/WHM-token fallback). No secrets exposed.
|
||
|
||
## Infrastructure & Servers
|
||
|
||
- **IX server**: `ix.azcomputerguru.com` / 172.16.3.10 (ext 72.194.62.5), Apache/2.4.68 (cPanel), CloudLinux, Imunify360 8.13.5, 64 cores / 62 GB. SSH key auth as root from GURU-5070 (Tailscale).
|
||
- **ezfastautoglass.com**: cPanel user `ezfastautoglass` (uid 1084), docroot /home/ezfastautoglass/public_html, ea-php81, WordPress + WooCommerce + WP Rocket + Wordfence + Jetpack. PHP web memory_limit 256 M (.user.ini), CLI 128 M.
|
||
- **farwest**: user `farwest` (uid 1019). Domains: farwestwell.com (docroot /home/farwest/public_html) + addon/sub reliantpump.services = reliantpump.farwestwell.com (docroot /home/farwest/public_html/reliantpump.services, ea-php80). Goodlayers/Elementor marketing site, no commerce. WP Rocket page cache active (via AccelerateWP / clsop).
|
||
|
||
## Commands & Outputs
|
||
|
||
- `lveinfo --period=1d -d --order-by=PMemF|EPf` — found faulting accounts.
|
||
- `lvectl set <uid> --speed=N% --maxEntryProcs=N --nproc=N` ; `lvectl apply <uid>` ; verify `lvectl list-user`.
|
||
- `imunify360-agent ip-list local add --purpose drop 104.37.191.206` → OK.
|
||
- `imunify360-agent config update '{"WORDPRESS": {"ai_bot_protection": true, "ai_bot_protection_preset": "balanced"}}'` → enabled.
|
||
- `clwpos-admin maxcache --user farwest --domain reliantpump.services --enable` → "MAx Cache is not installed" (premium-gated).
|
||
- Caching proof: `curl ?cb=$RANDOM` = 2.68 s (uncached) vs plain url = 0.23 s (WP Rocket cached).
|
||
- Plugin fatal: `Call to undefined function create_function()` (PHP8) in social-media-icons-widget.php:67.
|
||
- wp-cli must run with `php -d memory_limit=512M $(command -v wp)` as the user (128 M CLI fatals).
|
||
|
||
## Pending / Incomplete Tasks
|
||
|
||
- **Re-check farwest faults + access log tomorrow** to confirm AI Bot Protection + WAF + cache tamped down the abuse; then **normalize farwest LVE limits back down** (9-core bump was an attack band-aid).
|
||
- **Re-check ezfastautoglass PMem faults** at the new 3 GB tomorrow.
|
||
- **farwestwell.com**: cleanly reinstall/remove the two broken plugins (social-media-icons-widget abandoned/PHP8; Yoast corrupted tables). Site is up with both deactivated.
|
||
- **ezfastautoglass MSP360 backup reclaim** (separate earlier task) still pending in the MSP360 console.
|
||
- Consider AccelerateWP Premium (Max Cache) for farwest if full-page caching robustness is wanted beyond WP Rocket's current state.
|
||
|
||
## Reference Information
|
||
|
||
- Imunify360 config keys: `WORDPRESS.ai_bot_protection`, `MOD_SEC` (FULL ruleset active), `DOS` (250/30s).
|
||
- farwest abuse: brute-forcer 104.37.191.206 (612 wp-login); GCP scraper across /16s 34.162/186/152/61/57/170/29/122 (357 IPs).
|
||
- Vault: `infrastructure/ix-server.sops.yaml`. Wiki: `wiki/systems/ix-server.md`.
|
||
- Coord: gitea skill broadcast (commit 354754e5) — already deployed to GURU-5070 global skills.
|