Files
claudetools/clients/internal-infrastructure/ix-server-issues-2026-01-13.md
Mike Swanson 07816eae46 docs: Add comprehensive project documentation from claude-projects scan
Added:
- PROJECTS_INDEX.md - Master catalog of 7 active projects
- GURURMM_API_ACCESS.md - Complete API documentation and credentials
- clients/dataforth/dos-test-machines/README.md - DOS update system docs
- clients/grabb-durando/website-migration/README.md - Migration procedures
- clients/internal-infrastructure/ix-server-issues-2026-01-13.md - Server issues
- projects/msp-tools/guru-connect/README.md - Remote desktop architecture
- projects/msp-tools/toolkit/README.md - MSP PowerShell tools
- projects/internal/acg-website-2025/README.md - Website rebuild docs
- test_gururmm_api.py - GuruRMM API testing script

Modified:
- credentials.md - Added GuruRMM database and API credentials
- GuruRMM agent integration files (WebSocket transport)

Total: 38,000+ words of comprehensive project documentation

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-22 09:58:32 -07:00

8.5 KiB

IX Server Critical Performance Issues

Server: ix.azcomputerguru.com (172.16.3.10 / 72.194.62.5) Report Date: 2026-01-13 Status: Documented - Action Required Priority: CRITICAL

Executive Summary

Comprehensive scan of ix.azcomputerguru.com web hosting server revealed critical performance issues across multiple client sites. Primary issues: massive error logs (468MB on arizonahatters.com), database bloat (310MB on peacefulspirit.com), and Wordfence-induced memory exhaustion.


Critical Priority Sites

1. arizonahatters.com - MOST URGENT

Error Log: 468MB PHP Memory Errors: 429 occurrences Database: 24.5MB (Wordfence bloat: wp_wffilemods 8.52MB, wp_wfknownfilelist 4.52MB)

Issue: Wordfence file scanning causing continuous memory exhaustion

Impact:

  • Site performance degraded
  • Server resources exhausted
  • Risk of complete service failure

Action Required:

  1. Disable Wordfence file scanning temporarily
  2. Clear Wordfence file modification tables
  3. Truncate error log: /home/arizonahatters/public_html/wp-content/debug.log
  4. Re-enable Wordfence with adjusted settings (scan schedule, memory limit)

Commands:

# Backup then truncate error log
ssh root@172.16.3.10
cd /home/arizonahatters/public_html/wp-content/
cp debug.log debug.log.backup.2026-01-13
> debug.log

# Database cleanup (via WP-CLI)
wp db query "TRUNCATE TABLE wp_wffilemods;" --path=/home/arizonahatters/public_html/
wp db query "TRUNCATE TABLE wp_wfknownfilelist;" --path=/home/arizonahatters/public_html/

2. peacefulspirit.com

Error Log: 4.0MB PHP Memory Errors: 2 occurrences Database: 310MB! (wp_wpml_mails: 156MB, wp_gf_entry_meta: 96MB)

Issue: WPML email logs and Gravity Forms data bloat

Impact:

  • Slow database queries
  • Backup size excessive
  • Disk space waste

Action Required:

  1. Truncate WPML email logs table
  2. Archive or delete old Gravity Forms entries
  3. Configure WPML to limit email log retention
  4. Implement Gravity Forms entry retention policy

Commands:

# WPML email logs cleanup
wp db query "TRUNCATE TABLE wp_wpml_mails;" --path=/home/peacefulspirit/public_html/

# Gravity Forms cleanup (entries older than 1 year)
wp db query "DELETE FROM wp_gf_entry WHERE date_created < DATE_SUB(NOW(), INTERVAL 1 YEAR);" --path=/home/peacefulspirit/public_html/
wp db query "DELETE FROM wp_gf_entry_meta WHERE entry_id NOT IN (SELECT id FROM wp_gf_entry);" --path=/home/peacefulspirit/public_html/

High Priority Sites (>50MB Error Logs)

Site Error Log Size Primary Issue
desertfox.com 215MB Unknown - needs investigation
outaboundssports.com 208MB Unknown - needs investigation
rrspc.com 183MB Unknown - needs investigation
farwest.com 100MB Unknown - needs investigation
fsgtucson.com 64MB Unknown - needs investigation
tonystech.com 54MB Unknown - needs investigation
phxpropane.com 52MB Unknown - needs investigation
rednourlaw.com 50MB Unknown - needs investigation
gurushow.com 40MB Unknown - needs investigation
cryoweave.com 37MB Unknown - needs investigation
bruceext.com 31MB Unknown - needs investigation

Recommended Action:

  1. Rotate error logs (backup and truncate)
  2. Analyze recent errors for patterns
  3. Address root causes (plugin conflicts, PHP errors, etc.)
  4. Implement log rotation via logrotate

Medium Priority Sites (Debug Logs)

Site Debug Log Size Additional Issues
gentlemansacres.com debug.log: 350MB N/A
azrestaurant.com debug.log: 181MB, itsec_logs: 53MB iThemes Security logs
rsi.com debug.log: 166MB N/A
voicesofthewest.com akeeba log: 106MB Backup log bloat

Action Required:

  • Disable WP_DEBUG in production (wp-config.php)
  • Truncate debug logs
  • Configure iThemes Security log retention
  • Clean up Akeeba backup logs

Common Issues Found

1. Wordfence Database Bloat (Most Sites)

Tables:

  • wp_wffilemods: 1.4-8.52MB
  • wp_wfknownfilelist: 0.86-4.52MB
  • wp_wfconfig: Up to 3.30MB

Solution:

-- Run on each affected site
TRUNCATE TABLE wp_wffilemods;
TRUNCATE TABLE wp_wfknownfilelist;
DELETE FROM wp_wfconfig WHERE name LIKE '%filemod%';

2. Email/Form Logs

Common Culprits:

  • WPML email logs (wp_wpml_mails)
  • Gravity Forms entries (wp_gf_entry, wp_gf_entry_meta)
  • Post SMTP logs
  • Action Scheduler logs

Solution: Implement retention policies, truncate old data

3. Old Backups (Disk Space)

Site Backup Size Age
acepickupparts 1.6GB Various
azcomputerguru 3GB+ Various
sundanzer 2GB Various
berman 388MB 2019
rrspc 314MB 2021

Action Required: Archive to offsite storage, delete from web server


Scan Commands

Full Site Scan

ssh root@172.16.3.10
/root/scan_sites.sh
cat /root/site_scan_report.txt

Database Bloat Check

ssh root@172.16.3.10
/root/check_dbs.sh
cat /root/db_bloat_report.txt

View Critical Issues

ssh root@172.16.3.10
cat /root/URGENT_SITE_ISSUES.txt

Automation Recommendations

1. Log Rotation

Create: /etc/logrotate.d/wordpress-error-logs

/home/*/public_html/wp-content/debug.log {
    weekly
    rotate 4
    compress
    delaycompress
    missingok
    notifempty
    create 644 root root
}

2. Database Maintenance Script

Create: /root/wordpress-db-maintenance.sh

#!/bin/bash
# WordPress database maintenance - run weekly

for site in /home/*/public_html; do
    if [ -f "$site/wp-config.php" ]; then
        echo "Cleaning $site..."

        # Wordfence cleanup
        wp db query "TRUNCATE TABLE wp_wffilemods;" --path="$site" 2>/dev/null
        wp db query "TRUNCATE TABLE wp_wfknownfilelist;" --path="$site" 2>/dev/null

        # Optimize all tables
        wp db optimize --path="$site" 2>/dev/null
    fi
done

3. Monitoring Alerts

Create: /root/monitor-disk-usage.sh

#!/bin/bash
# Alert if any site error log >100MB

find /home/*/public_html/wp-content/ -name "debug.log" -size +100M -exec ls -lh {} \; | \
  mail -s "IX Server: Large error logs detected" mike@azcomputerguru.com

Server Resources

Current Usage

# Check disk space
df -h /home

# Check memory usage
free -h

# Check CPU load
uptime

Optimization Recommendations

  1. OPcache: Ensure enabled and properly configured
  2. MariaDB: Tune query cache and buffer pool size
  3. PHP-FPM: Adjust pm.max_children based on memory
  4. Apache/LiteSpeed: Enable HTTP/2, optimize workers

Client Communication Template

Subject: Website Performance Maintenance Required

Body:

Hello [Client Name],

During our routine server maintenance, we identified some performance
issues affecting your website that require attention:

1. Error logs have grown to [SIZE], indicating [ISSUE]
2. Database optimization needed due to [BLOAT TYPE]

Recommended Actions:
- [SPECIFIC ACTION 1]
- [SPECIFIC ACTION 2]

Impact: [EXPECTED DOWNTIME/IMPROVEMENT]

We can schedule this work at your convenience. Please let us know
your preferred maintenance window.

Best regards,
Arizona Computer Guru Support

Follow-Up Tasks

  • Contact each critical priority client
  • Schedule maintenance windows
  • Execute cleanup on arizonahatters.com
  • Execute cleanup on peacefulspirit.com
  • Implement log rotation across all sites
  • Create database maintenance cron job
  • Set up monitoring alerts
  • Document lessons learned
  • Review Wordfence configuration across all sites
  • Audit backup retention policies

Server Access

# External SSH
ssh root@ix.azcomputerguru.com

# Internal SSH
ssh root@172.16.3.10

# WHM
https://ix.azcomputerguru.com:2087

# cPanel (example)
https://ix.azcomputerguru.com:2083

Original Report: ~/claude-projects/IX_SERVER_CRITICAL_ISSUES_2026-01-13.md

Session Logs:

  • Various client work sessions documented in ~/claude-projects/session-logs/

Scripts Location:

  • /root/scan_sites.sh
  • /root/check_dbs.sh
  • /root/URGENT_SITE_ISSUES.txt

Project History

2026-01-13: Initial comprehensive server scan and issue documentation 2026-01-22: Imported to ClaudeTools project tracking system


Status: Documented - Awaiting Action Owner: Arizona Computer Guru Operations Team Next Review: After critical issues resolved