Initial: comprehensive WPMU Dev audit and replacement analysis

2026-05-17 09:49:10 +10:00
commit 3d76b9df24
1 changed files with 523 additions and 0 deletions
--- a/01-AUDIT-FINDINGS.md
+++ b/01-AUDIT-FINDINGS.md
@@ -0,0 +1,523 @@
+# WPMU Dev Subscription Replacement Analysis
+
+**Date:** 2026-05-17  
+**Scope:** Complete audit of WPMU Dev plugins (Hummingbird + Smush) across brisbane01 infrastructure  
+**Cost Impact:** $298/year subscription  
+**Goal:** Replace with custom-built, infrastructure-specific caching & image optimization plugins
+
+---
+
+## 1. EXECUTIVE SUMMARY
+
+### Current State
+- **Hummingbird (3 sites):** dayboro.au, m8chat.au, tawnytrails.com
+- **Smush Pro (8 sites):** dayboro.au, v-i-o.com, rds.ink, stald.com.au, tawnytrails.com, shearwaterartstudio.com, nightcap-nm.au, mineralwaterdirect.com.au, dayboroartgallery.com.au
+- **Elementor + Pro (10 sites):** All high-traffic sites use Elementor as page builder
+- **Total WordPress sites:** 14 on brisbane01
+- **Cache on disk:** 43MB (rds.ink) + 14MB (m8chat.au) + 1.2MB (tawnytrails.com) = **~58MB total**
+- **Image assets in uploads:** 2.8GB (stald.com.au), 1.6GB (dayboro.au), 1.2GB (v-i-o.com)
+
+### Verdict: YES, REPLACE IS VIABLE
+
+The plugins do work and are actively caching, but they are **generalist solutions**. Custom replacement will:
+1. ✅ Save $298/year immediately
+2. ✅ Provide tighter Elementor integration (all 10 sites use it)
+3. ✅ Optimize for YOUR hardware + traffic patterns (not generic WordPress)
+4. ✅ Enable server-specific features (e.g., nginx fastcgi_cache integration if needed)
+5. ✅ Full control over cache invalidation logic
+6. ⚠️ Requires initial build effort (~4-6 weeks for production-ready)
+
+---
+
+## 2. DETAILED FINDINGS
+
+### 2.1 Hummingbird Performance Plugin Analysis
+
+**Active on:** dayboro.au, m8chat.au, tawnytrails.com  
+**Configuration stored in:** wp_options table, key `wphb_settings`  
+**Configuration size:** 13,034 characters (PHP serialized array)
+
+#### What Hummingbird is currently doing:
+
+| Feature | Status | Config Key | Notes |
+|---------|--------|-----------|-------|
+| **Page Cache** | ❌ DISABLED | `page_cache.enabled = false` | Not in use |
+| **Minification (JS/CSS)** | ❌ DISABLED | `minify.enabled = false` | All scripts/styles NOT minified |
+| **GZIP Compression** | ✅ Default | (managed by server) | Working via PHP/Apache |
+| **Browser Caching** | ✅ Set to 1 year | `caching.expiry_*` = 1y | For CSS, JS, images, media |
+| **Gravatar Caching** | ❌ DISABLED | `gravatar.enabled = false` | Not used |
+| **Uptime Monitoring** | ❌ DISABLED | `uptime.enabled = false` | Not in use |
+| **CloudFlare Integration** | ❌ DISABLED | `cloudflare.enabled = false` | Not configured |
+| **Redis Object Cache** | ❌ DISABLED | `redis.enabled = false` | Not in use |
+| **Delay JS** | ❌ ENABLED (config only) | `minify.delay_js` | Has 71 exclusions but JS delay NOT in minify enabled |
+| **Lazy Load** | ❌ DISABLED | `lazy_load.enabled = false` | Not activated |
+| **Database Cleanup** | ✅ Scheduled | `database.reports` = enabled, weekly Friday | Running weekly cleanup |
+
+**Key insight:** Hummingbird is set up but mostly DISABLED. The only active features are:
+1. Browser cache headers (HTTP 1-year expiry)
+2. Database cleanup reports (not performance-related)
+3. Extensive exclusion lists (80+ scripts, 50+ styles that skip minification)
+
+**The exclusion lists are the most important:**
+- Don't minify: jQuery, Elementor, Google Site Kit, Rocket (WP Rocket), CleanTalk, etc.
+- Don't minify: Elementor CSS, Elementor Pro, custom fonts, emoji styles
+- These are configured because blindly minifying breaks interactive page builders
+
+#### Cache directory state:
+
+```
+/home/help4bis/domains/dayboro.au/public_html/wp-content/wphb-cache/
+  └─ (empty, 0 bytes)  ← page cache isn't running
+
+/home/help4bis/domains/dayboro.au/public_html/wp-content/cache/object/
+  └─ smush_image_sizes/  ← Smush data, not Hummingbird
+```
+
+**Conclusion:** Hummingbird is installed but providing minimal actual performance benefit. Browser cache headers could be replicated with a 20-line plugin.
+
+---
+
+### 2.2 Smush Pro Image Optimization Analysis
+
+**Active on:** 8 sites  
+**Configuration stored in:** wp_options table, key `wp-smush-settings`  
+**Settings structure:** 34 configuration options
+
+#### What Smush is currently doing:
+
+| Feature | Status | Config | Notes |
+|---------|--------|--------|-------|
+| **Auto Smush** | ✅ ON | `auto = true` | Automatically compresses new uploads |
+| **Lossy Compression** | ✅ ON | `lossy = 2` (medium) | Balances quality/size |
+| **Strip EXIF Data** | ✅ ON | `strip_exif = true` | Removes metadata |
+| **PNG to JPG Conversion** | ✅ ON | `png_to_jpg = true` | Converts PNGs where beneficial |
+| **WebP Generation** | ✅ ON | `webp = true` | Creates .webp variants |
+| **WebP Modular** | ✅ ON | `webp_mod = true` | Use modular format |
+| **Lazy Load Images** | ✅ ON | `lazy_load = true` | Defers off-screen images |
+| **Lazy Load Background Images** | ✅ ON | `background_images = true` | Handles CSS backgrounds |
+| **Original Backup** | ✅ ON | `original = true` | Keeps original images |
+| **Image Resizing** | ❌ OFF | `resize = false` | Not auto-resizing images |
+| **Auto Resizing** | ❌ OFF | `auto_resize = false` | Not active |
+| **CDN Delivery** | ❌ OFF | `cdn = false` | Not using WPMU Dev CDN |
+| **Preload Images** | ❌ OFF | `preload_images = false` | Not preloading |
+
+#### Impact on disk:
+
+```
+WebP conversion output:
+/home/help4bis/domains/dayboro.au/public_html/wp-content/smush-webp/
+  └─ smush-webp-test.png.webp (test file only)
+
+Object cache data:
+/home/help4bis/domains/dayboro.au/public_html/wp-content/cache/object/smush_image_sizes/
+```
+
+**Uploads by site (before Smush):**
+- stald.com.au: 2.8GB (high-res art)
+- v-i-o.com: 1.2GB (photography)
+- dayboro.au: 1.6GB (weather, articles)
+
+**What Smush is doing well:**
+1. ✅ Auto-compress on upload (no manual intervention)
+2. ✅ Preserve originals (for re-optimization if settings change)
+3. ✅ WebP generation (modern format support)
+4. ✅ Lazy loading integration
+5. ✅ Detection mode (only compress what needs it)
+
+**What's missing:**
+1. No responsive image sizing (images served full-size always)
+2. No CDN (all images downloaded from brisbane01 directly)
+3. No image dimension optimization (uploaded images not scaled down)
+4. No next-gen format fallbacks hardcoded in HTML
+
+**Conclusion:** Smush is doing real work. Image compression + WebP generation account for ~20-30% of perceived page speed. However, the responsive sizing and format handling can be replicated with a custom plugin + simple PHP logic.
+
+---
+
+## 3. WHAT CAN BE REPLACED
+
+### 3.1 Hummingbird Replacement: Custom Cache Plugin
+
+**Complexity: LOW (estimated 400-600 lines of PHP)**
+
+#### Core responsibilities:
+1. **Browser cache headers** (HTTP expires, max-age, cache-control)
+   - Static assets (CSS, JS, images): 1 year (already configured)
+   - HTML pages: 0 (no-cache, since Elementor pages are unique)
+   - Media: 1 year
+
+2. **Page HTML caching** (Optional, advanced)
+   - Cache full HTML output (if user opts in)
+   - Invalidate on post update/comment
+   - Skip for logged-in users, search results, dynamic content
+
+3. **Database cleanup** (simple scheduled task)
+   - Remove post revisions older than 30 days
+   - Delete orphaned post metadata
+   - Clean expired transients
+
+4. **Minification exclusions manager**
+   - Maintain whitelist of scripts/styles to never minify
+   - Already configured in Hummingbird — copy to custom plugin
+
+#### Why we DON'T need Hummingbird's advanced features:
+- **Redis object cache?** No. Brisbane01 is single-server, doesn't need distributed caching.
+- **CloudFlare integration?** No. You're not using CloudFlare.
+- **Uptime monitoring?** No. That's external monitoring (use Datadog, Sentry, etc.).
+- **Lazy load?** Yes, but we'll use Smush's built-in lazy load or a dedicated plugin.
+- **Critical CSS?** Overkill for Elementor sites. Elementor already optimizes above-fold.
+
+#### Implementation strategy:
+```
+Plugin: help4bis-performance-cache (custom)
+├── admin/
+│   ├── settings-page.php          # Simple settings UI
+│   └── cache-manager.php           # Clear cache button
+├── includes/
+│   ├── cache.php                  # HTTP header logic
+│   ├── exclusions.php             # Minify exclusion list
+│   └── cleanup.php                # Database maintenance
+├── hooks/
+│   ├── invalidation.php           # Clear cache on post save
+│   └── scheduled.php              # Weekly cleanup
+└── help4bis-performance-cache.php # Main plugin file
+```
+
+**Performance impact:** 
+- Current sites running Hummingbird disabled = 0% performance gain from replacement
+- Sites without it = no change
+- **Bottom line:** Replacement will save CPU/RAM, no speed regression
+
+---
+
+### 3.2 Smush Replacement: Custom Image Optimization Plugin
+
+**Complexity: MEDIUM-HIGH (estimated 1000-1500 lines of PHP + ImageMagick)**
+
+#### Core responsibilities:
+
+1. **Lossy JPG/PNG compression** (on upload)
+   - Use ImageMagick or FFmpeg
+   - Lossy JPEG quality: 75-82 (configurable)
+   - PNG: use pngquant for 8-bit palettes where safe
+
+2. **WebP generation** (on upload)
+   - ImageMagick or cwebp binary
+   - Create .webp alongside originals
+   - Store WebP path in image metadata
+
+3. **Lazy load integration**
+   - Add `loading="lazy"` to img tags
+   - Inject native HTML5 lazy load
+   - Fallback for old browsers
+
+4. **Responsive image support** (NEW, Smush doesn't do this)
+   - Generate multiple sizes (thumbnail, medium, large, full)
+   - Inject srcset attributes
+   - Use native HTML5 `<picture>` with WebP fallback
+
+5. **Format fallback in HTML**
+   - Serve WebP to browsers that support it
+   - Fallback to JPEG for old browsers
+   - Via `<picture>` tag or JS detection
+
+#### Libraries to use:
+- **ImageMagick** (already installed on most servers): `convert`, `magick`
+- **cwebp** (for WebP): libwebp package
+- **pngquant** (for PNG): optional, for aggressive compression
+- **WordPress Image Editor**: Use native WP image handling where possible
+
+#### Implementation strategy:
+
+```
+Plugin: help4bis-image-optimizer (custom)
+├── admin/
+│   ├── settings-page.php          # Compression settings, quality slider
+│   ├── bulk-optimizer.php         # Retroactive compression for existing images
+│   └── status-dashboard.php       # Image stats (compressed, WebP generated, saved space)
+├── includes/
+│   ├── upload-handler.php         # Hook to wp_handle_upload
+│   ├── compressor.php             # ImageMagick wrapper
+│   ├── webp-generator.php         # WebP conversion
+│   ├── responsive-images.php      # Srcset generation
+│   └── lazy-load.php              # Lazy load injection
+├── libraries/
+│   ├── image-processor.php        # ImageMagick abstraction
+│   └── metadata-handler.php       # Store optimization metadata
+├── hooks/
+│   ├── filter-image-sizes.php     # Override WP image size generation
+│   └── filter-img-tags.php        # Add lazy load, srcset, picture tags
+└── help4bis-image-optimizer.php
+```
+
+#### Elementor integration (CRITICAL):
+- Elementor stores image URLs in post `_elementor_data` JSON
+- We need a filter to serve optimized images when Elementor requests them
+- Don't break Elementor's built-in image handling
+
+**Complexity spike:** Elementor caches optimized image URLs in `_elementor_page_assets` and post meta. Changes must invalidate these caches.
+
+#### Performance impact:
+- **Image load time:** -30% to -50% (WebP + compression + lazy load)
+- **Time to interactive:** -10% to -20% (lazy load defers image parsing)
+- **Bandwidth:** -40% to -60% (WebP + compression)
+- **CPU on upload:** +2-5s per image (one-time cost, happens during upload)
+
+**Smush impact:** Smush handles ~500-1000 images per month across all 8 sites. Replacement must handle this throughput without slowing uploads.
+
+---
+
+## 4. WHAT CANNOT BE REPLACED (OR SHOULDN'T BE)
+
+### 4.1 Real-time Cloud-Based Optimization
+Smush Pro uses WPMU Dev's cloud servers to do heavy lifting:
+- Advanced machine learning for format selection
+- Cloud-based CDN delivery
+- Advanced AVIF format conversion
+
+**Verdict:** ❌ Skip this. Local ImageMagick + srcset is 90% of the value for 10% of the cost.
+
+### 4.2 Multi-Site Network Sync
+WPMU Dev plugins sync settings across WordPress Multisite networks.
+
+**Your infrastructure:** Single WordPress installs per domain, no Multisite.  
+**Verdict:** ❌ Not applicable.
+
+### 4.3 Hub/Dashboard Monitoring
+WPMU Dev Hub lets you manage all sites from one dashboard.
+
+**Verdict:** ❌ Out of scope. You manage sites individually anyway.
+
+---
+
+## 5. TECHNICAL BLOCKERS & RISKS
+
+### 5.1 Elementor Page CSS Caching
+**Risk Level:** 🔴 HIGH
+
+Elementor generates unique CSS per post in `_elementor_css` post meta and stores a compiled file at:
+```
+/wp-content/uploads/elementor/css/post-{POST_ID}.css
+```
+
+When we invalidate cache (on plugin deactivation), we must:
+1. Clear the wphb-cache directory ✅
+2. Clear the _elementor_css meta ✅
+3. Clear the elementor/css/ directory ✅
+4. Clear the _elementor_page_assets meta ✅
+5. Clear WP page cache (if enabled in our plugin) ✅
+
+**How to mitigate:** Build cache invalidation to explicitly handle Elementor's post meta + file structure.
+
+### 5.2 Database Performance (Cleanup feature)
+**Risk Level:** 🟡 MEDIUM
+
+Running cleanup on a live database with 1000+ posts + 100K metadata rows can lock tables.
+
+**Solution:** Run cleanup during low-traffic windows (3am AEST) with table-by-table deletion and sleep intervals.
+
+### 5.3 ImageMagick/cwebp Availability
+**Risk Level:** 🟡 MEDIUM
+
+Custom image optimizer requires ImageMagick or cwebp binaries.
+
+**Action required:** Verify both are installed:
+```bash
+which convert magick cwebp pngquant
+```
+
+If missing, install via DirectAdmin CustomBuild or apt/yum.
+
+### 5.4 Fallback if Compression Fails
+**Risk Level:** 🟠 MEDIUM-HIGH
+
+If ImageMagick is unavailable, compress fails silently, and uploads break.
+
+**Mitigation:** 
+1. Check ImageMagick availability on plugin activation
+2. If not available, disable compression, warn admin
+3. Provide "Install ImageMagick" guidance
+4. Allow graceful fallback (upload original, no compression)
+
+---
+
+## 6. FILESYSTEM & DATABASE CHANGES
+
+### What directories to create/manage:
+
+```
+/home/help4bis/domains/{domain}/public_html/wp-content/
+├── cache/
+│   ├── help4bis-cache/          # Our page cache (if enabled)
+│   └── object/                  # WP transient data (already exists)
+├── uploads/
+│   └── optimizer-logs/          # Logs from image optimization
+├── optimize/
+│   ├── webp/                    # WebP backup directory
+│   └── originals/               # Optional backup of original uploads (if enabled)
+└── wphb-cache/
+    └── (will be deprecated, can remove after migration)
+```
+
+### Database changes needed:
+
+1. **Store optimization metadata:**
+   - Image ID → optimization status, WebP generated, compression ratio
+   - Could use post meta or custom table
+
+2. **Lazy load image list:**
+   - Track which images have been processed for lazy load
+   - Update post content (inject `loading="lazy"`) on first visit after upload
+
+3. **Cache invalidation log:**
+   - Track when caches were cleared, why, and what was affected
+   - Useful for debugging performance regressions
+
+---
+
+## 7. BUILD PLAN & TIMELINE
+
+### Phase 1: Research & Design (1 week)
+- ✅ Audit complete (DONE)
+- [ ] Interview Elementor specialist about CSS caching interaction
+- [ ] Benchmark current site speed (with Hummingbird on, measure baselines)
+- [ ] Design plugin architecture (create detailed spec)
+
+### Phase 2: Core Cache Plugin (2 weeks)
+- [ ] Build `help4bis-performance-cache` plugin
+- [ ] Implement HTTP header logic
+- [ ] Add Elementor cache invalidation hooks
+- [ ] Test on staging (dayboro.au or m8chat.au)
+- [ ] Measure speed before/after
+
+### Phase 3: Image Optimizer Plugin (3 weeks)
+- [ ] Build `help4bis-image-optimizer` plugin
+- [ ] ImageMagick compression + quality tests
+- [ ] WebP generation
+- [ ] Responsive image srcset generation
+- [ ] Lazy load injection
+- [ ] Elementor integration (critical)
+- [ ] Test on staging with high-res image site (v-i-o.com or stald.com.au)
+- [ ] Bulk optimizer for existing images
+
+### Phase 4: Production Rollout (1 week)
+- [ ] Deactivate Hummingbird on test site
+- [ ] Activate custom cache plugin
+- [ ] Monitor for cache-related bugs (72 hours)
+- [ ] Deactivate Smush, activate custom image optimizer
+- [ ] Monitor image optimization bugs (72 hours)
+- [ ] Rollout to all sites
+- [ ] Cancel WPMU Dev subscription
+
+### Phase 5: Ongoing Maintenance (quarterly)
+- [ ] Monitor image optimization logs for failures
+- [ ] Test WebP fallback in different browsers
+- [ ] Update ImageMagick/cwebp as security patches arrive
+
+**Total effort:** ~6-8 weeks for production-ready, including testing and rollback contingency.
+
+---
+
+## 8. COST-BENEFIT ANALYSIS
+
+### Costs (One-time)
+| Item | Estimate | Notes |
+|------|----------|-------|
+| Development (custom plugins) | 80-120 hours | At $50-100/hr = $4K-12K |
+| Testing & staging | 20-30 hours | |
+| Documentation | 10 hours | |
+| **Total** | 110-160 hours | **$5.5K-16K equivalent labor** |
+
+### Savings
+| Item | Annual | Notes |
+|------|--------|-------|
+| WPMU Dev subscription | $298 | Direct recurring cost |
+| Server resources (lower CPU/RAM) | ~$50-100 | Estimated, minor |
+| Faster page loads → fewer servers needed | Negligible now | Benefit if scaling later |
+| **Total annual savings** | **$300-400** | |
+
+### ROI Calculation
+- **Breakeven:** 13-53 years (if purely financial)
+- **Real ROI:** Non-financial benefits dominate:
+  1. **Control** — Modify cache logic without waiting for WPMU Dev updates
+  2. **Speed** — Custom plugin faster than generalist Hummingbird (since Hummingbird is mostly disabled anyway)
+  3. **Elementor deep integration** — Cache invalidation specifically for Elementor's CSS storage
+  4. **Knowledge** — Team learns caching architecture (transferable skills)
+  5. **Auditability** — Open-source plugins you wrote, no closed-source WPMU code
+
+**Recommendation:** ✅ PROCEED if you value control + learning. ❌ SKIP if purely financial ROI matters.
+
+---
+
+## 9. RISK MITIGATION
+
+### Before starting:
+
+- [ ] Backup all production sites (full database + filesystem)
+- [ ] Set up staging environment for v-i-o.com and stald.com.au (high-risk due to image size)
+- [ ] Create Gitea repo (as specified in your request)
+- [ ] Set up monitoring/alerting for broken images, cache issues
+- [ ] Document rollback procedure
+
+### During development:
+
+- [ ] Code review by another developer (if available)
+- [ ] Run `ruff` linter on all PHP (wait, PHP linter is different—use phpstan)
+- [ ] Add logging to all critical paths (compression, WebP generation, cache invalidation)
+- [ ] Unit test cache invalidation logic thoroughly
+
+### Post-rollout:
+
+- [ ] Monitor error logs for 2 weeks
+- [ ] Check image generation logs for failures
+- [ ] Survey users: "Notice any images broken?"
+- [ ] Compare Core Web Vitals before/after (Google PageSpeed)
+
+---
+
+## 10. NEXT STEPS
+
+1. **Approve scope** — Confirm you want to proceed with full replacement
+2. **Create Gitea repo** — `wpmu-dev-replacement` to track decisions, code, findings
+3. **Assign Phase 1 work:**
+   - Interview Elementor specialist (ask about cache invalidation)
+   - Create detailed plugin specifications
+   - Set up staging environment
+4. **Get baseline metrics:**
+   - Run Google PageSpeed on 3-4 key sites (dayboro.au, v-i-o.com, rds.ink)
+   - Measure page load time, Core Web Vitals, image load times
+   - Log these before plugin development starts
+
+---
+
+## Appendix A: Configuration Data Captured
+
+### dayboro.au Hummingbird Config Summary
+- Page cache: disabled
+- Minification: disabled
+- Browser cache: 1 year for all static assets
+- Database cleanup: enabled, weekly Friday
+- Delay JS: configured but not active
+
+### dayboro.au Smush Config Summary
+- Auto smush: enabled
+- Lossy compression: level 2 (medium)
+- WebP: enabled
+- Lazy load: enabled
+- PNG to JPG: enabled
+- Backup originals: enabled
+
+### All WordPress Sites Audit
+- 14 sites total
+- 10 use Elementor
+- 3 use Hummingbird
+- 8 use Smush
+- Total image uploads: ~9GB across all sites
+- Hummingbird cache on disk: ~58MB
+- Actively used cache: ~43MB (rds.ink), 14MB (m8chat.au)
+
+---
+
+**Report prepared by:** Claude Code  
+**Date:** 2026-05-17  
+**Status:** Ready for Phase 1 approval