Add: detailed technical specification for both plugins

02-TECHNICAL-SPECIFICATION.md

@@ -0,0 +1,541 @@
+# Technical Specification: Custom Caching & Image Optimization Plugins
+
+## Document Purpose
+This spec defines the exact architecture, API, and integration points for the two replacement plugins:
+1. `help4bis-performance-cache` — HTTP caching + database cleanup
+2. `help4bis-image-optimizer` — Image compression, WebP, lazy load, responsive images
+
+---
+
+## Plugin 1: help4bis-performance-cache
+
+### Scope
+- HTTP cache headers (browser-level, not page HTML caching)
+- Database cleanup (revisions, orphaned meta, expired transients)
+- Cache invalidation hooks
+- Elementor CSS cache integration
+- Simple admin UI (settings, clear button)
+
+### NOT in scope
+- Page HTML caching (complex, low ROI given Elementor sites are mostly dynamic)
+- Redis/memcached integration (single-server, not needed)
+- CloudFlare integration
+- Lazy loading (handled by image optimizer)
+
+### File Structure
+```
+help4bis-performance-cache/
+├── help4bis-performance-cache.php     # Plugin header + activation hooks
+├── admin/
+│   ├── class-settings-page.php        # WP admin page
+│   ├── class-cache-manager.php        # Clear cache handler
+│   └── partials/
+│       └── settings.html              # Settings form
+├── includes/
+│   ├── class-cache.php                # HTTP header logic
+│   ├── class-exclusions.php           # Minify exclusion manager
+│   ├── class-elementor-integration.php # Elementor cache invalidation
+│   └── class-database-cleanup.php     # Revision/transient cleanup
+├── hooks/
+│   ├── class-invalidation.php         # Post save/delete hooks
+│   └── class-scheduled-cleanup.php    # Cron jobs
+└── tests/
+    ├── test-cache.php
+    ├── test-invalidation.php
+    └── test-elementor-integration.php
+```
+
+### Configuration (in wp_options)
+```php
+// Single option containing all settings
+'h4b_cache_settings' => array(
+  'enabled' => true,
+  'browser_cache_css' => '1y',         // CSS browser cache
+  'browser_cache_js' => '1y',          // JS browser cache
+  'browser_cache_images' => '1y',      // Images browser cache
+  'browser_cache_media' => '1y',       // Media files browser cache
+  'browser_cache_html' => '0',         // HTML (always no-cache for dynamic)
+  'auto_cleanup' => true,
+  'cleanup_frequency' => 'weekly',     // weekly, biweekly, monthly
+  'cleanup_day' => 'friday',
+  'cleanup_time' => '03:00',           // 3 AM AEST
+  'cleanup_revisions_keep' => 30,      // Days of revisions to keep
+  'cleanup_transient_expired' => true, // Delete expired transients
+  'cleanup_orphaned_meta' => true,     // Delete orphaned postmeta
+  'exclude_scripts' => array(          // Scripts that skip minify (if added later)
+    'jquery',
+    'elementor-pro-app',
+    // ... (copy from Hummingbird config)
+  ),
+  'exclude_styles' => array(),         // Styles that skip minify
+);
+```
+
+### HTTP Headers Set by Plugin
+
+On all requests, set these headers:
+
+```
+Cache-Control: public, max-age=31536000   (for .css, .js, .jpg, .png, .gif, .woff, .woff2)
+Cache-Control: no-cache, no-store, must-revalidate (for .html, /wp-admin/, /wp-json/)
+Expires: <1 year from now>
+Vary: Accept-Encoding
+```
+
+For static assets, also set:
+```
+ETag: "<hash-of-file>"
+Last-Modified: <file-mtime>
+```
+
+### Elementor Integration
+
+**Problem:** Elementor caches compiled CSS in `_elementor_css` and posts with CSS file at:
+```
+/wp-content/uploads/elementor/css/post-{POST_ID}.css
+```
+
+**Solution:** Hook into Elementor's save action and clear our cache:
+
+```php
+add_action('elementor/editor/before_save', function($post_id) {
+  // Clear Elementor's CSS cache for this post
+  delete_post_meta($post_id, '_elementor_css');
+  delete_post_meta($post_id, '_elementor_page_assets');
+  
+  // Also clear the CSS file
+  $css_file = WP_CONTENT_DIR . "/uploads/elementor/css/post-{$post_id}.css";
+  if (file_exists($css_file)) {
+    unlink($css_file);
+  }
+});
+```
+
+### Admin UI
+
+Settings page at: **Dashboard → Performance → Cache**
+
+Fields:
+- [ ] Enable caching
+- Browser cache duration (select: 6m / 1y / custom)
+- [x] Auto-cleanup enabled
+- Cleanup frequency (select: weekly / biweekly / monthly)
+- Cleanup day + time (select day, input time)
+- [x] Delete post revisions older than X days (input: 30)
+- [x] Delete expired transients
+- [x] Delete orphaned postmeta
+- Button: "Clear cache now"
+
+Status display:
+- Last cache clear: [timestamp]
+- Last cleanup run: [timestamp]
+- Cache size on disk: [MB]
+
+### Activation Checklist
+
+On plugin activation:
+- [ ] Check WordPress version >= 5.0
+- [ ] Check PHP >= 7.4
+- [ ] Create default settings in wp_options
+- [ ] Schedule first cleanup cron job
+- [ ] Log activation in error_log
+
+On plugin deactivation:
+- [ ] Unschedule cleanup cron job
+- [ ] Do NOT delete settings (user might reactivate)
+- [ ] Log deactivation
+
+---
+
+## Plugin 2: help4bis-image-optimizer
+
+### Scope
+- Lossy compression (JPEG quality adjustment)
+- WebP generation on upload
+- Lazy load injection (native HTML5)
+- Responsive image srcset generation
+- Format fallback (WebP with JPEG fallback)
+- Bulk optimization for existing images
+- Elementor integration (image URL rewriting)
+
+### NOT in scope
+- CDN delivery (local only)
+- Advanced ML-based format detection (use simple rules: JPEG→WebP, PNG→WebP if large)
+- AVIF format (too new, poor browser support)
+- Responsive image srcset for post_content HTML (only for featured images + gallery)
+
+### File Structure
+```
+help4bis-image-optimizer/
+├── help4bis-image-optimizer.php
+├── admin/
+│   ├── class-settings-page.php
+│   ├── class-bulk-optimizer.php      # Background processing existing images
+│   ├── class-status-dashboard.php    # Stats: images optimized, space saved
+│   └── partials/
+│       ├── settings.html
+│       ├── bulk-optimizer.html
+│       └── dashboard.html
+├── includes/
+│   ├── class-upload-handler.php      # Hook wp_handle_upload
+│   ├── class-compressor.php          # ImageMagick wrapper
+│   ├── class-webp-generator.php      # WebP conversion
+│   ├── class-responsive-images.php   # Srcset generation
+│   ├── class-lazy-load.php           # lazy load injection
+│   ├── class-elementor-integration.php # Override Elementor image URLs
+│   ├── class-image-metadata.php      # Store optimization status in postmeta
+│   └── class-fallback-handler.php    # Graceful degradation if ImageMagick fails
+├── hooks/
+│   ├── class-filters.php             # img tag filters
+│   └── class-actions.php             # Upload actions
+├── vendor/
+│   └── image-processor.php           # ImageMagick abstraction layer
+└── tests/
+    ├── test-compression.php
+    ├── test-webp-generation.php
+    ├── test-lazy-load.php
+    └── test-elementor-integration.php
+```
+
+### Configuration (in wp_options)
+```php
+'h4b_image_optimizer_settings' => array(
+  'enabled' => true,
+  'compression_enabled' => true,
+  'jpeg_quality' => 78,                // 70-85, balance quality/size
+  'png_aggressive' => false,           // Use pngquant if available
+  'webp_enabled' => true,
+  'webp_quality' => 78,                // Same as JPEG
+  'lazy_load_enabled' => true,
+  'lazy_load_threshold' => 0,          // 0 = all images, 1000 = skip first 1000px
+  'responsive_srcset' => true,         // Generate srcset for featured images
+  'backup_originals' => true,          // Keep backup of pre-optimization images
+  'bulk_optimize_batch_size' => 5,     // Process 5 images per cron run (don't overload)
+  'preserve_exif' => false,            // Strip metadata
+  'skip_images_smaller_than' => 50000, // Don't compress tiny images (50KB)
+  'imagemagick_available' => true,     // Auto-detected on activation
+  'fallback_mode' => 'preserve',       // preserve=upload original, skip=don't upload
+);
+```
+
+### HTTP Upload Handler
+
+**Hook:** `wp_handle_upload_prefilter` / `wp_handle_upload`
+
+**Process:**
+1. User uploads image via WordPress media uploader
+2. WordPress saves the file (e.g., `/uploads/2026/05/image.jpg`)
+3. Our plugin intercepts:
+   ```php
+   add_filter('wp_handle_upload', function($upload) {
+     if (is_image($upload['file'])) {
+       compress_image($upload['file']);
+       generate_webp($upload['file']);
+       update_image_metadata($upload['file']);
+     }
+     return $upload;
+   });
+   ```
+4. Compression + WebP generation (1-5 seconds per image)
+5. Store metadata: `_h4b_optimized = true, _h4b_webp_path = ...`
+
+### Image Compression Logic
+
+```php
+class Compressor {
+  public function compress_jpeg($src, $quality = 78) {
+    $cmd = sprintf(
+      "convert '%s' -quality %d -strip '%s'",
+      escapeshellarg($src),
+      intval($quality),
+      escapeshellarg($src)
+    );
+    exec($cmd, $output, $ret);
+    return $ret === 0;  // 0 = success
+  }
+
+  public function compress_png($src, $aggressive = false) {
+    if ($aggressive && command_exists('pngquant')) {
+      // Reduce palette to 256 colors
+      $cmd = sprintf("pngquant 256 --strip '%s' -o '%s'",
+        escapeshellarg($src), escapeshellarg($src));
+      exec($cmd);
+    } else {
+      // Just strip metadata
+      $cmd = sprintf("convert '%s' -strip '%s'",
+        escapeshellarg($src), escapeshellarg($src));
+      exec($cmd);
+    }
+  }
+}
+```
+
+### WebP Generation
+
+```php
+class WebpGenerator {
+  public function generate_webp($src) {
+    $webp_path = preg_replace('/\.(jpg|jpeg|png)$/i', '.webp', $src);
+    
+    if (file_exists($src)) {
+      // Use cwebp if available (faster), else ImageMagick
+      if (command_exists('cwebp')) {
+        $cmd = sprintf("cwebp -q 78 '%s' -o '%s'",
+          escapeshellarg($src), escapeshellarg($webp_path));
+      } else {
+        $cmd = sprintf("convert '%s' -define webp:method=6 '%s'",
+          escapeshellarg($src), escapeshellarg($webp_path));
+      }
+      
+      exec($cmd, $output, $ret);
+      
+      // Check output size; if WebP isn't smaller, delete it
+      if ($ret === 0 && filesize($webp_path) < filesize($src)) {
+        return true;  // WebP created and is smaller
+      } else {
+        @unlink($webp_path);
+        return false; // WebP not beneficial, stick with original
+      }
+    }
+  }
+}
+```
+
+### Lazy Load Injection
+
+**HTML before:**
+```html
+<img src="/uploads/2026/05/image.jpg" alt="..." class="wp-image-123" />
+```
+
+**HTML after (our plugin):**
+```html
+<img 
+  src="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg'%3E%3C/svg%3E"
+  data-src="/uploads/2026/05/image.jpg"
+  alt="..."
+  class="wp-image-123"
+  loading="lazy"
+  width="800"
+  height="600"
+/>
+```
+
+**Implementation:**
+```php
+add_filter('the_content', function($content) {
+  return preg_replace_callback(
+    '/<img\s+([^>]*?)src="([^"]*?)"([^>]*)>/i',
+    function($matches) {
+      $attrs = $matches[1] . $matches[3];
+      $src = $matches[2];
+      
+      // Don't lazy-load if already has loading="lazy"
+      if (strpos($attrs, 'loading=') !== false) {
+        return $matches[0];
+      }
+      
+      // Create placeholder SVG (1x1 transparent)
+      $placeholder = "data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg'%3E%3C/svg%3E";
+      
+      return sprintf(
+        '<img %s src="%s" data-src="%s" loading="lazy" />',
+        $attrs,
+        $placeholder,
+        esc_url($src)
+      );
+    },
+    $content
+  );
+});
+```
+
+**Browser native support:** `loading="lazy"` is supported in all modern browsers (2024+). Old browsers ignore the attribute and load normally.
+
+### Responsive Images (Srcset)
+
+For featured images, generate multiple sizes:
+
+```php
+// On upload, if it's a featured image, create these sizes:
+// - thumbnail: 150×150
+// - medium: 300×300
+// - medium_large: 768×768
+// - large: 1024×1024
+// - full: original size
+
+// Then inject into featured image markup:
+$sizes_srcset = array();
+foreach ([150, 300, 768, 1024, 0] as $size) {
+  $src = wp_get_attachment_image_src($attach_id, $size);
+  if ($src) {
+    $sizes_srcset[] = sprintf("%s %dw", $src[0], $src[1]);
+  }
+}
+
+// Result:
+<img 
+  src="..." 
+  srcset="image-150.jpg 150w, image-300.jpg 300w, image-768.jpg 768w, ..."
+  sizes="(max-width: 600px) 100vw, (max-width: 1200px) 50vw, 33vw"
+  alt="..."
+/>
+```
+
+### Elementor Integration
+
+**Problem:** Elementor stores image URLs in `_elementor_data` JSON. When we show optimized images, Elementor's cache might be stale.
+
+**Solution:** On image optimization, invalidate Elementor's post assets:
+
+```php
+add_action('h4b_image_optimized', function($attachment_id) {
+  // Find all Elementor posts that reference this image
+  $posts = get_posts(array(
+    'meta_query' => array(
+      array(
+        'key' => '_elementor_data',
+        'compare' => 'LIKE',
+        'value' => $attachment_id,
+      ),
+    ),
+  ));
+  
+  foreach ($posts as $post) {
+    // Clear Elementor's CSS and asset cache for this post
+    delete_post_meta($post->ID, '_elementor_page_assets');
+    delete_post_meta($post->ID, '_elementor_css');
+  }
+});
+```
+
+Also, hook into Elementor's image rendering:
+
+```php
+add_filter('elementor/rendering/render_element/final_html', function($html) {
+  // Inject lazy load into all img tags within Elementor rendered output
+  return preg_replace_callback('/<img\s+([^>]*)>/i', function($matches) {
+    // Add lazy load if not present
+    if (strpos($matches[1], 'loading=') === false) {
+      return '<img ' . $matches[1] . ' loading="lazy" />';
+    }
+    return $matches[0];
+  }, $html);
+});
+```
+
+### Bulk Optimizer for Existing Images
+
+**Process:**
+1. User clicks "Optimize all existing images" in admin
+2. Plugin creates background cron task
+3. Each cron run processes 5 images (configurable batch size)
+4. Progress bar in admin: "Optimized 245 of 2,150 images (11%)"
+5. Takes 5-10 minutes for 2000 images (spread across multiple cron runs)
+
+**Implementation:**
+```php
+// On bulk optimizer start:
+update_option('h4b_bulk_optimize_status', array(
+  'total' => $total_attachments,
+  'processed' => 0,
+  'started' => current_time('mysql'),
+  'batch_size' => 5,
+));
+
+// Cron job every 2 minutes:
+wp_schedule_event(current_time('timestamp'), '2min', 'h4b_bulk_optimize_cron');
+
+add_action('h4b_bulk_optimize_cron', function() {
+  $status = get_option('h4b_bulk_optimize_status');
+  if (!$status || $status['processed'] >= $status['total']) {
+    wp_clear_scheduled_hook('h4b_bulk_optimize_cron');
+    return;
+  }
+  
+  // Get next batch
+  $batch = new WP_Query(array(
+    'post_type' => 'attachment',
+    'post_mime_type' => 'image',
+    'posts_per_page' => $status['batch_size'],
+    'offset' => $status['processed'],
+  ));
+  
+  foreach ($batch->posts as $attachment) {
+    compress_image($attachment->ID);
+    generate_webp($attachment->ID);
+  }
+  
+  $status['processed'] += count($batch->posts);
+  update_option('h4b_bulk_optimize_status', $status);
+});
+```
+
+### Fallback & Error Handling
+
+**If ImageMagick is not available:**
+
+On activation, check:
+```php
+register_activation_hook(__FILE__, function() {
+  $has_convert = shell_exec('which convert 2>/dev/null');
+  $has_magick = shell_exec('which magick 2>/dev/null');
+  $has_cwebp = shell_exec('which cwebp 2>/dev/null');
+  
+  if (!$has_convert && !$has_magick) {
+    // ImageMagick not installed
+    update_option('h4b_imagemagick_available', false);
+    add_action('admin_notices', function() {
+      echo '<div class="notice notice-error">';
+      echo '<p><strong>Image Optimizer:</strong> ImageMagick not found. Install via: apt install imagemagick</p>';
+      echo '</div>';
+    });
+  }
+});
+```
+
+**Graceful fallback:** If compression fails, upload the original image uncompressed. Log the error, but don't break the upload.
+
+---
+
+## Testing Strategy
+
+### Plugin 1: Cache
+- [ ] HTTP headers are set correctly on requests for .css, .js, .jpg
+- [ ] Elementor CSS cache is cleared when post is saved
+- [ ] Database cleanup runs on schedule and removes revisions
+- [ ] Orphaned postmeta is cleaned up
+- [ ] Cache clear button works
+
+### Plugin 2: Image Optimizer
+- [ ] JPEG images are compressed to target quality
+- [ ] WebP is generated and is smaller than JPEG
+- [ ] Lazy load tag is injected
+- [ ] Elementor image URLs are rewritten
+- [ ] Bulk optimizer progresses correctly
+- [ ] If ImageMagick unavailable, original image is uploaded (fallback)
+- [ ] Featured image srcset is generated correctly
+
+### Integration Testing
+- [ ] Disable Hummingbird, enable cache plugin → no speed regression
+- [ ] Disable Smush, enable image optimizer → speed improvement or equal
+- [ ] All Elementor pages still display correctly
+- [ ] All image galleries still work
+
+---
+
+## Deployment Checklist
+
+- [ ] Code reviewed
+- [ ] All tests passing
+- [ ] Ruff linter clean
+- [ ] Backup of production database taken
+- [ ] Staging environment mirrors production
+- [ ] ImageMagick + cwebp verified installed
+- [ ] Hummingbird deactivated, cache plugin activated (1 test site)
+- [ ] Monitor for 72 hours
+- [ ] Smush deactivated, image optimizer activated (1 test site)
+- [ ] Monitor for 72 hours
+- [ ] Roll out to remaining sites
+- [ ] WPMU Dev subscription cancelled
+