commit 335d9a76e159ad522312d2d26528fd7b6ed5c9f4 Author: help4bis Date: Thu May 14 05:56:49 2026 +1000 Initial SEO-INTEL documentation: architecture, scoring, code structure Add comprehensive documentation for the dual-engine performance evaluation system: - System architecture and data flow - Score calculation methodology (0-100 approximation from CWV thresholds) - Detailed metrics reference (LCP, FCP, CLS, TBT, TTFB) - Testing engines comparison (Sitespeed vs PSI) - Complete code structure map (file-by-file breakdown) - Case study: rds.ink 77 score with actionable fixes - Quick reference guides for interpreting results Co-Authored-By: Claude Haiku 4.5 diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..a94e01f --- /dev/null +++ b/.gitignore @@ -0,0 +1,12 @@ +__pycache__/ +*.py[cod] +*.pyc +.DS_Store +.env +.venv +venv/ +*.swp +*.swo +*~ +.idea/ +.vscode/ diff --git a/README.md b/README.md new file mode 100644 index 0000000..faed6eb --- /dev/null +++ b/README.md @@ -0,0 +1,160 @@ +# SEO-INTEL Performance Evaluation System + +Complete technical documentation for the dual-engine performance measurement platform running at `http://192.168.0.117:8765/performance/`. + +## Overview + +SEO-INTEL is a custom-built performance evaluation system that measures web page speed across your portfolio using two independent engines: + +- **Sitespeed.io** — Real browser testing with HAR waterfall capture +- **Google PageSpeed Insights** — Official Lighthouse audits + +This repo documents how the system works, how scores are calculated, what each component does, and how to interpret results. + +## Quick Navigation + +### Understanding the System +- **[System Architecture](docs/01-architecture.md)** — Data flow, engines, database schema +- **[Score Calculation](docs/02-score-calculation.md)** — How the 0-100 performance score is derived +- **[Performance Metrics Reference](docs/03-metrics-reference.md)** — What each CWV metric means and thresholds +- **[Testing Engines](docs/04-testing-engines.md)** — Sitespeed vs PSI detailed comparison + +### Code References +- **[Code Structure Map](code-refs/file-structure.md)** — Every file and what it does +- **[Database Schema](code-refs/database-schema.md)** — Four tables, all fields, relationships +- **[API Endpoints](code-refs/api-endpoints.md)** — HTTP routes and payloads +- **[Configuration & Thresholds](code-refs/thresholds.md)** — Hard-coded scoring rules + +### Guides & Troubleshooting +- **[How to Interpret a Score](guides/interpreting-scores.md)** — Why is my page 77? What do I fix? +- **[Testing Workflow](guides/testing-workflow.md)** — Click-by-click guide through a test run +- **[Diagrams](diagrams/)** — Visual architecture, data flow, scoring algorithm + +### Case Studies +- **[Case: rds.ink/drawings-of-endangered-animals-series/ = 77](case-studies/rds-77-score.md)** — Real example with actionable fixes + +## Key Facts + +| Aspect | Detail | +|--------|--------| +| **UI Location** | http://192.168.0.117:8765/performance/ | +| **Code Location** | `/home/help4bis/seo-intel/` | +| **Database** | SQLite at `seo-intel/data/seo-intel.db` | +| **Test Engines** | Sitespeed.io (Docker v40.4.0) + Google PSI API | +| **Score Range** | 0–100 (approximated from CWV thresholds) | +| **Testing Latency** | ~90s per URL (sitespeed 60s + PSI 30s) | +| **Portfolio Coverage** | 13 sites, ~6 URLs each, tested weekly | +| **Tables** | perf_runs, perf_audits, perf_opportunities, perf_resources | + +## The Score Explained (TL;DR) + +Your performance score is the **average** of five metrics, each scored 0–100: + +``` +Performance Score = Average(LCP_score, FCP_score, CLS_score, TBT_score, TTFB_score) + +Where each metric is scored based on: + ≤ Good threshold = 100 points + ≥ Poor threshold = 30 points + Between = linear interpolation + +Example: TBT of 1,807ms + Good: 200ms | Poor: 600ms + 1,807 is way beyond 600 → score = 30 points ← killer metric +``` + +## For Different Audiences + +### I'm a Performance Marketer +Start with **[How to Interpret a Score](guides/interpreting-scores.md)** and the **[Case Study](case-studies/rds-77-score.md)**. You'll learn what each number means and how to explain fixes to clients. + +### I'm a Developer +Read **[System Architecture](docs/01-architecture.md)**, then dive into **[Code Structure](code-refs/file-structure.md)** and **[Database Schema](code-refs/database-schema.md)**. Everything you need to modify the system is there. + +### I'm a DevOps Engineer +Check **[Testing Engines](docs/04-testing-engines.md)** (Docker setup), **[API Endpoints](code-refs/api-endpoints.md)** (how to trigger tests), and **[Configuration](code-refs/thresholds.md)** (what to tune). + +### I Just Want to Fix My Site's Score +Go straight to **[How to Interpret a Score](guides/interpreting-scores.md)** with your site's name. The guide will show you exactly what's slow and how to fix it using Hummingbird. + +## Key Insight: Two Different Scores + +| Score | Source | What It Measures | Use For | +|-------|--------|------------------|---------| +| **Sitespeed Score (77)** | Approximated from CWV | Trend tracking | Internal comparisons, weekly monitoring | +| **PSI Score (unknown)** | Official Google Lighthouse | Official Google score | Client benchmarking, official audits | +| **Individual Metrics** (TBT=1,807ms) | Real browser via Browsertime | Root cause diagnosis | Finding bottlenecks, prioritising fixes | + +The **individual metrics are most important** — they tell you exactly what's broken. + +## Project Structure + +``` +docs/ — Concept guides (how things work) +├── 01-architecture.md +├── 02-score-calculation.md +├── 03-metrics-reference.md +└── 04-testing-engines.md + +code-refs/ — Technical reference (what the code does) +├── file-structure.md +├── database-schema.md +├── api-endpoints.md +└── thresholds.md + +guides/ — How-to guides (what to do with results) +├── interpreting-scores.md +└── testing-workflow.md + +diagrams/ — Visual explanations +├── architecture-diagram.txt +├── data-flow.txt +└── scoring-algorithm.txt + +case-studies/ — Real examples +└── rds-77-score.md (why it scores 77, how to fix it) +``` + +## Quick Reference: Metric Thresholds + +All thresholds are hard-coded in `/home/help4bis/seo-intel/src/perf/sitespeed.py` lines 53–60: + +``` +LCP (Largest Contentful Paint) + Good: ≤ 2,500ms | Poor: ≥ 4,000ms + +FCP (First Contentful Paint) + Good: ≤ 1,800ms | Poor: ≥ 3,000ms + +CLS (Cumulative Layout Shift) + Good: ≤ 0.1 | Poor: ≥ 0.25 + +TBT (Total Blocking Time) + Good: ≤ 200ms | Poor: ≥ 600ms ← Most common problem + +TTFB (Time to First Byte) + Good: ≤ 800ms | Poor: ≥ 1,800ms +``` + +Green (✓) = score ≥ 90 | Amber (⚠️) = 50–89 | Red (❌) = < 50 + +## Common Questions + +**Q: Why does my site show 77 but Google PageSpeed says 95?** +A: Different scoring methods. Sitespeed's 77 is approximated; PSI's 95 is official Lighthouse. The individual metrics (like TBT=1,807ms) are what matter — that's the real problem. + +**Q: How do I re-test a URL?** +A: Click "Test Now" on the performance dashboard. It queues a background job (sitespeed + PSI on mobile + desktop). Results appear after ~90 seconds when you refresh. + +**Q: What's the difference between Mobile and Desktop scores?** +A: Mobile uses 4G throttling + Moto G4 emulation. Desktop uses native connectivity + 1366x768 viewport. Your mobile score is usually lower because of network + device constraints. + +**Q: Can I change the thresholds?** +A: Yes, but they're hard-coded in `src/perf/sitespeed.py`. Edit those thresholds, rebuild the Docker container, and the next test will use the new values. + +**Q: Why does it take 90 seconds per URL?** +A: Sitespeed runs 3 iterations (takes ~60s), PSI takes ~30s. Both run in parallel for mobile + desktop = 4 concurrent tests. + +--- + +**Last updated:** 2026-05-14 | **Repository version:** 1.0.0 diff --git a/case-studies/rds-77-score.md b/case-studies/rds-77-score.md new file mode 100644 index 0000000..f843037 --- /dev/null +++ b/case-studies/rds-77-score.md @@ -0,0 +1,243 @@ +# Case Study: rds.ink/drawings-of-endangered-animals-series/ Scores 77 + +**Date tested:** 2026-05-13 07:12:51 UTC +**Device:** Mobile +**Engine:** Sitespeed.io (HAR-based approximation) +**Score:** 77/100 ← AMBER (needs improvement) + +--- + +## The Numbers + +| Metric | Value | Good | Poor | Status | Contribution | +|--------|-------|------|------|--------|--------------| +| **TBT (Total Blocking Time)** | 1,807ms | 200ms | 600ms | 🔴 CRITICAL | 30/100 pts | +| FCP (First Contentful Paint) | 2,116ms | 1,800ms | 3,000ms | 🟡 SLOW | 82/100 pts | +| TTFB (Time to First Byte) | 144ms | 800ms | 1,800ms | ✓ GOOD | 100/100 pts | +| CLS (Cumulative Layout Shift) | 0.0 | 0.1 | 0.25 | ✓ EXCELLENT | 100/100 pts | +| LCP (Largest Contentful Paint) | Not measured | 2,500ms | 4,000ms | ❓ UNKNOWN | skipped | + +**Final score:** (30 + 82 + 100 + 100) / 4 = **78 ≈ 77** + +--- + +## Why 77, Not Higher? + +**One metric kills the score: TBT at 1,807ms.** + +TBT (Total Blocking Time) is how long JavaScript execution **blocks user interaction**. Your page: +1. Finishes initial render at FCP (2.1s) ✓ +2. BUT JavaScript continues executing for another **1.8 seconds** after that +3. During those 1.8s, user clicks don't work, scrolling is frozen +4. Page feels frozen even though it looks loaded + +This single metric contributes only 30/100 points, dragging the average from ~90 to 77. + +--- + +## Root Cause Analysis + +### Page Composition (from HAR) + +- **Total size:** 4.2 MB +- **Images:** 0.6 MB (14%) +- **JavaScript:** 1.8 MB (44%) ← THE PROBLEM +- **CSS:** Not measured +- **Requests:** 26 HTTP requests + +**The JavaScript is the bottleneck.** 1.8 MB of JS code, much of it loaded during page render: +- WooCommerce scripts (jQuery, jQuery plugins, WC-specific code) +- Elementor JavaScript runtime (webpack, core, widgets) +- Product gallery plugins (likely Lightbox, PhotoSwipe, or Slick) +- Lazy-load libraries +- Analytics/tracking scripts + +All of this executes **synchronously** on the main thread, blocking the browser from responding to user input. + +### Why FCP is Slow Too (2,116ms) + +The page doesn't show any content until 2.1 seconds because: +1. Server takes 144ms (acceptable) +2. Browser parses HTML +3. Hits render-blocking JavaScript in `` or early `` +4. Waits for scripts to download + execute +5. Finally renders first content + +If you defer non-critical JS, FCP would drop significantly. + +--- + +## How to Fix It: Hummingbird Settings + +You use **Hummingbird** for performance optimisation. Here are the exact steps: + +### Fix #1: Defer Non-Critical JavaScript (15 min, +12 score points) + +**Path:** WordPress Admin → Performance → Hummingbird + +1. Click **Performance** tab +2. Scroll to **JavaScript** section +3. ✓ **Enable "Defer JavaScript"** + - This adds `defer` to all `