How We Test What Works: A/B Testing SEO Across Platforms and Budgets

When you manage one website, you have opinions about what works. When you manage many websites across different industries, platforms, and budgets, you have data.

We run websites for clients in cannabis, healthcare, legal, entertainment, e-commerce, education, and non-profit. Each site has different traffic volumes, different audience behaviors, different CMS platforms, and different budget constraints. What works on a high-traffic e-commerce site doesn't always work on a low-traffic professional services site. What converts in cannabis doesn't convert the same way in legal tech.

That diversity is a testing lab. Every change we implement across the portfolio generates data that informs what we do next — not based on what a blog post said would work, but based on what we measured working.

What SEO A/B testing actually looks like

SEO A/B testing isn't the same as CRO A/B testing. You're not showing different versions of a page to different visitors simultaneously — Google sees one version of your page, and showing it different versions is cloaking, which gets penalized.

Instead, SEO split testing works at the page-group level. You identify a set of similar pages (product pages, blog posts, service pages), divide them into test and control groups, change one variable on the test group, and measure the organic traffic difference over 2-4 weeks.

Variables we commonly test:

Title tag structure. Does putting the brand name at the beginning or end of the title tag affect click-through rate? The answer depends on brand recognition — known brands benefit from front-loading, unknown brands benefit from leading with the keyword. We've measured this across client portfolios and the split is consistent.

Meta description length and format. Does a question-format meta description outperform a statement? Does including a number ("5 steps to...") increase CTR? On informational content, questions outperform statements by 8-15% in our testing. On transactional content, the difference is negligible.

Heading structure. Does rewriting H2s as questions improve organic traffic to a section? This one consistently wins — question-format headings align with how people search and how AI systems parse content. The lift is typically 5-12% in organic impressions for the affected sections.

Internal linking density. Does adding 2-3 contextual internal links per post improve rankings for linked pages? Yes, consistently, across every client. The effect is measurable within 2-3 weeks and compounds as the link graph grows. This is the single most underused SEO tactic we encounter.

What we've learned from testing across budgets

Not every business has the same SEO budget, and the strategy needs to match the resources.

Minimal budget ($0-200/month). The highest-ROI activities at this tier are: fixing title tags and meta descriptions on existing pages, adding FAQ schema to high-traffic pages, ensuring mobile responsiveness, and publishing one quality piece of content per month. We've seen sites at this budget tier double their organic traffic within six months by doing nothing but fixing title tags and adding structured data.

Mid-range budget ($500-2,000/month). Add: server-side analytics (accurate data to base decisions on), monthly content publication on a consistent schedule, behavioral analytics to identify conversion friction, and quarterly technical SEO audits. This tier is where compounding starts to show — the combination of content, technical health, and data-driven optimization produces measurable results by month four.

Full investment ($3,000+/month). Add: dedicated content strategy with pillar/cluster architecture, entity SEO and GEO optimization, A/B testing on page elements, comprehensive schema markup, and active link-building through guest content and industry participation. This tier builds the kind of compounding organic engine that eventually becomes the largest traffic and lead source.

The mistake companies make is implementing enterprise-tier tactics on a minimal budget, or implementing minimal-tier tactics on an enterprise budget. The strategy needs to match the resources, and the resources need to match the business goals.

Platform-specific findings

Different CMS platforms and hosting environments produce different SEO baselines, and we've measured the gaps:

WordPress is the most common platform we encounter. Its SEO ceiling is high but its floor is low — a misconfigured WordPress site with the wrong theme, too many plugins, and no caching can load in 6+ seconds and generate hundreds of crawl errors. A well-configured WordPress site with a lightweight theme, server-side caching, and proper plugin selection performs competitively with any platform.

SvelteKit and Next.js produce the fastest pages by default because they render server-side with minimal JavaScript hydration. Our fastest-loading client sites are all SvelteKit. The SEO advantage from page speed alone is measurable — typically 10-20% better Core Web Vitals scores than equivalent WordPress implementations.

Shopify and Squarespace are optimized for conversion, not for SEO flexibility. They handle the basics well (mobile responsiveness, SSL, basic schema) but limit structural SEO options — URL structure, heading hierarchy, and technical markup are constrained by the platform. Clients on these platforms benefit most from content strategy and off-page SEO, since on-page technical optimization is limited.

Ghost is excellent for content-first sites. Clean HTML output, fast rendering, built-in structured data for articles, and simple content management. The limitation is extensibility — custom functionality requires separate infrastructure, which means Ghost works best as a content layer alongside a separate application.

The platform isn't the strategy. But the platform affects what strategies are available, and we adjust recommendations based on what the platform makes easy, hard, and impossible.

The testing mindset

The most valuable output of systematic testing isn't any individual finding. It's the habit of making decisions based on measured results instead of assumptions.

Every client engagement starts with the same question: "What do we think will work, and how will we know if it did?" The answer defines the test. The test produces data. The data informs the next decision. The cycle continues.

Companies that build this cycle into their operations — test, measure, adjust, repeat — consistently outperform companies that implement a strategy once and hope. SEO isn't a destination. It's a feedback loop.

---

Related reading

• Search Console Goes Deeper Than Rankings
• What a Technical SEO Audit Actually Checks (And What You Can Fix Today)
• E-E-A-T Is Not a Checklist
• The Problem with Generic Tech Recommendations
• If the Recommendation Came Before the Questions, It's Not Advisory

Frequently asked questions about a/b testing seo across platforms and budgets

What is SEO A/B testing?

SEO A/B testing is a methodology for measuring the impact of page-level changes on organic search performance. Unlike CRO A/B testing (which shows different page versions to different visitors), SEO A/B testing divides similar pages into test and control groups, changes one variable on the test group, and measures the organic traffic difference over 2-4 weeks.

Can A/B testing hurt my SEO?

No, when done properly. Google supports testing as long as you avoid cloaking (showing different content to Googlebot than to users). Use temporary 302 redirects for variant pages if needed, add rel="canonical" tags, and keep test durations reasonable (2-4 weeks). Google's documentation explicitly states that A/B testing does not violate their guidelines.

What's the most impactful SEO test for small businesses?

Title tag optimization. Rewriting title tags on existing pages to better match search intent — and measuring the click-through rate change in Google Search Console — is the highest-ROI test a small business can run. It costs nothing, takes an afternoon, and frequently produces double-digit improvements in organic click-through rate.

How much traffic do I need to run meaningful SEO tests?

More than most small sites have on individual pages, which is why page-group testing is the standard approach. If you have twenty similar blog posts each getting 100 visits/month, you can test across the group (ten test, ten control) with a combined sample of 2,000 monthly sessions — enough to detect a 10-15% change with statistical significance within 3-4 weeks.