Kozler | Shopify RPV Growth Library

When Should You Start A/B Testing Your Ecommerce Store?

zac@kozler.com.au (Zachary Kohler) — Tue, 28 Jul 2026 04:50:51 GMT

There's a real cost to starting A/B testing too early, and it's not the one most people expect. It's not that testing is a bad idea, it's that testing without enough data behind it burns time and money on results you can't actually trust.

The threshold that actually matters: orders, not revenue

The number to watch is roughly 600 orders a month, which tends to line up with somewhere around $1M a year in revenue for most stores. Revenue is really just a proxy here, what actually determines whether testing works is order volume, because that's what determines whether a test can reach statistical significance in a reasonable timeframe.

What Is A/B Testing? And Why You Need It For Your Ecommerce Business

zac@kozler.com.au (Zachary Kohler) — Tue, 28 Jul 2026 04:26:02 GMT

A/B testing is simple in concept: you split your audience between two versions of your website, and you find out which one actually makes more money.

Half your traffic sees the version you already have. The other half sees a version with a specific change, That could be a new headline, a different navigation, whatever the hypothesis is.

Why revenue per visitor was never a one-person job, and why that shapes the CRO agency vs in-house team decision

zac@kozler.com.au (Zachary Kohler) — Tue, 14 Jul 2026 05:01:27 GMT

If your brand is doing millions of dollars a year on Shopify, you've probably had the CRO agency vs in-house team debate at least once, usually right after you've asked one ecommerce manager to "optimise the website" and realised that's not really a job for one person.

Why has my shopify conversion rate dropped from its usual baseline?

zac@kozler.com.au (Zachary Kohler) — Tue, 14 Jul 2026 04:39:02 GMT

Your ad spend hasn't changed. Your product hasn't changed. But your Shopify conversion rate has dropped, and now you're refreshing analytics every hour trying to figure out why.