“Today, I realised that the word ‘bed’ actually looks like a bed. Mind blown.”


  • Categories

Search Resources

The awesome Phrasee split test calculator

Hey! If you’ve made it this far, then you probably know that “statistical significance” isn’t just the strongest Tinder opener ever – it’s also a REALLY important thing to achieve when split testing.

This simple calculator will use actual statistics to determine how many splits you should use, and how big each sample size should be.

You’ll also get triggered an email with a bunch more information about subject line testing. So that’s cool.

There’s more info further down the page if you’re interested. But if you just wanna get your numbers – go for it. Awesome!

Become a subject line superhero with our Phrasee split test calculator, and because you’re awesome we’ll sign you up to receive other great content.

Here’s a bunch more info:

OK, here’s a bit of information on proper experimental design. It’s sort of academic because what you ultimately care about is results. But hey, in case you’re interested, read on…

Email responses follow what’s called a “binomial distribution”. That’s because each result is binary – it’s either: opened or not opened, clicked or not clicked, purchased or not purchased.

In the long run, a binomial distribution converges upon a “Gaussian distribution” (aka “normal” aka “bell curve”). This is due to the central tenet of frequentist statistics – the central limit theorem – which means that independent variables (i.e. person 1, person 2, … person 1000 on your list) will, with enough data, converge upon a mean.

From a split testing standpoint, this matters… and here’s why: you need to be able to estimate the minimum amount of data you need to get to mean convergence as reliably as possible.

You also need to consider things like statistical power – how likely it is that you’re seeing a random run of false positives – and your estimated effect size.

To make matters more difficult, not all ESPs make split testing very easy (here’s a guide to what ESPs can split test, btw).

In a perfect, clinically controlled world, you’d be able to run infinite split tests. But that’s not feasible.

Lastly, there’s the problem of sampling bias. You can’t be 100% certain that your ESP will always randomly select your sample groups. And even if they do, there’s a high probability that there will be inherent skews in your data, thus reducing the reliability of your results.

That’s why this email split testing calculator is awesome.

We’ve helped our clients run literally THOUSANDS of split tests over the years. We know what works in email marketing, probably better than anyone else.

We’ve done the stats. We’ve designed countless scientific experiments. And we know what we’re talking about. Oh yeah, our Chief Scientist Dr Neil Yager literally wrote the book on data mining. So, yeah, you can probably believe what the handsome devil says.

And now you can benefit from part of our knowledge.

BTW – setting up your test is only half the battle – you also need to figure out what you want to actually achieve. If you want more opens, more clicks, and more conversions… well, that’s what we do. Just ask our customers.

Get in touch if you want to take your email results to the next level!

The current AI landscape

The current AI landscape

What is Artificial Intelligence? Great question – and one we get asked a lot. The definition of artificial intelligence (AI) differs depending on who you ask. Why? Because there are many different types of AI.

Welcome to Phrasee Pheelings™ Lite

Welcome to Phrasee Pheelings™ Lite

Phrasee Pheelings™ is the world’s first Artificial Intelligence technology that predicts your audience’s emotional response to your marketing language. And now a Lite version of the tech is available to you – for ‘phree’!

Discover how applying actual AI to language can deliver big results!