Since you’re reading a blog on advanced analytics, I’m going to assume that you have been exposed to the magical and amazing awesomeness of experimentation and testing.
It truly is the bee’s knees.
You are likely aware that there are entire sub-cultures (and their attendant Substacks) dedicated to the most granular ideas around experimentation (usually of the landing page optimization variety). There are fat books to teach you how to experiment (or die!). People have become Gurus hawking it.
The magnificent cherry on this delicious cake: It is super easy to get started. There are really easy to use free tools, or tools that are extremely affordable and also good.
ALL IT TAKES IS FIVE MINUTES!!!
And yet, chances are you really don’t know anyone directly who uses experimentation as a part of their regular business practice.
Wah wah wah waaah.
How is this possible?
It turns out experimentation, even of the simple landing page variety, is insanely difficult for reasons that have nothing to do with the capacity of tools, or the brilliance of the individual or the team sitting behind the tool (you!).
It is everything else:
Company. Processes. Ideas. Creatives. Speed. Insights worth testing. Public relations. HiPPOs. Business complexity. Execution. And more.
Today, from my blood, sweat and tears shed working on the front lines, a set of two reflections:
1. What does a robust experimentation program contain?
2. Why do so many experimentation programs end in disappointing failure?
My hope is that these reflections will inspire a stronger assessment of your company, culture, and people, which will, in turn, trigger corrective steps resulting in a regular, robust, remarkable testing program.
First, a little step back to imagine the bigger picture.
This blog post was originally published as