- It might take you a year to find a scalable channel if you don’t perform rapid-fire testing;
- You don’t need perfect tests, focus on speed;
- You will perform multiple tests that fail, but that’s ok!
- Your scalable channel will become unscalable at some point.
Caveat: Is all growth good? No. Metric driven growth measured by vanity metrics can be very dangerous. It is worth reading this article on the characteristics of good and bad growth by Nic Brisbourne as he gives some great pointers on what to avoid.
To deliver good, sustainable, growth you need to adopt a growth mindset. The growth mindset really focusses on one thing: expediting the execution of growth experiments. Growth experiments are covered in this article on channel testing.
Most companies eventually execute growth experiments. However exciting companies, the ones that grow quickly and sustainably, grasp the nettle. In turn, they break down the list of tests they want to perform and then place them into a Gantt chart with weekly (or more frequent) sprints assigned to testing each experiment. The number of experiments that you decide to test each week will reflect the amount of resource you have available (more on this below). Expediting growth tests allows you to learn faster, fail faster and iterate faster. Think about it like this: 20% of growth tests show some signs that they are working, 10% or less show signs that they are working in a scalable manner. That means you’re going to have to perform at least ten tests to get to a channel that might give you a scalable channel. Therefore, if you perform one test a month it might take you the best part of a year to find that channel, by which point you might be having many sleepless nights. Avoid this by performing 10 tests in 10 weeks. Sound impossible? It’s really not.
Embrace channel failure
Not all tests will work. Period. In fact I estimate that only one in ten will give you a scalable channel, and two in ten will give you a channel that shows some signs of growth. Rapid-fire testing is aimed to get you over these head-in-hands moments fast and with as little pain as possible. Once you get comfortable that you’re going to be failing a lot and failing fast it becomes much easier to move on and increase the pace of testing. If you’re not sure whether the test has worked, simply ask yourself could I get 10x more from this channel? Yes means it has, no means it hasn’t. “I don’t know” usually means it hasn’t either. The nicest example that I have seen of a company that embraces channel failure, which in turn cultivates a culture of rapid-fire testing was at Pact coffee, who are a subscription fresh coffee business. At Pact they have their CPA wall of shame. It’s a great way to publically be open about channel failure. It gives everyone visibility and a desire to improve.
Resource can be seen to be a problem. In a SCRUM project management scenario you will often see the Iron Triangle, where quality is prized above all else:
What this triangle says is that all three points on the triangle cannot co-exist. With SCRUM, you are obliged to fix quality as a constant. You then have to fix of the three points as another constant. Then the remaining two points are variables. For example, if you wanted to focus on quality and cost, you’d have to be flexible with scope and/or time. Using a growth mindset you should rejig this triangle to look more like this:
Here you have time as a constant. Rapid-fire testing requires a focus on time above all else. Scope then comes in as the second constant which means that in a typical project management scenario you’re going to have to sacrifice on cost (resource a.k.a. humans) or quality. However, what if you didn’t have to build anything? What if everything you wanted to do already existed via some pre-written code that you can insert into your page to hack this together? In this scenario the cost (resource) variable is reduced as no (or very minimal) development is required and the quality variable is also reduced as these pre-developed tools will have tested the quality piece for you.
What does this look like in practice? Rapid-fire testing like this usually entails using pre-existing plugins/apps/widgets/scripts that can be inserted into your <head> tag of your site within 30 seconds. Also, it involves linking services together with things like IFTTT or Zapier, or making minor changes to your front-end using a testing tool such as Optimizely. Ultimately your test should be designed to be a non-robust hack with the sole purpose of informing you whether the test has worked. If the test worked in this hacked sense, then there is only upside to how the conversion rate will look when you develop it properly.
Pick a metric, but not any metric
It’s important to pick the correct metrics to test. Don’t pick metrics that are at the bottom of the funnel (e.g. revenue, sales) as there are so many compounding variables that will affect these metrics. Ensure that your metric however can be linked to another metric that is important. I.e. don’t pick social shares if you can’t see a meaningful correlation between social shares and sales. Once you’ve chosen your metric you need to focus the experiment on moving that metric. You need to be able to make sure that if the metric does move that you’re sure that it’s because of the action you have taken and not some other influencing factor. Understanding the baseline for your metric can be important here if you know there are other factors at work.
Shun best practice for now
When performing rapid-fire testing you will be breaking “rules”. This is whether you’re hacking together some tech to make the test actually functional or whether you’re skirting around the edges of some guidelines for some ads you plan to run. Don’t worry about this. You can refine the tests later once they’ve proved their worth. Also, if everyone just implemented best practice there wouldn’t really be any innovation. Without innovation you’ll struggle to get results that over-index from your competition as best practice really is the market standard.
The Silver Bullet Fallacy
The whole point of rapid-fire testing is to increase your knowledge about your customers and marketing channels in the fastest way possible. This aligns with the Build-Measure-Learn cycle that dominates Lean Startup thinking and it’s simply an expansion of that away from product development and into growth marketing.
Ben Horowitz - “There are no silver bullets” VS Stephen King’s Silver Bullet
Silver bullets can kill Werewolves, but they can’t solve your growth challenges.
They simply don’t exist when it comes to finding a channel, technique or hack to grow your business. You’ll need to test many of these to find a collection of some that work together to deliver you ROI positive, scalable growth. In fact some channels you use might only give you unscalable growth, which is fine as long as you realise this early on. Always ask yourself “can I get 10x more from the channel”. At some stage you might think that you have found a silver bullet as you suddenly get startling growth. This isn’t a silver bullet. This is a scalable channel. The scary thing with scalable channels is that you don’t know when they’ll start to become unscalable (with a high marginal CPA and poor ROI). All channels have an efficiency threshold. Which means that even when you’ve found a scalable channel you need to continue your hunt for more, whilst at the same time optimising your existing channels hard.