- Learn how to scientifically perform channel tests;
- Judge your results before you've spent any money;
- Know when you're pushing a channel too hard.
Why do you need to test channels?
Testing marketing channels is crucial to determining where the future growth of your business is going to come from. Future investors will want to see evidence of channel tests as well as data that suggests that some channels are showing signs that they will be scalable.
What does a channel test look like?
A channel test is much like a small scientific experiment. As such, we need a hypothesis, a test method, some results and a conclusion. This should all be worked out before the test begins.
- Hypothesis - What is our assumption?
- Test Method - What steps do we need to take to test this assumption?
- Results - What data do we need to prove or disprove this assumption?
- Conclusion - Has the data proved or disproved our assumption?
When using paid marketing channels you’ll be spending money on media, which you hope will return you something back. In your hypothesis step you need to ask yourself not only what is the assumption that we are testing, but also what action you want the marketing to deliver? The assumption might be as vague as that we can use Facebook Ads effectively to drive signups. The aim of the marketing is to then deliver signups on the website, so the thing we need to measure is signups. Bear in mind that your action can be anything as long as you can measure it.
Once you’ve decided what you’re measuring you need to figure out a way to set up a test whereby you can measure this action. For example, you might need to set up some paid campaigns on Facebook, which will give you cost data and you might need to set up some tracking on your website to attribute clicks from Facebook to actions on your site. Before you launch your campaign, you should test your flow works and is attributing correctly.
Perhaps counterintuitively, you can and should work out your results section before you begin your test. To do this you should work out how much the action you have selected and are measuring against is worth to you. This is easy when looking at sales, but less so with actions that don’t have an inherent value associated to them such as phone calls, sign-ups, shares and so on. If you are using an action that doesn’t have an inherent value, you will need to work out the funnel from that action to some sort of revenue conversion event later on in the flow in order to assign it a value. If you don’t have data on your conversion flow you should take a look at this article where it shows you how to go about getting free traffic in order to estimate your step conversion rates. To get your desired results you will need to work backwards from the action step that you identified all the way up to the media you’re thinking of purchasing to determine the whole conversion flow. You need to include all the individual steps where conversion rates appear. Note down which ones you are in control of.
An example of how this might work for phone calls leading to holiday bookings coming from SEM might look like this:
Holiday Booking Action Value = £500
Holiday Booking Actions = 1 (fix this value)
Conversion to booking = 5%*
Phone Call (Action) = (1/0.05) = 20
Conversion to phone call = 10%* (known from free traffic conversion tests)
Site Visits (Step) = (20/0.10) = 200
Site Visits = Clicks from AdWords
AdWords Clicks (Step) = 200
At this stage you now know that you need 200 clicks to generate £500 of revenue. The next step is to estimate what your initial cost per click (CPC) via AdWords needs to be to break even.
Clicks = 200
Breakeven Cost = £500
Therefore CPC (500/200) needs to be £2.50 for breakeven.
*Lines marked with asterisks are conversion rates that you can influence.
It’s worth noting that to increase overall conversion you’re better off spending time and effort higher up the funnel, i.e. at stages where more people are affected. In the example above that means that improving the conversion to phone call conversion rate will have an overall larger effect than improving the conversion to booking number. You can get some data on some initial CPCs from the Google Adwords Keyword Planning Tool without running a test. This is a good way to sense check your results before you run the test checking whether your target CPA is within touching distance of the initial CPCs that the Keyword Planning Tool gives out.
Once your test has finished you need to compile your results and assess whether you have proved or disproved your assumption. Using the results template above you should be able to work out whether you have hit your target CPA. If you have, fantastic, this is an opportunity to spend more time developing this marketing channel further, whilst also bearing in mind that there is probably some optimisation that could be done to further lower the CPA. If your test has failed to meet your target CPA you will need to really assess all the steps in the funnel to find out which steps have caused this CPA to be missed. It may be a step that you can influence. If so, redesign the test with this step set up differently. If the results are way off your target CPA then you may need to admit that this test didn’t work. The key here is to accept failure quickly so that you don’t pour more money into the test unnecessarily.
Every marketing channel you work with has a limited amount that it can scale. Successful growth teams utilise multiple channels concurrently in what I see as being a big game of marketing Snakes & Ladders.
Your successful marketing channels are your ladders and you unsuccessful tests are your snakes. You’ll notice that the ladders are differing in length as some channels scale much higher than others (cf. AdWords to handing out flyers outside a train station). This is the same with some of the failed tests, some will take much more time, money and effort to set up and analyse than others. If you continue to use ladders and reduce your use of snakes your business will grow. Whilst Snakes & Ladders is a game of luck, it’s based on a dice roll, the marketing version isn’t. Pick channels astutely, expedite channel success and expedite even more channel failure.
Scalability & Efficiency
Most marketing channels behave like this chart:
What this chart shows is that for a time you can continue to spend on a channel and the number of conversions you receive will largely follow in a linear fashion. This then moves into a proportional relationship and then an exponential relationship. With digital channels and also television I have found this 3 stage relationship to be very observable.
If you differentiate this curve you get the marginal CPA curve, it looks like this:
As you’ll see from this chart the CPA you’re paying for all conversions on the right hand side of the chart are huge multiples of the early conversions’ CPA. Judging scale with marketing channels is really a function of understanding these charts, recreating them with your own data and then making sure you’re definitely not in the far right exponential sections. If you are, reduce spend immediately as you’re working the channel too hard. Judging where the inflection point of these curves lie is key to not overspending. Successful growth teams diversify their channel mix early on to always be way below the marginal CPA inflection point, it’s important to always be on the lookout for new channels and to continue to test them.