- You need to conduct both qualitative and quantitative testing together to get the full picture.
- Test your assumptions, regardless of whether you are an expert.
- Make sure data rules over internal opinion, and prioritise your customers’ feedback.
Often data is represented as numbers because it’s easy to communicate and analyse. A deeper dive into why people do things, not just what people do is harder to interpret. In order to progress and learn from our mistakes and successes, you need a holistic view. That means using both quantitative and qualitative data to inform your decisions.
It’s essential that you’re tracking the pages, links, buttons and elements of your site or app to know what’s going on. Track everything! You’ll understand people’s motives and engagement much more clearly if you can see their path, how long they linger for, what they click on next, and when they drop out or convert. You’ll make better decisions on how to scale if you’re more informed about how the product is performing. Make sure you’re collecting plenty of data (https://thepathforward.io/data-discipline/), even if you might not think you need it right now. There are many tools for capturing this kind of data, which you can read about in this guide: (https://thepathforward.io/analytics-data-the-startup-marketers-guide/).
Discovering why events occur can help you find patterns and plan ahead. If you see consistent or periodic spikes or dips in site traffic and conversion rate then it’s helpful to know the catalyst. That way, you can re-introduce things that cause the peaks in conversion and help avoid the troughs. Sometimes the causes are obvious and controllable, for example your site is offering a sale, or you’ve increased marketing spend. Other times, it’s caused by other stimuli, or is related to the nature of your product, causes you can only discover by talking with customers. It could be a subscription-based product, or something that’s only helpful or relevant to people for a period of time or season.
If you understand your audience’s needs and their drivers, then you may also be more informed about their modes of behaviour. The most effective way to discover why people do what they do is, of course, by asking them.
Many, if not all, of your current targets may be metrics-based as growth, stability and scalability are paramount to your business’ success. However, don’t forget that a lot of interaction with your product and brand will be emotional for people. You need to make sure you measure success partly on how people feel. Conduct user interviews to gather qualitative data in order to capture this. It will give you insights you won't get from metrics alone. As fluid as these emotions may seem, you can still identify strong trends.
Let’s imagine you’ve created an minimum viable product (MVP) and your business is up and running. The next step is to grow your following, your repeat customer-base and your profit. Even if you have plenty of data, it’s sometimes hard to know what to do next with your product. Iterative multi-variant testing can help you when there isn’t one clear route forward. You can pitch a few ideas against each other to find the one which best suits your audience.
You need to be able to correlate your product decisions directly with analytics. It’s essential here to change things that are big enough to make a difference, but not so big that you can’t attribute the metrics to a specific element or variation. It’s hard not to worry about risks here, especially if your ‘control’ (the constant that you want to test against) is performing well. You can feel calmer knowing that you can manage the percentage of traffic to each variant. Make sure, though, that it’s a large enough pool to reach significance quickly. Otherwise you’ll be running the same test for a long time and won’t reap the benefits of results soon enough. Quick tests with a fair positive trend allow you to find new controls and test other theories sooner.
The time to reach significance in a test pool varies between companies, products and tests. If you want more thorough help to calculate sample sizes and determine how long to run a test, use this handy guide by Optimizely.
Be flexible and open to iterative exploration. Don’t try and solve things all at once. Instead, learn more about micro user behaviours by working out issues at key journey points. This will inevitably build up a more detailed macro-view of how your business is progressing.
Customer data vs expert opinion
Avoid falling into a pattern of pushing product decisions based on internal opinion. Although your team will be full of bright, passionate people, they can’t categorically say what the user will think or do. Even if they fall into the remit of your target audience, they can’t speak for the audience as an entirety. Remember that a requirement for your team of UX practitioners, developers and even the CEO is to neutrally serve the needs of the customers without creating a bias toward their own thoughts. It is sometimes easier to do that when you’re not part of the audience. This may increase the need for you to conduct more thorough user interviews to gain adequate insight.
Until you have tested your hypothesis, reached a significant audience scale and gathered the results, it’s all just conjecture. Guesswork and estimations can slow down progression, or even send your business down the wrong path, because it’s not assessing and meeting explicit customer need. Unless you create value for people, you won’t see a great return. Look at the numbers and listen to the reasons customers give for what they like, click, buy, and what they don’t. Their word is gospel!
There really is nothing more valuable than testing your assumptions. It’s also a great way to settle differing opinions within your team. Within a group of people with many varied experiences, it’s natural that everyone brings anecdotes of products and tests to hypothesise from. At Forward Partners, we use the phrase ‘Let’s test it!’ every day because we realise that the data we get from it is worth a great deal more than our own assumptions. Use our guides on testing your product with real people, and base your decisions on what you discover.