- There are three important steps to winning with data: Capture, Validation & Interpretation;
- You will probably need a developer to help you set up any analytics tool you use;
- Use compound metrics rather than absolute values to really gain insight.
Data is an important component of Lean Startup thinking. It not only powers the measure section of the build-measure-learn cycle, but also the insights section of the hypotheses-experiments-tests-insights cycle known as the Lean Minimum Viable Product cycle. It is impossible to complete these cycles without mastering the following three data stages: data capture, data validation and data interpretation.
If you’re running a website or even a mobile app then Google Analytics (GA) is most likely going to be your first port of call for data capture. You can find more details on GA and how to get a snippet of GA code for your website here. GA sends a lot of basic “out of the box” data about the sources of your users, their behaviour on your site and attributes of your users like their operating system, location and so on. This is unlikely to be enough for any online business, so be prepared to go through a process of tagging your website with extra code to pick out important event based features of your site. GA has new(ish) advanced ecommerce functionality, which allows you to collect great merchandising and transactional data, but this will normally require some custom configuration and probably a developer. Another option would be to investigate using a tag management solution. If using a tag management solution (e.g. Google Tag Manager or Adobe’s Dynamic Tag Management) be aware that complex sites will likely require you to still add some code to ensure the tag fires at the correct time with the desired data. Tag management solutions attempt to take website tagging, which is typically a marketing requirement out of the hands of developers and into the hands of marketers who (in theory) can’t code. However in my experience this is never a 100% code-free switch unless your site has few and basic features that need measurement.
If you plan to use multiple tools for capturing and interpreting data from your site then you could also use a tool called Segment to parse this data to the relevant tool at the right time. Doing this will save you setting up each tool independently and may well save you expensive development time in the long run.
If you are collecting offline conversion data, which may be things like inbound telephone calls, inbound emails and so on, you may want to think about what the best way to track this data is and then how to attribute it correctly. Tools like IFTTT and Zapier allow you to automate data transfer between various applications. I often use these tools to pull in offline data into a Google Sheets spreadsheet that can then, in turn, pull in data from Google Analytics and then join the data within the spreadsheet. Methods like this avoid the need to script anything, which is a perk for the code-wary marketers out there.
Once you have captured data successfully you need to ensure it’s correct. This is a crucial and often overlooked stage, which can lead to frustrating and embarrassing meetings if data is indeed found to be incorrect. You have a duty to perform this step and ensure data you report on is valid, especially when in discussion with investors. Errors in data can occur at the time of both data capture and data interpretation. Investors require absolute clarity when discussing data so this is an important step if you’re fundraising.
Whenever you release a new version of your application you should be wary that new scripts could be interfering with existing tagging. You should ensure tagging is tested pre-release. I’ve experienced complete data blackouts post release, that in turn have forced rollbacks to earlier versions. This is a frustrating and expensive procedure to have to go through, particularly when it’s almost always avoidable. Try incorporating an analytics testing process as part of your deployment schedule to mitigate against this happening.
If you are only using one data capture tool, consider adding another to your site. Validating data against itself is a flawed process as incorrect data will tend to agree with itself. The addition of another tool allows you to compare metrics. An important consideration however is that your two tools might be processing the data in different ways. One common example of this is the measurement of sessions or visits. Usually a visit is defined as ending after 30 minutes of inactivity from the visitor’s cookie, however some tools allow you to alter this inactivity cut-off time, potentially then causing discrepancy in your numbers.
Interpreting valid data is an important way to get under the skin of your business and allows you to really understand how your business is performing. Finding the right metrics to focus on is, however, sometimes challenging and usually an iterative process that changes as your business evolves. The term “Vanity Metrics” has now become a bit of a Lean buzzword, but it is really really important to identify these and to not focus on them. It’s also not great to bring these up in any investor meetings you have as they could and should be picked apart. Without a focus on actionable metrics you won’t be able to cycle through the build-measure-learn cycle, which is crucial part of The Path Forward as it will deliver you rapid learning.
Metrics: Actionable vs Vanity?
Tools like GA give a whole host of vanity metrics out of the box. Top level metrics like pageviews don’t really tell you anything meaningful, they do however normally present the largest number. Vanity metrics tend to be the largest numbers you can find.
One technique to turn a vanity metric into an actionable metric usually involves dividing another metric by it.
Consider this example:
- Monthly unique users: 1,000,000 (a nice, big, chunky vanity metric)
- Monthly signups: 1,000 (independently not a teeny number)
- Now take monthly signups and divide by your Vanity Metric to create your sign up conversion rate: ( 1,000 / 1,000,000 ) = 0.1% (oh crap)
This signup conversion rate is a much more actionable metric. It’s a compound metric that takes into account more than one raw metric and gives the end user a better understanding of what’s going on, in this case it’s showing that their user to signup conversion rate actually seems pretty low. This then raises questions like: Is the signup process effective or is it even working? Are the users we’re attracting the right audience for our service? Questions are good.
Once you’ve got a compound metric like this, it’s worth trending the data. Single frame of reference metrics rarely tell the whole story, so trend it and get some perspective on what might have changed over time. No doubt this will raise more questions, but questions are pivotal when challenging assumptions. The notion of challenging assumptions is a key doctrine within The Path Forward.
- Avinash's blog: http://www.kaushik.net/avinash/
- The GA help centre: https://support.google.com/analytics/?hl=en#topic=3544906
- The GA analytics academy: https://analyticsacademy.withgoogle.com/explorer