Data is the lifeblood of a conversion rate optimization team. Without it, your analytical capabilities (and thus your testing) would be limited.
Therefore like all potential resources, you have to handle your data carefully. Keep it fresh and prepared for evaluation. It must be complete and accessible, structured and uncorrupted.
Managing and preparing data is so important that it makes up a significant portion of a data analyst’s job. According to Blue Hill Research, the average data analyst spends about two hours each day (or about $22,000/year) just preparing data.
Healthy data creates valuable analytics. Robust and accurate analytics help you discover testing insights. Insights, of course, lead to experiments.
Naturally, if data is so important to the entire CRO process, we should take steps to protect and manage it properly.
1. Refresh Your Data Regularly
Depending on the manner you collect data and the tools you use, you may “snapshot” your data for evaluation and use it for a period of time before updating to a new sample. While this method technically works, the information is rarely current.
Best practice is to work in an agile environment with a centralized data repository. Here, real-time data is accessible by the entire team. This ensures the information is always current. If your analytics tool permits filtering, you won’t have to take any steps to snapshot and manually alter data sets.
2. Add a Metadata Layer
Use metadata to provide information to other users and testers about your data. This is especially important if you archive old data that may need to be retrieved someday.
There are basically three types of metadata.
- Structural metadata describes the characteristics of data, like file types, sizes, versions, relationships to other data groups, and the internal structure of the data.
- Administrative metadata is information about managing or using the data, like the purpose of the data, reproduction rights, access limitations, where it can be transferred, etc.
- Descriptive metadata describes the data itself, such as the author, originating location, keywords, abstract, etc.
Create a metadata standard (or schema) that will apply to all data sets. The CRO industry doesn’t have its own standard, but many industries do (like healthcare, government or finance). There are also some general purpose standards that might be useful, like Darwin Core or Data Documentation Initiative.
3. Scrub Out Personal Information
CRO is typically an agile environment, so you probably use live data from existing databases and actual users.
For ethical and liability purposes, it’s best to eliminate personally identifying information from your data whenever possible. In most cases, your conversion optimization team doesn’t need to know the names, phone numbers, birth dates, or other sensitive information of your customers. Conclusions can be drawn from those tables with simple scripts; your people don’t need direct access.
This isn’t to say that your CRO team is untrustworthy. But in the event of a security breach, you want to be able to point to every safety measure you can.
Additionally, scrub out any data that is sensitive to your company or would somehow affect your ability to be competitive. Online Ethics Center has an interesting article on responsibly collecting, using and interpreting data.
4. Define a Data Dictionary
A data dictionary (sometimes called a data definition matrix) is an internal document that defines certain terms and attributes about your data. Its purpose is to clarify terminology amongst your team within the context of your projects so there’s no confusion.
This is an important way to alleviate inconsistencies across your projects and keep the team communicating effectively. It also makes data easier to analyze. “It helps avoid project mishaps such as requiring information in a field that a business stakeholder can’t reasonably be expected to provide, or expecting the wrong type of information in a field,” say Business Analysts Bridging the Gap.
For example, let’s say you’re working on an ecommerce store, attempting to increase general conversions (product purchases). The Black Friday to Cyber Monday weekend is an unusual time of the year for retail, so you want to exclude sales that occur during this period from your yearly metrics.
In this instance, your dictionary would set a data standard for conventions like “yearly page views.” The definition would instruct the team to exclude the Black Friday to Cyber Monday period.
5. More Data Is Better
In most cases, having more data is better. You never know what you’ll decide to test. If you fail to collect a piece of information or filter it out of general collecting, you’ll be disappointed when you eventually need it.
Even if something seems irrelevant, it’s best to have your analytics tool capture it anyway (providing storage space and cost aren’t issues). As your website or application evolves, the variables that affect your testing may change too. When something becomes relevant to your CRO testing, you’ll be delighted when you can go back and examine it from years prior.
Furthermore, generous data sets smooth out irregularities. Using our ecommerce store example, a popular promotion could make a year’s sales figures unusual. Over time, however, as general sales and other promotions are accounted, revenue becomes predictable.
6. Complete Your Experiments
You’ll pull insights from your analytics to create tests, but that’s not the only data you should protect. The data you collect from your testing is important as well.
The best thing you can do for this type of data is to run your CRO experiments to the end. If you decide to test a particular variable for, say, seven days, you should allow the test to run its course, even if the results aren’t favorable after the second day. It’s difficult to draw conclusions on short-term data because it could easily be influenced by an unknown variable.
If you asked smart questions and made reasonable assumptions, no test should cause disastrous results.
7. Put Data in Context
The data that comes out of your experiments is specific to your website or application. It can’t be applied elsewhere. Disregard anyone who tells you they know what a specific outcome means in the context of “all websites” or “all software.”
For instance, let’s say your ecommerce store (back to our example) has a landing page for a giveaway. You have removed all header, footer, and unnecessary links from the page to give visitors one option: enter their email address.
As you pore through your analytics, you notice the page has a bounce rate of 70%. On its surface, that’s an ugly number. Someone who runs a content-heavy blog would be unhappy with that number.
But in the context of your landing page that offers only one option, it’s not so bad. If the visitor refuses to enter the promotion, all they can do is leave. It’s a get-in-or-get-out funnel. In the context of this example, a big bounce rate doesn’t indicate a problem (although you may want to take steps to work it down).
By following these practices, you’ll preserve the most important asset you have: your data. When you’re ready to optimize your CRO process, request a demo of our software, Effective Experiments, so we can show you how to really put that data to use.