In this data tip, we will take a quick look at some of the most fundamental rules of working with data. These are rules your IT department probably knows cold, but that the rest of us don’t fully understand. In today’s world, we are going to get better results from our data if we all become a little more literate about some of the basics.
1. Source matters. In general, those who are closest to the source of the data are in the best position to bring it into the organization in a way that is accurate and relevant. As data moves through an organization, it tends to be copied and added to in ways that can affect its quality. Ensure that you always know the provenance of the data and empower those departments and staff that collect it to have the first and last word on accuracy and manage the process for commenting on it and changing it.
2. Strive for consistency. Establish a baseline for data consistency. You want your data set to be internally consistent, but you don’t want to throw out the potential insight that new data can bring. If you are seeing a lot of unexpected values, that could be an important business signal. To achieve this balance, you need to establish a consistent baseline.
3. Revisit relevance. Your data strategy should be driven by business requirements and aligned with business processes. But these are a moving target. What seems relevant when you first build your strategy will change. Make it part of your data strategy to revisit the variables you are collecting and tracking on a quarterly basis. Include in the review process the data owners as well as the business strategists. It is important that the people who touch the data and the people who analyze it are on the same page.
4. Question completeness. Do you have data on all of the areas of your business that you need to measure and understand? Is there anything substantially missing from your data that weakens your ability to use and apply it as widely as you’d like? Always ask yourself if you have the right data. As Albert Einstein said, “Not everything that can be counted counts, and not everything that counts can be counted.”
5. Timeliness is all: Is there a delay between when you get your data in a usable, machine-readable form and when you need to act on it? Will you collect data in real-time or once a month? What is your data’s shelf life? The timeliness of data must be explicitly documented and be acceptable to the business. This includes the expected frequency rate at which data elements need to be refreshed.
With these basic ideas in place, you can begin to work on a more mature data strategy. According to a survey by Business Intelligence Research, most organizations report that getting the business to take data quality and integrity seriously is a real challenge. How would your organization rank in term of the data quality activities listed in the chart above? If you can learn to manage these challenges, you will be way ahead of the curve.