Data quality | HESA

Data quality is about fitness for purpose.

Poor data quality leads to poor information quality with a number of serious implications:

• Reputational damge
• Financial loss
• Poor decision making
• Cost/complexity of producing known outputs.

Data quality is often considered as a purely data cleaning activity. But it is so much more than that – as much about setting the quality aspirations as it is for meeting them.

Data quality requirements are traditionally defined as accuracy, completeness, consistency, currency, precision, privacy, reasonableness, integrity, timeliness, uniqueness and validity. It should be seen then that these metrics are at the core of any data management function. Therefore data quality is an output of better data management. Many organisations begin by trying to improve data quality without understanding the wider aspects of the problem they are trying to solve.

These requirements must be driven by business processes and rules – not setting arbitrary targets that are meaningless in a wider context. Assessing what is required must be both bottom up (‘what problems do we think we have today’) and top down (‘what are our organisational aspirations for this data’). One provides operational issues, the other business context. Both are important when setting quality metrics.

These metrics must be relevant, measurable, accepted, owned and have a transparent ability to be tracked.

The most important of these is ‘relevant’. Managing data quality is a continuous process requiring determination, capability and time. This is not an undertaking that should be performed against a set of metrics that do not meet both the bottom-up and top-down requirements discussed above.

A test is to consider the data quality hidden in business rules. Many of these rules demand that data is presented at a certain quality at a certain time and in a certain format. You can build on this by introducing Data-SLA (service level agreements) as part of a business process to ensure the data quality requirements and metrics are fit for purpose.

Data quality management is likely to require technology support for profiling, troubleshooting, managing and measuring. It’s important to work with the technology function to ensure any tooling is also fit for purpose and can have wide use across the organisation.

If data is truly to be viewed – as it must – as a corporate asset, it has to managed as one. Data quality management is the vehicle to show that both the requirements and the measurements are in place to do so.

References

Captech (url): Three business examples of data quality issues