From January to March 2019 the MIT CDOIQ-iCDO ‘Data Governance Practices in the Public and Private sector’ project released a survey requesting that companies’ discuss their experiences and approaches as it relates to their ongoing data policies. We are interested in how organizations define data policy within their respective industry. Throughout this period, we had 95 responses, and we hope to roll out another wave of the survey in the coming weeks. As this study is in its preliminary stages, we invite our industry partners to participate if they have not already done so by going to https://www.surveymonkey.com/r/TGTYJ55.
In this article we are taking an exploratory look at the question; What metric categories are used by your Data Governance program?
To tease out the importance of data governance from our industry partners, we asked for the metrics used in their programs. Of the participates that responded, we see that in order, the top five metrics are Completeness, Data Integrity, Stewardship, Timeliness, and Consistency.
A Tale of Two Organizations
We see different results when splitting those who answered the questions into those that have a CDO office and those that do not. Organizations that have a CDO post tend to prefer Completeness, Stewardship, Data Integrity, Ownership, and Timeliness; whereas organizations that do not have a CDO within their organization show preference to Completeness, Data Integrity, Consistency, Timeliness, and Correctness.
What came first the CDO or Data Integrity?
Potentially a classic chicken and egg scenario, which begs our consideration. This dilemma illustrates one of the most substantial hurdles that CDO’s must overcome is the Data Quality issue while fostering Stewardship of the data while competing with the definition of Ownership.
Why does Stewardship come before Data Integrity (Quality) with companies that have a CDO? This is most likely because organizations that have a CDO office have already taken care of the necessary Data Quality measures, which means that they are now in an era of Data Stewardship whereas companies that do not have a CDO may not have taken care of the Data Integrity issues and are therefore more worried about the Consistency and Correctness of their data. This matter highlights the issue of high CDO turnover within some organizations. Does this mean that some organizations are not ready to have a CDO meaning the first CDO takes care of Data Quality issues thereby paving the way for the next CDO?
The similarities between those that have and those that do not have a CDO Office are as follows. They value; Timeliness, Completeness, and Data Integrity. The differences are Correctness, Stewardship, Ownership, and Consistency. Companies which have a CDO Office tend to be more interested in Stewardship and Ownership whereas companies with no CDO Office tend to insist on more Consistency and Correctness. For the sake of brevity, we will discuss only the top five metrics; Timeliness, Completeness, Consistency, Stewardship, and Data Integrity.
Timeliness reflects how up-to-date the data are concerning the task for which it is used. However, the definition of timeliness depends on the organization in order to create an adequate measure of timeliness. Timeliness metrics are dependent on the industry in which they are applied. Timeliness problems can often be interpreted as an accessibility problem. Accessibility means that analysts have access to the data. Not only giving individuals permissions but also the appropriate tools that make the data usable and analyzable.
Completeness is more than just ‘no missing data.’ This can mean a single piece of data within a single record, or complete records are missing. The reasons for completeness issues are widespread, but they often fall into three categories: schema, column, and population completeness. All of which should be addressed in order to have ‘complete’ data.
Consistency can also be a view from several perspectives. However, overall data should be in agreement. When there are conflicts, one source should be considered the master source or they are both unused until the source of disagreement is understood and fixed. Consistency makes tasks shorter (increasing timeliness) and simpler by being more consistent in how data is presented.
Stewards (Custodians) are commonly responsible for data content, context, and associated business rules as well as the safe custody, transport, storage of the data and implementation of business rules. These leaders should help promote data-driven culture, and actively support the role of data within an organization.
Data integrity is the maintenance and assurance of the accuracy and consistency of data over its entire life-cycle. It is critical to the design and implementation of any system which stores, processes, or retrieves data.
Data quality and Data governance cannot be reduced to the number of basic implementation. It should encompass all of the topics mentioned in this article. Businesses should be proactive with data quality and encouraging analysts to create repeatable processes. These are shared responsibilities; with right policies, organizations may truly become data-driven.