Home // International Journal On Advances in Software, volume 11, numbers 3 and 4, 2018 // View article

Automated Continuous Data Quality Measurement with QuaIIe

Authors:
Lisa Ehrlinger
Bernhard Werth
Wolfram Wöß

Keywords: Data Quality; Measurement; Monitoring; Estimation; Trust

Abstract:
Data quality measurement is essential to gain knowledge about data used for decision-making and to evaluate the trustworthiness of those decisions. Example applications, which are based on automated decision-making, are self-driving cars, smart factories, and weather forecast. One-time data quality measurement is an important starting point for any data quality project to detect critical data that does not meet expectations and to define improvement goals for data cleansing activities. The complementary task of continuous data quality measurement is essential to ensure that data continues to conform to requirements and to detect unexpected changes in the data. However, most existing data quality tools allow quality measurement at a specific point in time while leaving the automation and scheduling to the user. In this paper, we highlight the need for (1) domain-independent ad hoc measurement, to provide a quick insight of an information system's qualitative condition, and (2) continuous data quality measurement, to observe how data quality evolves over time. Both requirements can be achieved with our data quality tool QuaIIe (Quality Assessment for Integrated Information Environments), which we developed to calculate metrics for the quality dimensions accuracy, correctness, completeness, pertinence, timeliness, minimality, readability, and normalization on both data-level and schema-level. The quality measurements can be either exported as a user- and machine-readable quality report, or they can be periodically stored in a database, which allows for long-term analysis. In this paper, we demonstrate the application of QuaIIe for ad hoc and continuous data quality measurement.

Pages: 400 to 417

Copyright: Copyright (c) to authors, 2018. Used with permission.

Publication date: December 30, 2018

Published in: journal

ISSN: 1942-2628