Home // International Journal On Advances in Networks and Services, volume 3, numbers 1 and 2, 2010 // View article

Quality and Performance Optimization of Sensor Data Stream Processing

Authors:
Anja Klein
Wolfgang Lehner

Keywords: Data Quality; Data Stream Processing; Query Optimization; Heuristic Optimization

Abstract:
Intelligent sensor devices together with data stream management systems allow the automatic recording and processing of huge data volumes to guide any kind of process control or business decision. However, a crucial problem is posed by data quality deficiencies due to imprecise sensors, environmental influences, transfer failures, etc. If not handled carefully, they misguide decisions and lead to inappropriate reactions. In this paper, we present the quality-driven optimization of stream processing that improves the resulting quality of data and service. After an introduction to data quality management in data streams, we define the targeted optimization problem comprising the optimization objectives and parameters that configure the required stream processing operators. Based on the generic optimization framework, we discuss and evaluate the optimization execution in batch and continuous mode. Further, we shed light on the crucial definition of the stream partition length used for the optimization that significantly influences the optimization performance. Finally, we provide a detailed validation of the proposed optimization strategies as well as the scalability of the overall approach not only at artificial data streams, but also using the real-world example of contact lens production monitoring.

Pages: 249 to 262

Copyright: Copyright (c) to authors, 2010. Used with permission.

Publication date: September 5, 2010

Published in: journal

ISSN: 1942-2644