Home // International Journal On Advances in Internet Technology, volume 7, numbers 1 and 2, 2014 // View article

Correlation and Consolidation of Distributed Logging Data in Enterprise Clouds

Authors:
Sven Reissmann
Dustin Frisch
Christian Pape
Sebastian Rieger

Keywords: Syslog Correlation; Log Analysis; Anomaly Detection; Monitoring; Enterprise Cloud

Abstract:
Due to the availability of virtualization technologies and related cloud infrastructures, the amount and also the complexity of logging data of systems and services grow steadily. Automated correlation and aggregation techniques are required to support a contemporary processing and interpretation of relevant logging data. In the past, this was achieved using highly centralized logging systems. Based on this fact, the paper introduces a prototype for an automated semantical correlation, aggregation and condensation of logging information. The prototype relies on a NoSQL storage back-end that is used to persist consolidated messages of distributed logging sources in a highly performant manner. This step of consolidation includes strategies for minimizing long-term storage, and by using correlation techniques also offers possibilities to detect anomalies in the stream of processed messages. In this context, we will present the special requirements of handling scalable logging systems in highly dynamic infrastructures like enterprise cloud environments, which provide dynamic systems, services and applications.

Pages: 39 to 51

Copyright: Copyright (c) to authors, 2014. Used with permission.

Publication date: June 30, 2014

Published in: journal

ISSN: 1942-2652