Home // DEPEND 2015, The Eighth International Conference on Dependability // View article

On Handling Redundancy for Failure Log Analysis of Cluster Systems

Authors:
Nentawe Gurumdimma
Arshad Jhumka
Maria Liakata
Edward Chuah
James Browne

Keywords: Cluster Log Data; Unsupervised learning; Compres- sion; Levenshtein distance; filtering

Abstract:
System event logs contain information that capture the sequence of events occurring in the system. They are often the primary source of information from large-scale distributed systems, such as cluster systems, which enable system administrators to determine the causes and detect system failures. Due to the complex interactions between the system hardware and software components, the system event logs are typically huge in size, comprising streams of interleaved log messages. However, only a small fraction of those log messages are relevant for analysis. We thus develop a novel, generic log compression or filtering (i.e., redundancy removal) technique to address this problem. We apply the technique over three different log files obtained from two different production systems and validate the technique through the application of an unsupervised failure detection approach. Our results are positive: (i) our technique achieves good compression, (ii) log analysis yields better results for our filtering method than normal approach.

Pages: 7 to 14

Copyright: Copyright (c) IARIA, 2015

Publication date: August 23, 2015

Published in: conference

ISSN: 2308-4324

ISBN: 978-1-61208-429-9

Location: Venice, Italy

Dates: from August 23, 2015 to August 28, 2015