Home // CLOUD COMPUTING 2010, The First International Conference on Cloud Computing, GRIDs, and Virtualization // View article

A Generalized MapReduce Approach for Efficient mining of Large data Sets in the GRID

Authors:
Matthias Roehm
Matthias Grabert
Franz Schweiggert

Keywords: Data mining, Grid, MapReduce

Abstract:
The growing computerization in modern academic and industrial sectors is generating huge volumes of electronic data. Data mining is considered the technology to extract knowledge from these data. With an ever increasing amount of data and complexity of modern data mining applications, the demand for resources is rising tremendously. Grid and Cloud technologies promise to meet the requirements of heterogeneous, large-scale and distributed data mining applications. The DataMiningGrid system was developed to address some of these issues and provide high performance and scalability, sophisticated support for different types of users, flexible extensibility features, and support of relevant standards. While the DataMiningGrid, like most of the related grid systems, focused on compute-intensive applications, Google's MapReduce paradigm and Cloud-Computing brought up new solutions for efficient data analysis. Based on the DataMiningGrid, we developed the DataMiningGrid-Divide&Conquer system that combines these important technologies into a general-purpose data mining system suited for the different aspects of today's data analysis challenges. The system forms the core of the Fleet Data Acquisition Miner for analyzing the data generated by the Daimler fuel cell vehicle fleet.

Pages: 14 to 19

Copyright: Copyright (c) IARIA, 2010

Publication date: November 21, 2010

Published in: conference

ISSN: 2308-4294

ISBN: 978-1-61208-106-9

Location: Lisbon, Portugal

Dates: from November 21, 2010 to November 26, 2010