Home // UBICOMM 2014, The Eighth International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies // View article

Bringing Context to Apache Hadoop

Authors:
Guilherme Weigert Cassales
Andrea Schwertner Charão
Manuele Kirsch-Pinheiro
Carine Souveyet
Luiz Angelo Steffenel

Keywords: Context-awareness; MapReduce; Apache Hadoop; job scheduling

Abstract:
One of the first challenges when deploying MapReduce over pervasive grids is that Apache Hadoop, the most known MapReduce distribution, requires a highly structured environ- ment such as a dedicated cluster or a cloud infrastructure. In pervasive environments, context-awareness becomes essential to coordinate the resources (task scheduling, data placement, etc.) and to adapt them to the environment variable behavior. In this paper, we present our first efforts to improve Hadoop by introducing context-awareness on its scheduling algorithms. The experiments demonstrate that context-awareness allows Hadoop to better scale based on actual resource availability, therefore improving the task allocation pattern and rationalizing resource usage in a heterogeneous dynamic network.

Pages: 252 to 258

Copyright: Copyright (c) IARIA, 2014

Publication date: August 24, 2014

Published in: conference

ISSN: 2308-4278

ISBN: 978-1-61208-353-7

Location: Rome, Italy

Dates: from August 24, 2014 to August 28, 2014