No Detail Found!

Improving hadoop through data placement strategy

Author(s)
MOHAMED EDDOUJAJI Dep. Doctoral Studies Research Center, National School of Applied Sciences,,Morocco
HASSAN SAMADI Dep. Doctoral Studies Research Center, National School of Applied Sciences,,Morocco
MOHAMMED BOUHORMA Dep. Doctoral Studies Research Center, Faculty of Sciences and Techniques,,Morocco
Abstract
Hadoop assumes that the computing capability of cluster's nodes is similar. In such a homogeneous environment, every node is assigned an identical load, such approach allows total use of cluster's resources and minimize multiple idle or over headed nodes. However, in real world applications, clusters are frequently deployed in a heterogeneous context [9,13-15]. In such environments, there is possibly multiple different physical servers and different virtual nodes specifications, which provide per consequence different services capabilities. Therefore, Hadoop still uses the same native strategy that distributes data blocks equally among each DataNode, similarly the load is evenly assigned between nodes, then the basic overall performance of Hadoop may also be reduced. The main objective during the phases of this research is to find an optimal scheme and an improved architecture to optimize the classical architecture of HADOOP and MapReduce, by focusing mainly on the algorithm of locality and data distribution in a heterogeneous ecosystem composed of several nodes heterogeneous. Despite of the native Data Placement strategy that Hadoop framework maintains by default, we present in the following a new approach that take in consideration the difference between the cluster nodes computing capabilities, with respect to the nature of tasks to adjust data blocks distribution. The design of our solution is presented under two major phases, the first one is implemented during the HDFS input, and the second one is implemented while processing tasks are initiated.
Publication Details
Page(s) 1951-1959
DOI DOI not available
Published Journal: Journal of Theoretical and Applied Information Technology, Volume: 100, Issue: 6, Year: 2022

Request the full text from PASTIC

Keywords
Data, Heterogeneous Systems, Big Data, Hadoop Distributed File System, Mapfile, Distributed Storage, Sequencefile, Distributed Systems, Distributed computing
0

Citations

0

Downloads

13

Views