A Distributed NameNode Cluster for a Highly-Available Hadoop Distributed File System
A Distributed NameNode Cluster for a Highly-Available Hadoop Distributed File System.Recently, Hadoop attracts much attention of engineers and researchers as an emerging and effective framework for Big Data. HDFS (Hadoop Distributed File System) can manage huge amount of data with high performance and reliability using only commodity hardware. However, HDFS requires a single master node, called a NameNode, to manage the entire namespace of the file system.
This causes the SPOF (Single Point Of Failure) problem because the file system becomes inaccessible when the NameNode fails. This also causes a bottleneck of efficiency since all the access requests to the file system have to contact the NameNode. Finally the scale up of a namespace is difficult because the NameNode manages all metadata of the namespace on its own memory, which is limited and expensive resource. In this paper, we propose a new HDFS architecture consisting of several NameNodes to resolve all the above problems.
Similar IEEE Project Titles
- Applying Eco-Threading Framework to Memory-Intensive Hadoop Applications
- An approach for fast and parallel video processing on Apache Hadoop clusters
- A Hadoop Extension to Process Mail Folders and its Application to a Spam Dataset
- Design and Evaluation of Network-Levitated Merge for Hadoop Acceleration .
- Perldoop: Efficient execution of Perl scripts on Hadoop clusters .