Performance Implications of SSDs in Virtualized Hadoop Clusters
Performance Implications of SSDs in Virtualized Hadoop Clusters.BigData manipulates a massive volume of data for which the traditional techniques are not effective. Apache Hadoop is currently a most popular software framework supporting BigData analysis. As the scale of Hadoop cluster grows larger, building Hadoop clusters in virtualized environment draws a great attention. However, the performance optimization of Hadoop cluster in virtualized environment is difficult because of the virtualization overhead.
In this paper the performance implications of SSDs in virtualized Hadoop clusters is identified and the overhead of virtualization is shown to be minimized with SSDs. The study presented in this paper reveals that the main virtualization overhead is I/O bottleneck due to fragmented and randomized I/O workload aggravated by virtualization. However, SSDs are more tolerable to the workload than HDDs. As a result, the virtualization overhead with SSDs is much less than with HDDs. Also, in the case of SSDs, the virtualized Hadoop cluster sustains good performance regardless of the number of VMs.
Similar IEEE Project Titles
- ALOJA: A systematic study of Hadoop deployment variables to enable automated characterization of cost-effectiveness
- An Adaptive Auto-configuration Tool for Hadoop
- Exploiting Hadoop Topology in Virtualized Environments
- Automating the Hadoop configuration for easy setup in resilient cloud systems
- Unbinds data and tasks to improving the Hadoop performance