ALOJA: A systematic study of Hadoop deployment variables to enable automated characterization of cost-effectiveness
ALOJA: A systematic study of Hadoop deployment variables to enable automated characterization of cost-effectiveness.This article presents the ALOJA project, an initiative to produce mechanisms for an automated characterization of cost-effectiveness of Hadoop deployments and reports its initial results. ALOJA is the latest phase of a long-term collaborative engagement between BSC and Microsoft which, over the past 6 years has explored a range of different aspects of computing systems, software technologies and performance profiling. While during the last 5 years, Hadoop has become the de-facto platform for Big Data deployments, still little is understood of how the different layers of the software and hardware deployment options affects its performance. Early ALOJA results show that Hadoop’s runtime performance, and therefore its price, are critically affected by relatively simple software and hardware configuration choices e.g., number of mappers, compression, or volume configuration.
Project ALOJA presents a vendor-neutral repository featuring over 5000 Hadoop runs, a test bed, and tools to evaluate the cost-effectiveness of different hardware, parameter tuning, and Cloud services for Hadoop. As few organizations have the time or performance profiling expertise, we expect our growing repository will benefit Hadoop customers to meet their Big Data application needs. ALOJA seeks to provide both knowledge and an online service to with which users make better informed configuration choices for their Hadoop compute infrastructure whether this be on-premise or cloud-based.
Similar IEEE Project Titles
- An Adaptive Auto-configuration Tool for Hadoop
- Exploiting Hadoop Topology in Virtualized Environments
- Automating the Hadoop configuration for easy setup in resilient cloud systems
- Unbinds data and tasks to improving the Hadoop performance
- FRESH: Fair and Efficient Slot Configuration and Scheduling for Hadoop Clusters