Boosting Degraded Reads in Heterogeneous Erasure-Coded Storage Systems.
Boosting Degraded Reads in Heterogeneous Erasure-Coded Storage Systems.Distributed storage systems provide large-scale data storage services, yet they are confronted with frequent node failures. To ensure data availability, a storage system often introduces data redundancy via replication or erasure coding. As erasure coding incurs significantly less redundancy overhead than replication under the same fault tolerance, it has been increasingly adopted in largescale storagesystems. In erasure–coded storage systems, degraded reads to temporarily unavailable data are very common, and hence boosting the performance of degraded reads becomes important. One challenge is that storage nodes tend to be heterogeneous with different storage capacities and I/O bandwidths.
To this end, we propose FastDR, a system that addresses node heterogeneity and exploits I/O parallelism, so as to boost the performance of degraded reads to temporarily unavailable data. FastDR incorporates a greedy algorithm that seeks to reduce the data transfer cost of reading surviving data for degradedreads, while allowing the search of the efficient degraded read solution to be completed in a timely manner. We implement a FastDR prototype, and conduct extensive evaluation through simulation studies as well as testbed experiments on a Hadoop cluster with 10 storage nodes. We demonstrate that our FastDR achieves efficient degraded reads compared to existing approaches.
Similar IEEE Project Tiltles
- SiftD: A CPU & GPU distributed hybrid system for SIFT.
- A standard for benchmarking big data system.
- Power system disaster-mitigating dispatch platform based on big data.
- Bootstrapping K-means for big data analysis.
- Analysing Hadoop performance in a multi-user IaaS Cloud .