Large scale data storage and processing of insulator leakage current using HBase and mapreduce
Large scale data storage and processing of insulator leakage current using HBase and mapreduce.In the smart grid environment, huge volumes of data will be accumulated from the condition monitoring system of power equipment. Using the traditional centralized storage architecture and relational databases, the performance of data querying and processing is slow, and cannot meet real-time requirements of the power equipment condition monitoring system.
Meanwhile, MapReduce is a desirable parallel programming platform that is widely applied in kinds of data process fields.In this paper, a case-study on distributed storage using HBase and parallel processing of insulator leakage current data is presented. We propose efficient MapReduce based algorithms for parallel join query, parallel characteristics extraction and analogous assessment of insulator contamination degree. We evaluate our work on real large scale datasets utilizing Hadoop platform. Results reveal that the speedup and scale-up of our work are competent.
Similar IEEE Project Titles
- Parallelizing generalized one-dimensional bin packing problem using MapReduce
- Dache: A data aware caching for big-data applications using the MapReduce framework
- In-Map/In-Reduce: Concurrent Job Execution in MapReduce
- Aeromancer: A Workflow Manager for Large-Scale MapReduce-Based Scientific Workflows
- In unity there is strength: Showcasing a unified big data platform with MapReduce Over both object and file storage