Language based web crawling on big data.

Language based web crawling on big data      

                                 Language based web crawling on big data. Online textual and visual data that are created and used by web users have been increasing dramatically and continually. This increase has caused the need for easy and fast access to online data and facilitated the development of alternative means of access to this data. Nowadays, web crawlers are the most efficient and popular tools used for accessing big volumes of data available on the web.

Big-Data Projects

Big-Data Projects

In this paper, a web crawler which works on a distributed Hadoop cluster for crawling web pages with content of a predefined language is described. A language identification tool is developed for enabling the system to focus only on a specific language. In this study, the accuracy of the language identification tool is evaluated on a small data set (consisting of 4729 web pages). The performance of the focused web crawling system is reported on a big data set of 86 million web pages containing Turkish content

Similar IEEE Project Titles

Save


Work Progress

PHD - 24

M.TECH - 125

B.TECH -95

BIG DATA -110.

HADOOP -90.

ON-GOING Hadoop Projects

HADOOP MAP -90.

HADOOP YARN -27.

HADOOP HEBROS - 25.

HADOOP ZOOKEEPER -18.

Achievements – Hadoop Solutions

Hadoop-Projects-Achievement-Awards

Twitter Feed

Customer Review

Hadoop Solutions 5 Star Rating: Recommended 4.9 - 5 based on 1000+ ratings. 1000+ user reviews.