Home // ICCGI 2019, The Fourteenth International Multi-Conference on Computing in the Global Information Technology // View article

Survey on Smart Web Crawler Algorithm Based on Semantic Search Engine

Authors:
Muneer Bani Yassein
Ismail Hmeidi
Yaser Khamayseh
Wail Mardini
Heba Al-Jarrah
Dragana Krstic

Keywords: Web Crawling Algorithms; Uniform Resource Locator (URL); Best-first search; Heuristic function; Web Crawler.

Abstract:
With the increasing amount of information and inflation of pages in the World Wide Web, it becomes difficult for the user to retrieve relevant information it needs. In other words, it leads to a big challenge for a search engine, on how to extract the exact results that match with the user query. Hence, we need smart programs such as Web Crawlers to overcome this problem. Web crawlers use different kinds of algorithms, some of these algorithms using the heuristic function to achieve the goal in an efficient manner such as A*, Adaptive A*, Best-search algorithm, and some other algorithms are not using a heuristic function such as Depth-first search and Breadth-first search. This survey covers these two types of algorithms and focuses on the mechanisms and the difference between them.

Pages: 9 to 12

Copyright: Copyright (c) IARIA, 2019

Publication date: June 30, 2019

Published in: conference

ISSN: 2308-4529

ISBN: 978-1-61208-718-4

Location: Rome, Italy

Dates: from June 30, 2019 to July 4, 2019