To learn about how search engine works, we have to first understand what is search engine? There are various definitions of search engines given by universities and search engine developers. The most commonly used definition is.
A search engine is a program designed which helps to find information stored on a computer system such as the World Wide Web or a personal computer. The search engine allows one to ask for content meeting specific criteria (typically those containing a given word or phrase) and retrieving a list of references that match those criteria. Search engines use regularly updated indexes to operate quickly and efficiently.
There are two types of search engines i.e. Robot based search engines and Human edited search engines. Now we will explain you in details about working of the both types that is essential for the SEO Company to optimize the web site by search engine optimization strategies to generate better result.
Robot based search engine:
This is also known as crawler, spider or ant based search engine. These are based on information that is collected, sorted and analyzed automatically from the website by indexing spiders. A software program, (known as a "robot", "spider" or "crawler") reads or indexes the web pages, follows links between pages and sites and collects information stored for later use. The information collected is analyzed into an "index" which is a large database of all the sites the crawler visited and read. A Web crawler is one type of bot, or software agent. In general, it starts with a list of URLs to visit, called the seeds. As the crawler visits these URLs, it identifies all the hyperlinks in the page and adds them to the list of URLs to visit, called the crawl frontier. URLs from the frontier are repeatedly visited according to a set of policies of search engines.
Using combination of policies web crawlers are behaving such as :
* Page selection policy to state which page to download
* Page re-visit policy for checking any updating in web pages
* Politeness policy that states how to avoid overloading websites
* Parallelization policy that states how to coordinate distributed web crawlers.
Human-powered search engines:
This type of search engine relies on humans to submit information that is subsequently indexed and catalogued. Only information that is submitted is put into the index. You submit a short description to the directory for your entire website or SEO writer writes one for websites they review. A search looks for matches only in the descriptions submitted for search engine marketing.
It is very important to know that how sites are listed in search engines. The spider crawls in the web site starting from known pages and follows all the links in the sites. The spider also visits the pages which are submitted manually. Search engines are trying to encourage site owners to pay for the privilege of having their pages to visit by spider.
In both type of search engine, when you search for any word or specific query, you are actually searching through the index created by search engine. The results produced by the search engine will depend on the contents or website competition in the index. Each page stored in the database directory is ranked based on the contents of each web page including the title of the page, Meta tags, text, images etc. A good site, with good content, might be more likely to get reviewed.
Specific page's relevance ranking for a specific query depends on the factors such as page relevance to the words and concepts in the query, its overall link popularity and whether or not it is being penalized for excessive search engine optimization (SEO).
The index of search engine is large database of information which are methodically collected and stored by search engines. The search result pages are depending on these indexes. Search engines frequently updates these indexes as there are thousands of new sites added everyday. Since the search results are based on the index, if the index hasn't been updated since a Web page becomes invalid the search engine treats the page as still an active link even though it no longer is. It will remain that way until the index is updated. When you add or update content in your website and resubmit it in search engine, the search engine stores updated information in its database. But updated result will only appear when search engine updates its index.
Sometimes when you search in different search engines the same search shows different results. One of the reasons is all search results page depends on the algorithm to search through indexes. All search engines use different algorithm method for search. The algorithm is what the search engines use to determine the relevance of the information in the index to what the user is searching for.
When search engine gives page rank it gives special weightage on keyword that appear first in title, Heading Tags, Bold face, description, Alt tags in images, keyword and keyword phrases in meta tags.
One of the elements that a search engine algorithm scans for is the frequency and location of keywords on a Web page. Web pages with higher no. of keywords are typically considered more relevant. For example, one method is to rank hits according to how many times your keywords appear and in which fields they appear (i.e., in heading tags, titles, Meta tags or plain text). Another method is to determine which documents are most frequently linked to other documents on the Web. Another common element that algorithms analyze is the way that pages link to other pages in the Website. By analyzing how pages link to each other, an engine can both determine what a page is about and the link pages (match keyword of original page and link page) and whether that page is considered "important" and deserving of a boost in search engine ranking.
As far as the user is concerned, relevancy ranking is critical, and becomes more so as the total volume of information increasing with the growth of websites. Most of the people have no time to go through scores of hits to determine which hyperlinks we should actually explore. The more clearly relevant the results are, the more we are likely to value the search engine.
http://www.topranker.in/search_engine_work_list_website.htm#working_of_search_engine_for_site_listing
