How search engines work

[hassen1]

he term "search engine" is often used generically to describe both crawler-based search engines and human-powered directories. These two types of search engines gather their listings in radically different ways.

Crawler-Based Search Engines

Crawler-based search engines, such as Google, create their listings automatically. They "crawl" or "spider" the web, then people search through what they have found.

If you change your web pages, crawler-based search engines eventually find these changes, and that can affect how you are listed. Page titles, body copy and other elements all play a role.

Human-Powered Directories

A human-powered directory, such as the Open Directory, depends on humans for its listings. You submit a short description to the directory for your entire site, or editors write one for sites they review. A search looks for matches only in the descriptions submitted.

Changing your web pages has no effect on your listing. Things that are useful for improving a listing with a search engine have nothing to do with improving a listing in a directory. The only exception is that a good site, with good content, might be more likely to get reviewed for free than a poor site.

"Hybrid Search Engines" Or Mixed Results

In the web's early days, it used to be that a search engine either presented crawler-based results or human-powered listings. Today, it extremely common for both types of results to be presented. Usually, a hybrid search engine will favor one type of listings over another. For example, MSN Search is more likely to present human-powered listings from LookSmart. However, it does also present crawler-based results (as provided by Inktomi), especially for more obscure queries.

[humpty]

hmmm, resourceful and informative, it's good to know and understand how they work if you wanna have any optimization done

[w3bmaster]

Very good post hassen.......clarifies everything . I was thinking yesterday where I could find dictionary like type definitions about se's and seo and I think I have find the information now .I appreciate all the information you've shared with us, including your post above. Thanks.

[Gracia]

All crawler-based search engines have the basic parts described above, but there are differences in how these parts are tuned. That is why the same search on different search engines often produces different results. Some of the significant differences between the major crawler-based search engines are summarized on the Search Engine Features Page. Information on this page has been drawn from the help pages of each search engine, along with knowledge gained from articles, reviews, books, independent research, tips from others and additional information received directly from the various search engines.

[mini_0]

More important and in abundance are the crawler based searchengines , so basically a webmaster pays more atention to optimising for them . This page by page crawl by the search engine spiders goes on 24/7 all over the web. This information is stored in a giant database and is indexed, which provides the search engine a roadmap back to the information by keyword, hyperlink, and Meta tag. These databases contain the locations of billions of documents for future reference.