How do spiders work?

[cloud9925]

I was wondering how spiders actually did their job? Are there any FAQ's out there that you have the url too? Or are there any really in depth informative sites out there that one knows of that I could puruse?

I would greatly love to learn about these spiders and how they do their job and even how they are programmed to do their job source code and what not.

I love learning about these technologies and would love to learn more about spiders and how they work.

Thanks

[netgal06]

May I suggest to to take a look at this website http://computer.howstuffworks.com/search-engine1.htm as It goes into greater depth in explaining the intricate working of a spider(with diagrams) , that I couldn’t possible explain here.

In short though, a spider visits websites and it examines s the sites content including meta tags (which is why its always important to have productive keywords on your site) Using the content it will compile a list of keywords that help users find what they are looking for.


For example if your website is on fishing it pays to use fishing keywords throughout your site. Even other words that have the same meaning as fishing. Cover all your bases.

[backstage]

Search engines send out automated programs - spiders , which use the web’s hyperlink structure to “crawl” it's pages.

After spiders crawl a page, its content needs to be put into a format that makes it easy to retrieve when a user queries the search engine. Thus, pages are stored in a giant, tightly managed database that makes up the search engine’s index. These indexes contain billions of documents, which are delivered to users in mere fractions of a second.

When a user queries a search engine, the engine examines it's index to find documents that match.

[maggots]

very well said backstage,
i just love forums like these where people help each other out without expecting any form of financial returns

[080522jk]

自动语音信息服务IVR（公司简介，产品介绍，通知等）　　从提高生产效率的角度考虑，呼叫中心呼叫中心系统应该尽量以自动方式为用户提供服务。电话录音盒本方案中的自动语音应答系统具有以下特性：电话打入时系统自动播放录制好的IVR欢迎词； IVR业务生成灵活方便，语音卡可以自行设计应答流程及多层结构；每一部分提示语音可自行单独录制； 系统接收、识别并记录来电的主叫号码；（公司简介，产品介绍，通知等）　　从提高生产效率的角 度考虑，电话监听呼叫中心系统应该尽量以自动方式为用户提供服务。呼叫中心本方案中的自动语音应答系统具有以下特性：电话打入时系统自动播放录制好的IVR欢迎词； IVR业务生成灵活方便，可以自行设计应答流程及多层结构；每一部分提示语音可自行单独录制； 系统接收、识别并记录来电的主叫号码；

[joef5052881]

Hong Kong's Sing Tao News Corp. news, Ma Ying-jeou and Taiwan's Vice President Vincent Siew, 17 in the afternoon with a reception. He said that the young had to defend the Diaoyu Islands into action and to the sovereignty of the Diaoyu Islands and the demarcation issue, he stressed that the Diaoyu Islands is a territory of the Republic of China, Taiwan is also an island, the Diaoyu Islands belong to Taiwan's Ilan County jurisdiction. He said that Taipei the same determination to defend the sovereignty of the Diaoyu Islands, should be through peaceful means to resolve the sovereignty dispute, it can not solve the non-peaceful means should be considered. Ma Ying-jeou reiterated that the Japanese ships sank fishing boats in Taiwan, to demand that Japan compensate and apologizeHong Kong's Sing Tao News Corp. sources, Legislator Lin pointed out that KMT Chairman Wu Po-hsiung and senior Presidential Office that the civil and government's hardline attitude, has made Japan's attitude change, hope to have time for further coordination. Lin further said that to defend the Diaoyu Islands and the Coast Guard Administration yesterday vessels approaching the Diaoyu Islands have 0.4 sea miles, to a certain extent, has proclaimed the sovereignty of the Diaoyu Islands

[weiwei]

The colored lighting apparently wow gold helps influence the brain’s production of melatonin wow gold which is linked to the body’s transitions wow power leveling between feeling awake and feeling sleepy. So when you use a regular light bulb for reading before bed you can throw off the levels of melatonin which in turn makes it harder to fall asleep. The lighting wow power leveling in the NightCove however is specially tuned to keep things in balance which should make it easier to drift off when you’re ready. In the morning it then wakes you up gradually with a combination of blue and white lights to ensure you feel awake instead of wanting to wow powerleveling hit the snooze button.weiwei1978123