in my knowledge, website spidering/crawling is like looking for some keywords/entries present in a certain website. searching thru google is one example of web spidering/crawling. So google uses Web Spiders to locate some websites with a certain keyword.
|