| Topic G |
| Googlebot |
| Googlebot is the of one of the of which collects the documents from web site for their indexing and ranking. Google bots fetches the pages to get through them. |
| Version of googlebot |
| Each engines googlebot has two versions simply mentioned as; |
1. Deepbot
2.
Freshbot |
| 1. Deepbots |
| Deepbots are the deep crawlers which crawls the pages those are submitted by web owners. It often tries to follow the to the pages from Google . As Google is one of the prominent search engines various web sites are submitted routinely. So deepbot takes at least one month to crawl the pages of the web sites. Deepbot crawls only those pages where file is not inserted. |
| Deepbots are worth working for search engines because they help search engine in the crystal clear judgment of indexing and ranking. |
| 2. Freshbot |
| Freshbot crawl only pages. Many web masters to make their site more informative insert innovative information in the text. Most of the times it happens that the content is partly or fully changed by the web masters. Considering them as a new page index these pages. This inspection is done by freshbots. They crawl and inspect the pages and send for search engine ranking. |
| Robot.txt and Googlebot |
| Every web site has been given the facility of preventive file. If web site indicates any such file Googlebots can crawl only permitted pages of the web site. |
| Metaname and Googlebots |
| By adding the <metaname=”Googlebot” content=”noindex”> the web page could be prohibited from crawling by Googlebot. |
| These restrictions are for both the types deepbots and freshbots. |
| |
| |
