| Agent Name |
| For your surprised dear visitors Agent name is not the name of any company’s proprietor or agent but it is the identity of the or the browsers who crawls or browses on the web page. We all are well aware that for indexing and ranking crawlers crawl on the web page of various web sites. Yes friends! Now has given separate identity to these crawlers. |
| Few agents names are listed below |
1 Googlebot
2 Scooter
3 Gulliver
4 Infoseek sidewinder
5 lycos spider |
| Good & bad agents |
These all are good spiders who really crawl to give to web page. They are very particular about the custom created by search engines. (The custom of , , ,content purity etc.)
|
| But viewers there are some bad spiders also who try to deteriorate the quality of the web page. There are some spiders those are generated from the software called Teleport Pro where the owner of the agent is unknown. They have the capacity to the full mirror of your web site. So they are dangerous for your site. |
| Protect your site from bad agents |
Introducing file in your web page will be the best way to protect your web site from such bad agents.
|
Here is an example for you
User-agent: NameOfAgent
Disallow: / Don’t use a blank robot.txt which commands the crawler not to crawl your web site.
|
| Agent name deliver |
| Agent name delivery is the skill to control the spiders from viewing the page which is visible to visitors of website and crawl the page which is techniqually set to be viewed by the . We recognize this art by the name (the type of ) where different pages are displayed using same URL to search engine and viewers. This is generally practiced to manipulate spiders. |
| |
| |
