Russian Search Engine Spiders and User-Agents

You are here: Home » Rambler » Russian Search Engine Spiders and User-Agents

Russian Search Engine Spiders and User-Agents

Nick Wilsdon · June 5th, 2008

nanobot Russian Search Engine Spiders and User Agents It’s important to know the main user-agents and search engine spiders in Runet. This can give you information on the indexing of your site, if you understand the various roles each search engine robot plays. It can also allow you to see when your competitors or link partners are cloaking their content. Inserting the following user-agent strings into the User Agent FireFox Plugin would allow you to browse the web as Yandex or Rambler.

Yandex

Yandex/1.01.001 (compatible; Win16; I)
Main Yandex Robot

Yandex/1.01.001 (compatible; Win16; H)
Robot looking for mirror copies or duplicates

Yandex/1.01.001 (compatible; Win16; P)
Robot for Images

Yandex/1.02.000 (compatible; Win16; F)
Robots which indexes favicons

Yandex/1.03.000 (compatible; Win16; M)
Robot to find keywords highlighted in the cache

Yandex/1.03.003 (compatible; Win16; D)
When you add your URL to Yandex this spider is activated

YaDirectBot/1.0 (compatible; Win16; I)
Robot for sites which install the Yandex Direct publishing code (Yandex Direct Partners/Affiliates)

YandexBlog/0.99.101 (compatible; DOS3.30,B)
Robot which indexes XML files for blog search (RSS/Atom)

YandexSomething/1.0
Robot which indexes news streams for Yandex News

Yandex/2.01.000 (compatible; Win16; Dyatel; C)
Robot which checks the availability of sites inside Yandex Catalog/Directory. If they are not available for a few days the site will be removed.

Yandex/2.01.000 (compatible; Win16; Dyatel; D)
The robots moderates Yandex Direct links inside adverts

Yandex/2.01.000 (compatible; Win16; Dyatel; Z)
Robot for the Yandex Bookmarks service

Rambler

StackRambler/x.x
Main Rambler search (Russia) robot (81.19.6x.xx)

Google

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Googlebot/2.1 (+http://www.google.com/bot.html)

Googlebot/2.1 (+http://www.googlebot.com/bot.html)

**Amazing Nanobot image by Ajax, one of the artists at TheLensFlare.com**

Technorati Tags: Cloaking, Rambler, Search Engine Spiders, User-Agents, Yandex

About the Author:
Nick Wilsdon is a marketing specialist at E3internet; an SEM agency working in the Russian and East European markets. You can contact him through this form, LinkedIN or you may follow him on Twitter.