Russian Search Engine Spiders and User-Agents

It’s important to know the main user-agents and search engine spiders in Runet. This can give you information on the indexing of your site, if you understand the various roles each search engine robot plays. It can also allow you to see when your competitors or link partners are cloaking their content. Inserting the following user-agent strings into the User Agent FireFox Plugin would allow you to browse the web as Yandex or Rambler.

Yandex

Yandex/1.01.001 (compatible; Win16; I)
Main Yandex Robot

Yandex/1.01.001 (compatible; Win16; H)
Robot looking for mirror copies or duplicates

Yandex/1.01.001 (compatible; Win16; P)
Robot for Images

Yandex/1.02.000 (compatible; Win16; F)
Robots which indexes favicons

Yandex/1.03.000 (compatible; Win16; M)
Robot to find keywords highlighted in the cache

Yandex/1.03.003 (compatible; Win16; D)
When you add your URL to Yandex this spider is activated

YaDirectBot/1.0 (compatible; Win16; I)
Robot for sites which install the Yandex Direct publishing code (Yandex Direct Partners/Affiliates)

YandexBlog/0.99.101 (compatible; DOS3.30,B)
Robot which indexes XML files for blog search (RSS/Atom)

Yandex/2.01.000 (compatible; Win16; Dyatel; N)
Robot which indexes news streams for Yandex News

Yandex/2.01.000 (compatible; Win16; Dyatel; C)
Robot which checks the availability of sites inside Yandex Catalog/Directory. If they are not available for a few days the site will be removed.

Yandex/2.01.000 (compatible; Win16; Dyatel; D)
The robots moderates Yandex Direct links inside adverts

Yandex/2.01.000 (compatible; Win16; Dyatel; Z)
Robot for the Yandex Bookmarks service

Rambler

StackRambler/x.x
Main Rambler search (Russia) robot (81.19.6x.xx)

Google

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Googlebot/2.1 (+http://www.google.com/bot.html)

Googlebot/2.1 (+http://www.googlebot.com/bot.html)

**Amazing Nanobot image by Ajax, one of the artists at TheLensFlare.com**