SEO and SEM tips for Russian search engines

The evolution of Yandex search algorithm

Yandex as a search engine was created 16 years ago, back in 1997, and became a brand in 2000. A lot has changed during the years, both for users and for SEOs. Let’s have a look at the evolution of Yandex search algorithm.

There are no records or announcements about Yandex ranking formula dated earlier than 2007. 2007 was the year when Yandex started actively working with their search algorithm and communicate the changes to the community. Ever since, every new ranking formula was named after a city in Russia, and the logic behind naming them is taken from an old Russian game called “cities”: each newly introduced formula’s name starts from the last letter of the previous formula’s name:

Magadan -> Nakhodka -> Arzamas -> Snezhinsk -> Konakovo -> Obninsk -> Krasnodar -> Reykjavik -> Kaliningrad. On the map this journey would look like this:

yandex-search-algorithm-journey

Let’s look a what this journey looked like for Russian SEO.

Magadan (April 2008)

The search algorithm Magadan was a revolution. It had at least twice as many ranking factors as the previous one, where uniqueness of site content received quite an important role.

Yandex learned to understand transliteration (spelling Russian words with Latin characters – for instance, in URLs), commonly used abbreviations, and rank documents based on this knowledge.

Another interesting feature was that Yandex started to better understand text and to rank relevant documents even if they did not contain exact keyword combinations.

Magadan was also the algorithm that started indexing and ranking websites in English.

Nakhodka (September 2008)

Nakhodka came half a year later. At that point Yandex learned to understand the meaning of stop-words (prepositions, conjunctions etc.) and use them for ranking documents.

Thesaurus of the search engine was expanded, which lead to to better understanding and ranking of synonyms and spelling variations.

Arzamas (April 2009)

Arzamas was the algorithm that brought real local search to Yandex. It was based on classification of all search queries into two types: geo-dependant and geo-indepenant. For geo-depenant queries the local businesses received priority in local SERPs (e.g. a Novosibirsk taxi company site would rank better in Novosibirsk than a site of a taxi company from Moscow).

Additionally, Arzamas was the first ranking algorithm that went hard on pages with pop-up and pop-under advertising.

Snezhinsk (November 2009)

In Snezhinks Yandex introduced several thousands new ranking factors, which provoked somewhat of a panic in the Russian SEO community. The main changes I noted were:

- Trusted sites (brands) got generally higher positions in search results

- Unique content because a must, and sites with duplicate content got punished

- Old links became more valuable than newly acquired ones

- New type of penalty for low quality sites – AGS – was introduced

All these changes made it more difficult and time-consuming for new sites to get first page of Yandex search results.

From a technology perspective, MatrixNet (self-learning engine) was launched.

All these changes took Yandex SEO to a new level, where quality of SEO work became more important than quantity of links.

Konakovo (December 2009)

Local search algorithm, introduced initially for 19 regions in Russia with the launch of Arzamas, was extended to 1250 cities throughout the country.

Obninsk (September 2010)

The main difference of that Obninsk brought to Yandex SEO was devaluing a large amount of “SEO links” (paid links).

Other changes in the algorithm included changes to ranking formula for geo-independant search queries and better way of identifying the initial source of content on case of duplication.

Krasnodar (December 2010)

Krasnodar algorithm incorporated another intelligent technology – Spectrum – the goal of which was to understand user’s search intent and serve better search results for words with more than one meaning or search queries that are too generic.

Reykjavik (August 2011)

Reykjavik was the next step towards understanding the user’s search intent. This algorithm learned to identify language preferences of a user, and depending on that, display more or fewer search results in English.

The algorithm mainly concerned searches in foreign languages, hence named after a foreign city.

Kaliningrad (December 2012)

Kaliningrad brought personalized search to Yandex users.

According to the company’s announcement 75-80% of all search results became personalized for everyone, regardless of whether the user is logged in or not (unlike Google’s personalization).

Yandex claims that the audience responded better to the personalized results, while SEOs have to once again learn ow to adjust to the new environment.

Looking back, Yandex has come up with a new significant change in their search algorithm at least one per year, and sometimes twice. What will be the next one? All we can say right now is that probably the name will start with a D.

Anna

Anna is a blogger and online marketing professional specializing in SEO and SEM for Russian search engines. To see more of Anna's posts, follow her on Twitter, or LinkedIn.

More Posts

Yandex story to become a plot for a movie

I bet all of you have seen “Social Network”, the movie about Facebook. Apparently a Google movie is also coming up. Following their Western colleagues, Russian film makers decided to produce a movie about yet another IT success, the most popular Russian search engine Yandex.

The movie under production title “Startup” is a joint effort of producer Irina Smolko, screenwriter Dmitry Sobolev and director Roman Karimov. The basis for the plot is the development of the Russian IT industry from the 90s until today, where Yandex is obviously one of they key figures.

Roman Karimov. Image credit: http://www.kinopoisk.ru

The crew is currently meeting representatives of various Russian IT companies and working on the screenplay.

“Dmitry can write a story that can be of interest to foreign audiences”, said Smolko, and added that they have already started looking for potential partners abroad.

Dmitry Sobolev. Image credit: http://www.kinopoisk.ru

Yandex spokesman admitted that they met the film makers and talked about the project, however said that the production is done without any financial support from Yandex.

Anna

Anna is a blogger and online marketing professional specializing in SEO and SEM for Russian search engines. To see more of Anna's posts, follow her on Twitter, or LinkedIn.

More Posts

Yandex: The Human Factor

It’s a well known fact that Google uses human raters in their search quality assessment. Yandex mainly talks about their machine learning and other fancy technologies, while not many know that human factor is involved in Yandex search just as much as in Google’s.

How Yandex collects the assessment data

Russia is a big country, and having committed to displaying perfect (or almost perfect anyway) local search results, Yandex needs a lot of human raters in all regions of the country (which they do). The raters work remotely, from home, and the results of their work are closely monitored by Yandex.

Interestingly, Yandex hires a lot of these people through Amazon Mechanical Turk service.

The raters are given random SERPs from real search queries and rate the documents according to the scale:

Example of a result of manual assessment of two search result pages

Vital (the best answer possible; usually official sites of organizations)

Useful (very good and informative answer)

Relevant + (answers the question)

Relevant - (partly relevant, but does not answer the question fully)

Irrelevant (does not answer the question)

The raters are given tasks like to evaluate a specific document, evaluate search results for a particular query, evaluate site snippet in a SERP, compare two documents and pick the most relevant to a specific search query, compare two search results pages and pick the best.

Mainly the human assessments are used on top 10 results, but can be also applied to further positions, depending on Yandex’s goal.

How Yandex uses the human ratings

There are two main ways the human assessments are used at Yandex: for evaluating quality of search results and for “teaching” MatrixNet, the machine learning technology that powers Yandex search results.

Yandex search quality measurements

Yandex has many different metrics to measure the quality of search results, one of them being pFound. pFound measures probability of that the user will find the answer he / she is looking for, based on hypotheses that a) the user will browse the SERP from the top to the bottom and b) the user will click on every document until he / she finds the answer or leaves the SERP without the answer.

Yandex keeps track of historical values of pFound, measuring the effect of changes in the ranking formula on the value of the variable. The example below shows a big improvement of pFound value after Yandex changed the way of handling mistypes:

Educating MatrixNet

Yandex is very proud of their MatrixNet, which has indeed significantly improved their search results since it was implemented, but they also understand the shortcomings of using the algorithm for judging content written by humans. Algorithms rely on rules and look for patterns in order to determine whether the document is relevant or not, e.g.

the document contains the search term + the document is clicked + the document is linked -> the document is probably relevant

And as all SEOs know, there are always ways to cheat the algorithms.

Human raters, on the other hand, without knowing the rules, classify documents as relevant or irrelevant according to their knowledge, feeling, understanding and common sense. Reviewing the data received from human raters, the algorithm is trying to find and learn new, less obvious patters and factors, e.g.

The document is rated relevant when it contains X factors (words, page structure etc.)

The document is rated irrelevant when it contains Y factors (words, page structure etc.)

Other search quality improvement techniques at Yandex

The technology described above is very powerful, but it alone is not enough to understand the complexity of the Russian language. Some words have several absolutely different meanings (e.g. Napoleon – a cake or the French emperor?), some search terms are too generic terms like, for example, “vidoes” (what does the user want – to watch, to make or to download?).

Yandex has been focusing on determining user intent for quite some time now. I mentioned some of these attempts in my earlier posts.

For better understanding of user intent, Yandex uses, among other techniques, search session analysis and SERP experiments.

When performing search session analysis, Yandex look at what queries users type in, how they reformulate their queries if no answer found from the first attempt and where users click eventually, if click at all.

SERP experiments basically means that a change in the search algorithm is being rolled out on a certain percentage of search queries. After running the experiment for a period of time, Yandex analyzes how the change affected user behavior, e.g. how much the percentage on non-clicked results changed, what was the average position of 1st clicked result and about 10500 other factors.

Yandex has a smart and complicated technology that is becoming better and better at understanding users. I find this fascinating. What’s your take on it?

Anna

Anna is a blogger and online marketing professional specializing in SEO and SEM for Russian search engines. To see more of Anna's posts, follow her on Twitter, or LinkedIn.

More Posts

Yandex Goes Poland!

The latest gossip in Russian Internet circles is that the next step for Yandex after their NASDAQ IPO, successful launch in Turkey and recent partnership with Czech leading search engine Seznam is to launch in Poland. Rumors say that Yandex has already hired a business developer in Poland, with a very impressive track record including companies like Dell and Apple.

Polish Internet audience is estimated to be around 40 million people, where Google holds 97% market share. Entering a market with such strong Google presence might sounds like a strange idea, especially given 2 facts from the recent past:

1. Yandex already made an attempt to enter Polish market back in 2001, but pulled off due to the crisis in .com industry as well as uncertain state of Polish economy.

2. The announcement of international expansion plans Yandex made in summer was very much about entering markets where Google does not have a strong dominant position. The first of these markets was Turkey. Yandex also mentioned Vietnam, Indonesia and Egypt.

So why Poland?

The rumors about Yandex planning an acquisition of Onet.pl (one of the biggest portals in Poland) were circulating in the tech press since summer 2011 (see, for example, Bloomberg report from July). Purchasing of the other major portal Wp.pl also gets mentioned here and there.

Both portals have look and feel of Yahoo!, i.e. publish a lot of categorized content and at the same time have search. Onet.pl search box is currently powered by Google; wp.pl has its own search engine.

In case of this purchase, Yandex will actually claim a big chunk of Polish market right away:

Anna

Anna is a blogger and online marketing professional specializing in SEO and SEM for Russian search engines. To see more of Anna's posts, follow her on Twitter, or LinkedIn.

More Posts

Yandex ranked the world’s 5th largest search engine

In September 2011 Yandex processed over 3 billion search queries, reports comScore. This allowed Yandex to climb 2 positions up passing South Korean NHN Corporation (Naver) and eBay.

The leader is, obviously, Google with 118 billion queries (globally), followed by Chinese Baidu with 11 billion queries.

Yandex has been showing exceptionally fast growth in both traffic and revenues over the last years. Earlier this year the company raised $1,4 billion in a NASDAQ IPO, and a few months later reported astonishing 93% jump in net income comparing to Q3 of 2010. The company’s value is around $8 billion, which makes it one of the top 25 leading online buisnesses in the world:

Source: http://kpcb.com

Anna

Anna is a blogger and online marketing professional specializing in SEO and SEM for Russian search engines. To see more of Anna's posts, follow her on Twitter, or LinkedIn.

More Posts

Yandex SERP updates analyzer

Yandex is very different from Google, when it comes to stable rankings and updates of search results. It is very important to keep in mind that while Google SERPs are never 100% stable, in Yandex not many changes happen on a daily basis.

This used to be the case with Google before as well, but it is still happening in Yandex. As I mentioned in one of my previous posts, every 2 weeks or so Yandex rolls out a big update and the SERPs change noticeably within 1-2 days. Compare the bars on the graph (vertical axis indicates percentage of change in the SERPs):

Yandex SERP update analyzer

I would like to introduce you to a great tool to monitor these Yandex updates – Update Analyzer (available in English!). The tool was developed by the biggest Russian SEO company Ashmanov and Partners, which is also famous for organizing of  the main Russian SEO conference Optimization.ru and a big SEM event eTarget.

The tool monitors and quantifies a number of parameters across all search engines used by Russian people, including even smaller ones, such as GoGo and Aport. While I have never figured out how to use Spam level or SEO-pressing, I use the SERP udate analyzer for Yandex on a regular basis.

Keep in mind that during the SERP update sites tend to disappear and reappear in the SERPs. If a strong competitor or yourself is missing from the search results, it is very likely a Yandex SERP update is in progress. Do not freak out. The update takes 1-2 days and after that the SERPs come back to normal. I recommend to check the updates stats every now and then to avoid such stressful situations!

Enjoy!

Anna

Anna is a blogger and online marketing professional specializing in SEO and SEM for Russian search engines. To see more of Anna's posts, follow her on Twitter, or LinkedIn.

More Posts

Interview with the CEO of Yandex

Interesting interview. Found on the net a couple of days ago. The CEO of Yandex is talking (in English!) abough Yandex and his visions.

My favorite quotes:

  • We planned IPO a year ago. Now we don’t need IPO. We have enough money.
  • There is much more money available in Russia than brains
  • 90% of searches are in Russian
  • Pople use Yandex more than Google, because we search better

Anna

Anna is a blogger and online marketing professional specializing in SEO and SEM for Russian search engines. To see more of Anna's posts, follow her on Twitter, or LinkedIn.

More Posts

Snezhinsk has gone live

Amazingly fast, the new search algorithm Snezhinsk went live on Yandex. It was announced today at 19.55 Moscow time on Yandex Webmaster blog (webmaster.ya.ru). Yandex, unlike Google, which promised to wait with Caffeine until after the Christmas break, was not afraid to give a hard time to webmasters and businesses right before winter holidays.

I noticed Snezhinsk SERPs were appearing every now and then earlier this week, which shows that the new algorithm went live earlier in several data centers..

Apparently, there are over 1000 ranking factors now… Looks like Yandex is catching up with Google. Not as kind, but almost as picky =)

I will test and report =) Stay tuned =)

Anna

Anna is a blogger and online marketing professional specializing in SEO and SEM for Russian search engines. To see more of Anna's posts, follow her on Twitter, or LinkedIn.

More Posts

Russian link directories and press release sites – good and bad

masksBacklinks of a typical Russian website is a total nightmare. You will probably find lots of spammy directories and social bookmarks. This is how the SEOs in Russia usually begin their SEO efforts to get sites quickly indexed and receive initial PR and Quotation Index.

There are dozens of services and scripts offering to register your site in hundreds of directories (one of the most common is called 1PS). There are also directories working with reciprocal linking. Well well… This seems to work in Russia, and people keep doing it, however linking to all the link farms will lead to penalization for link spam.

My personal opinion: stay away from spammy directories. Might work today, but won’t work tomorrow. Focus on getting into few, but valuable ones, which will give your site trust in the eyes of search engines and searchers, and maybe even some traffic. Here is some to start with:

http://yaca.yandex.ru/ – link directory of Yandex. Being listed there will boost your quotation index and attract visitors (as valued by Yandex as DMOZ by Google).
There are two ways to get it: to pay or not to pay and hope for the best. Paid listing is a matter of days. It costs around $400. Site might however be rejected due to bad quality. YaCa guys are strict! Free listing is a matter of luck. You can apply and wait for several months. They never inform you about their decision.

http://list.mail.ru – link directory of the biggest email service provider and news/ entertainment portal. According to their rules, only charity organizations, educational institutions and such’s sites can be accepted for free. Commercial sites have to pay, but then again, if you pay – you get a direct link in return. Traffic from this directory is almost zero.

http://top100.rambler.ru/top100/ – link directory of Rambler, another local search engine.

I also recommend look for directories in your specific niche.

Bonus : a big list of Russian press-release sites: http://forum.searchengines.ru/showthread.php?t=345469. I just found it in a forum. Haven’t check them yet, but at least some must be good =)

Anna

Anna is a blogger and online marketing professional specializing in SEO and SEM for Russian search engines. To see more of Anna's posts, follow her on Twitter, or LinkedIn.

More Posts

Regional SERPs on Yandex

In April Yandex launched a new algorithm known as Arzamas these days (Arzamas is a name of a Russian city btw). Apart from many new factors affecting rankings, such as for example duplicate content filter that removed over one million of pages from the index in one night, in the new algorithm was implemented a new geotargeting feature.

Even before it was possible to search among websites that belonged to a specific region by selecting the region manually in advanced search options. Arzamas went further. Arzamas assigns websites to a specific region automatically. How? Haha.. we wish we knew =) Sometimes it gets it all wrong…

The known factors defining a region your site is attached to are:

- location-by-IP data
- domain names
- content-related features: ZIP and area codes on your contact page
- region you assign to the site yourself when registering it in Yandex Directory

Some people speculate that origin of your backlinks has an impact, i.e. if sites from, say, Vladivostok link to you that means your site is also from Vladivostok.

You can easily find out the region a site assigned to with the help of Yandex bar. Just hover over the Quotation Index indicator. Three possible options are available:

Region – Russia. This means the site will rank the same all over the country

Yandex bar: region Russia

Yandex bar: region Russia

Region – unknown. Here it is hard to predict how this will affect your cross-regional rankings. From my experience, you can still see differences among regional SERPs for smaller sites.

Yandex bar: region unknown

Yandex bar: region unknown

Region – (city/ region name). This means your site will rank better in that particular region.

Yandex bar: region assigned

Yandex bar: region assigned

Yandex provides a tool for comparison of regional SERPs: http://webmaster.yandex.ru/wmconsole/compare_regions.xml. Please try it out. The results can surprise you =P

Right now regional SERPs fully function for Moscow and St. Petersburg from what is known officially. Algorithms for the other regions of Russia are to be released later. Yandex claimed that the goal is to distinguish between 50 regions at least.

Please note that when searching in Yandex from Europe/ US the SEPRs are completely messed up. I highly recommend to use a Russian proxy when checking rankings.

And the last note for this post. Today Yandex released a beta version of Arzamas 1.1, which you can test here: http://buki.yandex.ru/. This new algorithm will be used for all Russian regions excluding Moscow, St. Petersburg and Ekaterinburg. Apparently if you add &lr=* (* is a code of your region) while searching in this beta version, you can see your regional SERPs. By adding &lr=225 Yandex eta will show SERPs for region “Russia”.

Anna

Anna is a blogger and online marketing professional specializing in SEO and SEM for Russian search engines. To see more of Anna's posts, follow her on Twitter, or LinkedIn.

More Posts