Fast indexing in Yandex. Search index


Hello! Today I will tell you very important things, if you miss them you can lose traffic. Often, as you work on websites, you need to determine which pages are indexed and which are not. Which pages require additional attention to be included in the index.

This is especially noticeable when working with online stores: when working with a huge number of products/sections, more and more new pages are constantly being added. Therefore, strict control over the indexing of newly added pages is needed so as not to lose search traffic.

In this short lesson I will tell you how I check site pages for indexing.

How to check pages for indexing

I have already described how I check the indexing of a specific page using the RDS bar.

Well, or you can simply enter this code into Yandex:

Url:www..ru/about

Or for Google:

Info:https://site/about

Of course the URL website/about change it to yours.

But what to do if you need to check the indexing of tens/hundreds, or even more articles? I proceed as follows:

  1. We put a wonderful free YCCY program(you can download it from here).
  2. Let's launch it and go to Indexator:

  3. On the left side of the program we load the list of URLs that need to be checked for index:

  4. In the settings when working with Yandex, I set it to work via Yandex XML. and what they eat it with, I already told you:

  5. We select the search engine we are interested in, mark what interests us (in our case, “Indexing”) and click on the “Start checking” button:

  6. And voila, on the right you will see a separate list of indexed pages, as well as what is not yet in the index:

What to do with unindexed pages?

Services to speed up indexing

I usually run unindexed pages using different services; I prefer this getbot.guru. Yes, of course, not 100% of pages get into the index using it, but still, on average, about 70-80% of the URLs crawled usually get in (it also depends heavily on the adequacy of the pages).

The service is, of course, paid, you need to pay for each URL. For those pages that are not included in the index, a refund will follow, which is very fair and tempting (depending on the tariff). And I re-send these pages that were not included in the index to the service. Again, with the next update, some of these pages are included in the index.

I think you can figure it out on your own, it’s not complicated. Register -> Create a project -> Launch the project. The only thing is that difficulties may arise when choosing a tariff. I prefer to work with the “Absolute Update” tariff (click on the image to enlarge):

By the way, please note that the service can also check pages for indexing. The issue price is around 10 kopecks per 1 URL. I prefer the free YCCY, which I wrote about above.

The Yandex Quickbot is sent to pages from the project, the indexing of the project pages in Yandex is periodically checked, and the Quickbot is sent again to pages not included in the index. After completing the task, for pages not included in the index, funds are automatically returned to the balance. We do not use social networks and spam methods to attract fast bots. To operate the service, we use our own network of news sites, which we own.

Speed ​​up indexing using Twitter or news sites

Well, if you don’t like running with the help of such services, then you can use Twitter. Search engines “eat” links on Twitter very well if the accounts are more or less adequate.

I have already written about this in relative detail in the lesson ““. There, in addition to Twitter, I looked at other ways of fast indexing.

Reindexing existing pages

Similarly, you can send a page for re-indexing. By checking the page cache in search engines, you can find out whether the page has been indexed or not. I talked about how to check the page cache in .

Working with an index using Comparser

YCCY may be an old program, but I really like it. I use it to check the indexing of pages I know.

If you need to find out which pages of the site have not yet been included in the index (I already wrote a whole lesson about it). Its principle is simple: it downloads all site URLs that you have allowed for indexing (this item can be changed in the settings) and checks each page for indexing. Or, with a simple request, it unloads the first 1000 pages from the index.

In this instruction we will consider the issue of adding our new site for indexing to various search engines.

I decided to mention both popular search engines and those that you may not have heard of.

Site indexing in Yandex

To add a site for indexing, just enter the url of the main page and the captcha. Captcha is a few numbers that protect against automatic registrations. After you click the “add” button, several options for the development of events are possible.

1) The message “your site has been added” signals the successful addition of a resource to the queue for indexing in Yandex.
2) If the message “Your hosting is not responding” appears, it means that your server is down at this moment. You can try adding a site later or find better hosting.
3) But if a message appears that “the specified URL is prohibited from indexing,” then things are bad. This indicates that sanctions have been imposed on your resource in the form of a site ban. It is quite possible that the domain you purchased already had a website that received sanctions. Using addurl, webmasters often check sites for bans in Yandex.

Site indexing in Google (Google)

The next most important search engine for our site is Google. The process of adding a site to indexing in Google is exactly the same as on Yandex. Google also has its own adurilka, which is located at: https://www.google.com/webmasters/tools/submit-url.

You also need to enter a captcha when adding. But there are two differences. If in Yandex you can simply add a URL without unnecessary movements, then in Google you need to be logged in to your account. Otherwise it won't work. Accordingly, if you don’t have an account there yet, you will have to create one. The second difference between Google and Yandex is the speed of indexing. Google indexes websites very quickly.

Site indexing in Rambler (Rambler.ru)

Of course, Rambler is not what it used to be, as many will say, and provides very little traffic. And anyway, why neglect them? The site indexing process in Rambler is the longest among other domestic search engines. And his adurilka has not been working for a long time, it was at: robot.rambler.ru/cgi-bin/addsite.cgi

He has been using the Yandex search database for a long time. Therefore, to get into the Rambler.ru index it is enough to add to Yandex.

Site indexing in Mail.ru (Mail)

The search engine Mail.ru also has a webmaster’s account. Adding a site for indexing in Mail.ru occurs through the addurl form, which is located at: go.mail.ru/addurl

Moreover, like Google, in order to apply for indexing, you need to create your own account and log into it. It won't work out otherwise. Mile has recently been trying to develop his own tools for webmasters.

Above we looked at the main domestic search engines in which we would like to index our site. I will give the following PS more for your general SEO erudition than for specific actions.

Search engine Aport.ru (Aport)

Aport.ru was once a search engine, with its own index database and adurilka. Now it has been turned into a product search engine in which you can compare prices for goods and services.

Search engine Nigma.ru (Nigma)

Nigma.ru is our Russian intelligent search engine. The total volume of its traffic is about three million requests per day. Obviously, traffic from Nygma should not be neglected. You can add your site for indexing in Nigma on the page nigma.ru/index_menu.php?menu_element=add_site.

Media navigator Tagoo.ru

The Tagoo.ru system is a media search engine that searches for media data. This includes music content, videos and programs. In order for your site to be indexed by the Tagoo system, you need to use the add form: tagoo.ru/ru/webmaster.php?mode=add_site.

Search engine Turtle.ru (Turtle)

The international search engine Turtle (Turtle) searches across the CIS countries in any language. For indexing, resources located in such domain zones as: ru, su, ua, am, az, ge, by, kz, kg, uz, md are accepted. To add a site for indexing in Turtle, you need to use Aduril: http://www.turtle.ru/add.html. It is advisable to wait for the message that your site has been accepted. It may not be added, but you won’t even know.

Foreign search engines

If the above were domestic search engines, then below there will be a list of foreign search engines.

Search engine Yahoo.com (Yahoo)

Search engine Bing.com (Bing)

The Bing search engine is owned by Microsoft and was created to replace Live Search. Microsoft hopes that the new brainchild will be much more popular than its predecessor. If you want your Bing.com site to be indexed, you can do so at http://www.bing.com/toolbox/submit-site-url.

Ukrainian search engines

And in conclusion of my review, I will cite two popular search engines in Ukraine.

Ukrainian search engine Meta.ua (Meta)

By and large, if your resource is good, well-made, then there should be no problems with its indexing. If the site, although not 100%, meets the requirements of search engines - “for people”, then they will be happy to look at you and index everything new that will be added.

But be that as it may, the first step in promoting a site is to add it to the PS index. Until the resource is indexed, by and large there is nothing to promote, because search engines will not know about it at all. Therefore, in this article I will look at what site indexing is in Yandex and how to submit a resource for indexing. I’ll also tell you how to check whether a site or a separate page is included in the Yandex index and what to do to speed up indexing by Yandex.

Indexing a site in Yandex is the robots crawling the yandex search engine of your site and adding all open pages to the database. The Russian search engine spider adds data about the site to the database: its pages, pictures, videos, documents that are searchable. Also, the search bot is engaged in indexing links and other elements that are not hidden by special tags and files.

The main ways to index a resource:

    Forced - you must submit the site for indexing to Yandex through a special form.

    Natural - the search spider manages to independently find your site by moving from external resources that link to the website.

The time it takes to index a site in Yandex is different for everyone and can range from a couple of hours to several weeks.

This depends on many factors: what values ​​are in Sitemap.xml, how often the resource is filled, how often mentions of the site appear on other resources. The indexing process is cyclical, so the robot will come to you at (almost) equal intervals of time. But with what frequency depends on the factors mentioned above and the specific robot.

The spider can index the entire website (if it is small) or a separate section (this applies to online stores or media). On frequently updated resources, such as media and information portals, there live so-called fast robots for quick site indexing in Yandex.

Sometimes technical problems (or problems with the server) may arise on the project; in this case, Yandex indexing of the site will not take place, which is why the search engine may resort to the following scenario:

  • immediately throw out unindexed pages from the database;
  • re-index the resource after a certain time;
  • set pages that were not indexed to be excluded from the database, and if it does not find them during re-indexing, it will be thrown out of the index.

How to speed up site indexing in Yandex

How to speed up indexing in Yandex is a common question on various webmaster forums. In fact, the life of the entire site depends on indexing: the position of the resource in the PS, the number of clients from them, the popularity of the project, profit, in the end.

I have prepared 10 methods that I hope will be useful to you. The first five are standard for constant indexing of a resource, and the next five will help you speed up the indexing of your site in Yandex:

    bookmarking services;

    RSS feed – will ensure the broadcast of new materials from your resource to subscribers’ emails and RSS directories;

    link exchanges - will ensure a stable increase in dofollow links from quality donors, if they are selected correctly (how to select correctly);

    – if you have not yet registered your site in directories, then I advise you to do so. Many people say that directories have died a long time ago or that registering in them will kill a site - this is not true. More precisely, it’s not the complete truth, if you register in all the directories in a row, then indeed your resource will only suffer from this. But with the correct selection of trust and good catalogs, the effect will undoubtedly be.

Checking site indexing in Yandex

  • The site and url operators. If you want to check the indexing of a site in Yandex, you can use standard search engine operators ..biz. (Naturally, instead of my domain, yours)

  • RDS bar. I consider it the best and fastest way to check the indexing of a page in Yandex. This plugin can be installed on all popular browsers and will immediately provide detailed information about the number of site pages in the index and the presence of specific material in it. With this extension, you will not waste time manually entering URLs in services or searches. In general, I recommend it, the RDS bar is extremely convenient:
  • Service Serphant. A multifunctional resource with which you can analyze a site: assessing the effectiveness and monitoring of sites, analyzing competitors’ pages, checking positions and site indexing. You can check page indexing for free using this link: https://serphunt.ru/indexing/. Thanks to batch checking (up to 50 addresses) and high reliability of the results, this service is one of the three best in my opinion.

  • XSEO service. A set of tools for webmasters, in XSEO.in you can look at the site indexing in Yandex. Also get a lot of additional useful information about your resource:

  • PR-CY and CY-PR services. A couple more services that will provide you with information about the total number of indexed pages:

  • Sitereport service. An excellent service that will point out all your mistakes in working on the site. It also has a section “Indexation”, where information will be presented for each page of the site, indicating whether it is indexed or not in the search engines Yandex and Google. Therefore, I recommend using this resource to detect problems on the site and check Yandex mass indexing:

Traffic from search engines is the most desirable for any resource. But in order for people to find a site in search results, it must first be included in the index. Some webmasters do not pay enough attention to this, hoping that search engines themselves will find their site or its new pages. Unfortunately, you can wait a very long time.

The indexing of the site in Yandex is especially different here. If Google grabs a page literally in a matter of days, then the main search engine of the Runet without outside help may not pay attention to the site for months.

But we can fix everything. If you know how to act, Yandex can index pages within a few minutes after they appear. And the faster your entries appear in the database of this search engine, the faster their promotion will begin - the easier it will be to get visitors and gain popularity on the Internet.

How quickly does a site get indexed in Yandex?

All resources can be divided into several groups based on the speed of indexing their new pages:

  • 1. Fast indexed sites. These primarily include news portals and well-known forums. New entries literally end up in the index in a few minutes, which is quite logical - news remains news for a very short time. Also, a quick entry into the Yandex database is ensured by constantly updated trust resources. They have already won his trust with the quality of their content and age.
  • 2. Well-indexed sites. These are resources that have accustomed the search engine to regular updates. They are not necessarily trusted or old, they simply constantly delight both users and search engines with new content. Yandex treats such sites well and pages are out of the index for no more than a few days.
  • 3. Sites that are difficult to index. This applies to new resources, especially if webmasters do not take any action to popularize them. In addition, sites that are not updated for a long time are poorly indexed. If you don’t add new entries for several months, Yandex stops visiting the site - it will return when it pleases, you can wait for a very long time.

This is the classification. It is clear that it is conditional, based on the observations of webmasters and optimizers. Yandex will never tell anyone on what principle it indexes sites and how it determines who to visit and who is not worthy of its attention. But very useful conclusions can be drawn from this division.

Which sites are quickly indexed by Yandex:

  • frequently updated: the more often records appear, the faster they get into the index;
  • trust: age, TIC indicator, traffic, number of pages - the more, the higher the trust of Yandex in the site, the more often the search engine robot comes to it.

Of course, indexing can be influenced by other factors, such as the subject of the site, external links, and behavioral factors. Therefore, no one will undertake to guarantee the quick appearance of pages in search results, even for trust sites. Just as it is not necessary that a newly created website will wait months for indexing in Yandex. Moreover, we ourselves can have some influence on this process.

How to index a site in Yandex faster?

To speed up indexing in Yandex, we recommend buying tweets on the twitterstock exchange; you can buy tweets from trust accounts for 3 rubles. The fast robot arrives very quickly

First of all, you need to add your site to the Yandex webmaster panel (Yandex.Webmaster). This way you will notify the search engine about the availability of a new resource. The process is simple, but very useful, especially since it will give you the opportunity to monitor how your site is perceived by Yandex.

In addition, you can add each new page to the addurilka (in the same Yandex.Webmaster, the “Report a new site” item). Although this is not a guarantee of indexing, it will not be superfluous.

  • create a site map: the best two types are HTML for users and XML for search robots. Thanks to this page, Yandex will be able to immediately see the structure of the site and all its records, which will allow it to quickly index everything;
  • do internal linking: links from one page to another lead the search engine through the entire resource, and it includes the found records in the index;
  • regularly update the site: get Yandex accustomed to the fact that you constantly have new entries - and it will visit the resource more often.

To make indexing in Yandex faster, you need to attract a search robot to your site from other resources through external links:

  • 1. Announcement on social networks and blog platforms. This is perhaps the most effective way to speed up indexing in Yandex today. For example, links from Twitter can lead a search robot to your site in a few minutes. Yandex loves its brainchild - Ya.ru blogs. VKontakte, LiveInternet, LiveJournal are also popular tools for quick indexing. It is clear that a lot depends on the promotion of the account or blog - if it is outside the Yandex index, then you should not expect that a link from it will help.
  • 2. Blog comments and forum posts. It is especially effective if these resources are well indexed - the robot will immediately follow the link to your site. This method is best suited when you need to add a new site to the Yandex index, because this process is quite labor-intensive and it is irrational to use it to speed up the indexing of each new entry.
  • 3. RSS broadcast in specialized catalogs. Yandex visits some RSS directories quite often, so if you add your site there and they broadcast new entries, this will help speed up indexing.
  • 4. Adding to social bookmarks. MyPlace, BeaverDobr, Delicious, Toodoo are the most famous of them. Search engines monitor updates in these services, so your pages have every chance of quickly getting into the index.
  • 5. Adding sites to white directories. It’s especially good if your resource manages to get into Yandex.Catalog. This will already indicate the search engine’s trust in it, and therefore good indexing.
  • 6. Buying links. The most effective links will be from trust resources, as well as from news portals - they themselves are quickly indexed, and will provide this to your site. You can also order links from blogs, for example, Ya.ru or LiveJournal - they are inexpensive and work great.

Of course, this list can be continued, supplemented and updated. Especially when you consider that search engines are constantly changing and improving their algorithms, which means that any of the methods may become irrelevant or, on the contrary, contribute more to the rapid indexing of the site in Yandex. But this is a good start for website promotion - using the methods listed above, you can show your resource to search engines, and at the same time improve it and increase your link mass.

Website indexing in search engines is important for every webmaster. After all, for high-quality promotion of a project, you should monitor its indexing. I will describe the process of checking indexing in Yandex.

Indexing in Yandex

The Yandex robot scans sites day after day in search of something “tasty.” Collects in the top results those sites and pages that, in his opinion, most deserve it. Well, or Yandex just wanted it that way, who knows :)

We, as real webmasters, will adhere to the theory that the better the site is made, the higher its position and the more traffic.

There are several ways to check site indexing in Yandex:

  • using Yandex Webmaster;
  • using search engine operators;
  • using extensions and plugins;
  • using online services.

Indexing website pages in Yandex Webmaster

To understand what the search engine dug up on our site, you need to go to our beloved Yandex Webmaster in the “Indexing” section.

Bypass statistics in Yandex Webmaster

First, let’s go to the “Bypass Statistics” item. This section allows you to find out which pages of your site the robot crawls. You can identify addresses that the robot was unable to load due to the unavailability of the server on which the site is located, or due to errors in the content of the pages themselves.

The section contains information about the pages:

  • new – pages that recently appeared on the site or the robot has just crawled them;
  • changed - pages that the Yandex search engine previously saw, but they have changed;
  • crawl history - the number of pages that Yandex crawled, taking into account the server response code (200, 301, 404 and others).

The graph shows new (green) and changed (blue) pages.

And this is a graph of the crawl history.

This item displays the pages that Yandex found.

N/a – URL is not known to the robot, i.e. the robot had never met her before.

What conclusions can be drawn from the screenshot:

  1. Yandex did not find the address /xenforo/xenforostyles/, which, in fact, is logical, because this page no longer exists.
  2. Yandex found the address /bystrye-ssylki-v-yandex-webmaster/, which is also quite logical, because new page.

So, in my case, Yandex Webmaster reflects what I expected to see: what is not needed, Yandex has removed, and what is needed, Yandex has added. This means that everything is fine with bypass, there are no blockages.

Pages in search

Search results are constantly changing - new sites are added, old ones are deleted, positions in search results are adjusted, and so on.

You can use the information in the “Pages in Search” section:

  • to track changes in the number of pages in Yandex;
  • to track added and excluded pages;
  • to find out the reasons for excluding a site from search results;
  • to obtain information about the date the search engine visited the site;
  • to receive information about changes in search results.

This section is needed to check the indexing of pages. Here Yandex Webmaster shows pages added to search results. If all your pages are added to the section (a new one will be added within a week), then everything is in order with the pages.

Checking the number of pages in the Yandex index using operators

In addition to Yandex Webmaster, you can check the indexing of a page using operators directly in the search itself.

We will use two operators:

  • “site” – search across all subdomains and pages of the specified site;
  • “host” – search for pages hosted on a given host.

Let's use the "site" operator. Note that there is no space between the operator and the site. 18 pages are in Yandex search.

Let's use the "host" operator. 19 pages indexed by Yandex.

Checking indexing using plugins and extensions

Check site indexing using services

There are a lot of such services. I'll show you two.

Serphunt

Serphunt is an online service for website analysis. They have a useful tool for checking page indexing.

You can simultaneously check up to 100 website pages using two search engines - Yandex and Google.

Click “Start scanning” and after a few seconds we get the result:


2024 gtavrl.ru.