Troubleshooting the site's position in the search

This section helps you independently determine the reason why your site changed its position or isn't displayed in Yandex search results anymore.

Step 1. Check to see which response was received by the robot before the site status in search was changed.

1. Go to the Crawl statistics page in Yandex.Webmaster.
2. In the list of changes, find the URL of the page that changed its position or disappeared from the search. Examples:

   

   In the example, the https://example.com/comment/articles/ks/ page changed its status and became unavailable for the robot due to the indexing prohibition. If the page isn't available to the robot, it is excluded from the search. Therefore, the page stops being displayed in search results for queries or the site's position is lowered.

   The page may become inaccessible to the robot for several reasons:

   • The page is closed from indexing.
   • The HTTP response code is other than 200 OK.

   You can close a page from indexing using one of the methods:

   • Using the Disallow directive in the robots.txt file. To remove the ban, make changes to robots.txt and check the correctness of the entries in the file using the Robots.txt analysis tool.
   • Using the noindex or none directive in the robots metatag in the source code of the site page or in the X-Robots-Tag HTTP header. Learn more. To remove the ban, make changes to the page code or change the HTTP header.

   The HTTP response code can change from 200 OK to 404 or 500, as well as to other codes 4XX, 5XX or 3XX.

Step 2. Check what response the robot gets after making changes to the site.

1. In Yandex.Webmaster, go to the Server response check page.
2. In the field, enter the URL of the page whose response you checked in step 1.

The server response matches the 200 OK HTTP code.The server response is other than 200 OK.

If the server response matches the 200 OK HTTP code, it means that the page will appear in the search results within two weeks.

See the recommendations depending on the HTTP response code, as well as explanations of other messages when checking the server response.

After fixing the error send the page for re-indexing.

Recommendations didn't help

The reasons for the site's low position remain unclearThe page is accessible to the robot and sent for reindexing, but does not appear in the search

If the page doesn't appear in the search and the site's position continues to go down, check out other reasons for position changes and excluding pages from search.