---
metadata:
  - name: generator
    content: Diplodoc Platform v5.55.3
alternate:
  - https://yandex.com.tr/support/webmaster/en/robot-workings/user-agent.md
  - https://yandex.com.tr/support/webmaster/tr/robot-workings/user-agent.md
  - href: en/robot-workings/user-agent.md
    type: text/markdown
    title: Markdown version
  - href: ../llms.txt
    type: text/markdown
    title: llms.txt
---
> **Documentation Index:** Fetch the complete configuration index at https://yandex.com.tr/support/webmaster/en/llms.txt


# The User-agent directive

In the `robots.txt` file, the indexing bot checks for records starting with `User-agent:` and looks for either the substring `Yandex` (the case doesn't matter) or `*`. If the `User-agent: Yandex` string is detected, the `User-agent: *` string is ignored. If the `User-agent: Yandex` and `User-agent: *` strings are not found, the robot is considered to have unlimited access.

You can enter separate directives for the Yandex robots: To learn more about how robots account for them, see the [Yandex robots in the server logs](https://yandex.com.tr/support/webmaster/en/robot-workings/check-yandex-robots.md#robot-in-logs) section.

If there are directives for a specific robot, directives `User-agent: Yahoo` and `User-agent: *` aren't used.

Example:

```no-highlight
User-agent: YandexBot # will be used only by the main indexing bot
Disallow: /*id=

User-agent: Yandex # will be used by all Yandex bots
Disallow: /*sid= # except the main indexing bot

User-agent: * # will not be used by Yandex bots
Disallow: /cgi-bin 
```

According to the standard, you should insert a blank line before every User-agent directive. The `#` character designates commentary. Everything following this character, up to the first line break, is disregarded. See details in the [Processing the # symbol](https://yandex.com.tr/support/webmaster/en/robot-workings/allow-disallow.md#hash) section.


{% note info %}

[Yandex robots](https://yandex.com.tr/support/webmaster/en/robot-workings/check-yandex-robots.md) can detect entries that include `User-agent:`.

{% endnote %}

{% note info "" %}

Some Yandex robots may ignore directives in `robots.txt`, including those for `User-agent: Yandex`. To set directives for such robots, add them to `robots.txt` specifying their exact `User-agent`. See details in the [Yandex robots in the server logs](https://yandex.com.tr/support/webmaster/en/robot-workings/check-yandex-robots.md#robot-in-logs) section.

{% endnote %}


<div class="cut-button">

{% cut "Contact support" %}




<div style="padding: 15px;
      margin: 10px 0;
      background: #FFFFFF;
      border-radius: 10px;
      border: 1px solid var(--yc-color-line-generic);">
<iframe style="background: #FFFFFF;"
      height="700"
      width="100%"
      frameborder="0"
      src="https://forms.yandex.com.tr/surveys/2426/?&iframe=1&lang=tr">
</iframe>
</div>


{% endcut %}

</div>

<!-- source: en/_includes/cut-button-style.md -->

<!-- endsource: en/_includes/cut-button-style.md -->

<!-- source: en/_includes/footer.md -->
<div width="100%" align="center" class="main-footer"> 
<br><br><br>
You can also go to 
<br><br>





[![](../_assets/logo/logo-pic.svg)](https://webmaster.yandex.com.tr/) [![](../_assets/logo/logo-en.svg)](https://webmaster.yandex.com.tr/)


</div>


<!-- source: en/_includes/styles/footer-style.md -->

<!-- endsource: en/_includes/styles/footer-style.md -->
<!-- endsource: en/_includes/footer.md -->