> For the complete documentation index, see [llms.txt](https://support.storefront.co.za/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://support.storefront.co.za/features/stores/store-settings/marketing-and-growth/robots.txt.md).

# Robots.txt

The robots.txt file tells search engine crawlers which parts of your store they may look at. It is how you keep pages that have no business appearing in search results, such as a customer's account pages or your checkout, out of Google.

### What Your Store Sets by Default

Every new store starts with sensible rules already in place, blocking the pages that should never be indexed while allowing customers to find your login and registration pages:

```
User-agent: *
Disallow: /error/
Disallow: /platformhealthcheck/
Disallow: /publish/
Disallow: /checkout/
Disallow: /cart/
Disallow: /account/
Disallow: /profile/
Disallow: /scripts/
Disallow: /styles/
Disallow: /images/
Disallow: /fonts/
Disallow: /ckeditor/
Disallow: /backoffice/
Allow: /account/login
Allow: /account/register
```

Most merchants never need to change this. You would add to it if you had pages of your own to keep out of search results, such as a landing page built for a single campaign.

### How to Update Your Robots.txt File

1. Select the store you want to edit the file on
2. Click on Manage Store Settings underneath Settings
3. Open Marketing & Growth
4. Open the Robots.txt section
5. Add a Disallow line for each address you want kept out of search results, or an Allow line for each one you want included
6. Save your changes
7. Visit yourstore.com/robots.txt to confirm the file reads as you expect

{% hint style="warning" %}
Be careful what you disallow. Blocking a section of your catalogue removes those products from Google entirely, which can take a long time to recover from. If you are unsure, leave the defaults alone.
{% endhint %}

robots.txt controls crawling, not access. It is not a security measure, and anyone can read the file. Use [Password lock](/features/stores/store-settings/security-and-privacy/password-lock.md) or [Catalogue privacy](/features/stores/store-settings/security-and-privacy/catalogue-privacy.md) to actually restrict access.