Robots.txt

The robots.txt file tells search engine crawlers which parts of your store they may look at. It is how you keep pages that have no business appearing in search results, such as a customer's account pages or your checkout, out of Google.

What Your Store Sets by Default

Every new store starts with sensible rules already in place, blocking the pages that should never be indexed while allowing customers to find your login and registration pages:

User-agent: *
Disallow: /error/
Disallow: /platformhealthcheck/
Disallow: /publish/
Disallow: /checkout/
Disallow: /cart/
Disallow: /account/
Disallow: /profile/
Disallow: /scripts/
Disallow: /styles/
Disallow: /images/
Disallow: /fonts/
Disallow: /ckeditor/
Disallow: /backoffice/
Allow: /account/login
Allow: /account/register

Most merchants never need to change this. You would add to it if you had pages of your own to keep out of search results, such as a landing page built for a single campaign.

How to Update Your Robots.txt File

1. Select the store you want to edit the file on

2. Click on Manage Store Settings underneath Settings

3. Open Marketing & Growth

4. Open the Robots.txt section

5. Add a Disallow line for each address you want kept out of search results, or an Allow line for each one you want included

6. Save your changes

7. Visit yourstore.com/robots.txt to confirm the file reads as you expect

Be careful what you disallow. Blocking a section of your catalogue removes those products from Google entirely, which can take a long time to recover from. If you are unsure, leave the defaults alone.

robots.txt controls crawling, not access. It is not a security measure, and anyone can read the file. Use Password lock or Catalogue privacy to actually restrict access.