Filter URL Crawling Issue (Mitigation Applied)

Known Issue:

Automated bots have been crawling filter URLs, especially where multi-select filters are in use. This has resulted in an excessive number of requests and unnecessary server load.

What we’ve done:

We’ve replaced the href attributes on filter <a> tags with a custom data-filter attribute.

• A jQuery event now listens for user interaction and manually redirects to the selected filter.

• This approach reduces the exposure of filter URLs to basic web scrapers and mitigates request overload.

Note:

This does not prevent advanced bots that simulate a browser and interact with the page (e.g. headless browsers), but it significantly reduces the impact of standard link crawlers.