Issue
By default, when a search engine spider visits your web site it follows all variable links, queries, search boxes and dynamic pages on your web site. This can end up in an endless loop causing excessive load on your web site and generating high traffic.
Solution
The problem can be solved by uploading a robots.txt file to your web site which will prevent the search engines from following any URLs that begin with /? (these are the common dynamic page references).
Insert this text into your robots.txt file:
User-agent: * Allow: /*?$ Disallow: /*?
Get in touch
For any additional help, give us a call on 0800 5000 24 (8AM to 6PM, Monday to Friday). Requests received outside these hours will be handled on the next business day.