Всем привет!
Постоянно отключается сайт из-за превышения лимитов. Нагрузку создают яндекс и гугл боты.
Помогите настроить robots.txt, чтобы снизить нагрузку
Вот тот, который используется сейчас:
User-agent: *
Disallow: /*index.php*
Disallow: /*route=account/
Disallow: /*route=affiliate/
Disallow: /*route=checkout/
Disallow: /*route=product/search
Disallow: /index.php?route=product/product*&manufacturer_id=
Disallow: /admin
Disallow: /catalog
Disallow: /download
Disallow: /system
Disallow: /*?sort=
Disallow: /*&sort=
Disallow: /*?order=
Disallow: /*&order=
Disallow: /*?limit=
Disallow: /*&limit=
Disallow: /*?filter_name=
Disallow: /*&filter_name=
Disallow: /*?filter_sub_category=
Disallow: /*&filter_sub_category=
Disallow: /*?filter_description=
Disallow: /*&filter_description=
Disallow: /*?tracking=
Disallow: /*&tracking=
Disallow: /m/
Disallow: /mobile/
Allow: /catalog/*.css
Allow: /catalog/*.js
Allow: /catalog/*.png
Allow: /catalog/*.gif
Crawl-delay: 15
User-agent: Yandex
Disallow: /*index.php*
Disallow: /*route=account/
Disallow: /*route=affiliate/
Disallow: /*route=checkout/
Disallow: /*route=product/search
Disallow: /index.php?route=product/product*&manufacturer_id=
Disallow: /admin
Disallow: /catalog
Disallow: /download
Disallow: /system
Disallow: /*?sort=
Disallow: /*&sort=
Disallow: /*?order=
Disallow: /*&order=
Disallow: /*?limit=
Disallow: /*&limit=
Disallow: /*?filter_name=
Disallow: /*&filter_name=
Disallow: /*?filter_sub_category=
Disallow: /*&filter_sub_category=
Disallow: /*?filter_description=
Disallow: /*&filter_description=
Disallow: /m/
Disallow: /mobile/
Allow: /export/yandex_yml.xml
Crawl-delay: 10
Clean-param: tracking
Host: https://sfera.by
Sitemap: https://sfera.by/index.php?route=feed/fast_sitemap