Make rate limiting of bot/crawler requests optional

Vojislav Jovanovicrequested to merge
issue/crawler_rate_limit-3528552:3528552-make-bot-limiting-optional into 3.x

Closes #3528552

Merge request reports