#3569 - Blocking nefarious crawlers via .htaccess changes

We do do some of this already, e.g. the rate limiting feature.

So essentially this is addition of a block list in the default configuration.

I probably would solve this via a tutorial to be honest, but there's a lot to read over, certainly could be some good ideas.

I had a similar file years ago which appeared to work well, but this project seems way more thorough and is active so I thought I'd raise the suggestion that it might be worth a look and you'll certainly make more sense of it than me.