#6238 - Add config option to block AI bots

  • By
  • Added
  • 4 views
Identifier #6238
Issue type Feature request or suggestion
Title Add config option to block AI bots
Status Open
Tags

Roadmap: v11 (custom)

Type: Anti-big-tech (custom)

Handling member Deleted
Addon core
Description Cloudflare today announced by default (for new sites) AI bots would be blocked, and that those using Cloudflare can control which bots are allowed (and even require them to pay).

Add a similar feature in Composr CMS; we already have means to check for bots, so also check to see if a particular bot is AI, and throw a 403 if the config option is enabled.
Steps to reproduce

Additional information This will allow sites not using Cloudflare (reasonable, as Cloudflare is considered "big tech") to still block AI crawlers from crawling a Composr site (also reasonable, because again "big tech" for a lot of these crawlers, and for IP protection).

robots.txt is not effective enough; AI bots are well known to ignore this.

This will require Composr to keep an updated list of UAs identifying AI bots. But we might be able to do that in the testing platform and build tools.
Funded? No
The system will post a comment when this issue is modified (e.g., status changes). To be notified of this, click "Enable comment notifications".

Rating

Unrated