Block AI bots from crawling composr.app (take 2)
Posted
#8681
(In Topic #2169)
A vote for "yay" on this measure will do the following:
- Activate a block on the Cloudflare level to block AI crawlers from crawling composr.app
- Once / if implemented, activate an AI bot block on the Composr CMS level for composr.app
- Once / if supported, activate an AI bot block on the server level for composr.app
- Define in robots.txt that no AI bots may crawl composr.app
A vote for "nay" on this measure will do the following:
- Continue to allow AI bots to crawl composr.app as they do now
Note: This only affects the Composr homesite. It does not affect the Composr repositories on GitHub or GitLab. And it does not affect other Composr sites (although we have a feature request on the tracker to add an AI blocking setting in Composr).
Potential arguments in favor of blocking AI bots:
- Privacy; bots train off of the content they scrape, which may include content posted by members (so long as it is accessible by guests). This includes profiles and forum posts.
- Performance; a large majority of visitors to composr.app are AI bots. Banning them might increase performance slightly.
Potential arguments against blocking AI bots:
- Visibility: Allowing AI bots to scrape the Composr homesite means more people could potentially learn about Composr if the AI brings it up. And this factor becomes more and more relevant over time as people change to using AI for searching.
- Against the nature of F.O.S.S: Composr is not proprietary, and perhaps neither should we treat the content of the homesite (e.g. the site and its content is open to all visitors and bots who respectfully use it).
0 guests and 0 members have recently viewed this.

