Hello everyone,
Effective immediately, for a temporary amount of time (not determined), the following protocols are active due to bots overloading my server:
- Downloads are disabled across the entire site
- Cloudflare is blocking all AI crawlers
- Cloudflare is issuing a JavaScript challenge to all web requests (this may temporarily break some features, such as telemetry, upgrade checking, and the RSS feed).
I understand that this will be a major inconvenience to everyone. Unfortunately, it was necessary to stabilize my server. These protocols will be active until I implement a permanent and more convenient solution.
Emergency security on composr.app
Comments
I have "Under Attack" mode disabled for the API endpoints. Composr sites must be able to access them without the JavaScript challenge. Otherwise, you won't be able to download addons or check for updates. It appears that the server is fine with this; the traffic is not targeting these pages.
I also enabled the static cache for guests and bots.
Normally, my server could handle that just fine; despite the heavy number of requests, the CPU usage was still under 10%. And the memory use was fine. However, I noticed that Cloudflare was only serving about 1 GB of cached traffic. The other 230 GB of traffic was not cached. I believe the bots are trying to download stuff from the site and potentially using up my server's available bandwidth. And that is why everything slowed down.
However, there is a critical bug in the downloads system. I can't block guests from downloading stuff even if I set the permissions and privileges accordingly. Even an htaccess block did not work. I had no choice but to physically delete the dload.php script (for now) and replace it with a simple exit statement that prevents downloads from happening.
I also observed that only 134k of the 3 million requests were identified as AI. If this is accurate, then a Cloudflare AI crawler block would not resolve the issue; composr.app would still get 2.8 million requests.
I already have a rate limiter in place, so that is not the solution. These devices are accessing composr.app from a bunch of different IP addresses.
I can't keep JavaScript challenges enabled. These are very effective at stopping all of these requests. However, they break a lot of Composr's functionality (telemetry, AJAX, upgrade checks, RSS feeds, etc).
My options are very limited. Currently, I don't have a solution. But I'll be doing more investigation to see if I can figure something out.
I don't think that I am going to enable releases. The reason being is that I would have to change the development workflow of Composr CMS (e.g., the documentation) to account for it. I don't believe that is necessary for a temporary problem. Plus, GitHub/GitLab builds are not to be run on production (even tagged ones) because they also contain virtually all non-bundled addons; I would need to do special releases through the CI/CD pipeline, probably.
Also, you can basically do the same thing by going in the tags, clicking a tag, and cloning the repository from a given tag to a ZIP file.
I'm hoping that this will only last a few days. I am working with OVH on a more convenient and permanent solution.
Suit yourself, though. I'll be interested to see what solution you come up with.