878
Codeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges.
(i.imgur.com)
cross-posted from: https://programming.dev/post/35852706
If this isn't fertile grounds for a massive class-action lawsuit, I don't know what would be.
whos the defendent, specifically?
No, that's a good point. We all bloody well know there isn't a single provider of LLM's that aren't sucking the entire Internet dry while gleefully ignoring robots.txt and expecting everybody else to pay the bill on their behalf, but the AI providers are getting really good at using other people IPs both to mask their identity and to evade blacklists, which is yet another abusive behavior.
But that's beside your point. So forget the class-action lawsuit in favor of the relevant Ombudsman.
Either way, this cannot go on. Donation-driven open source projects are being driven into the ground by exploding bandwidth and hosting costs, people are being forced to deploy tools like Anubis that eats additional resources - including the resources of every legitimate user. The cumulative damage this is doing is no joke.