this post was submitted on 18 Aug 2025
1127 points (99.0% liked)

Technology

74265 readers
4189 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago
MODERATORS
L3s@lemmy.world
enu@lemmy.world
technopagan@lemmy.world
L4s@lemmy.world
L3s@hackingne.ws
L4s@hackingne.ws
1127
Codeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges. (i.imgur.com)
submitted 2 days ago by Pro@programming.dev to c/technology@lemmy.world
231 comments fedilink hide all child comments
 

cross-posted from: https://programming.dev/post/35852706

Source.

you are viewing a single comment's thread
view the rest of the comments
[โ€“] interdimensionalmeme@lemmy.ml 2 points 1 day ago (1 children)

Do you know how trivial it is to screenshot a website and push it through an OCR ?

This battle is completely unwinnable, just put a full dumb.zip of the public data on the front door and nobody will waste their time with a scrapper.

Is the data public or is it not ? At this point all that you're doing anyway is entrench the power of openai, google and facebook while starving any possible alternative.

Anubis will never work, no version of anubis will ever be anything more than a temporary speed bump.

[โ€“] mfed1122@discuss.tchncs.de 1 points 1 day ago

Yeah, I do. I'm just grasping at straws. But you're right, the only real solution, ironically, is to have non-open sites where you need accounts to view content. I wouldn't mind seeing some private phpbb forums though.