this post was submitted on 26 Sep 2026
202 points (98.1% liked)
Technology
88271 readers
3729 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Our IT infrastructure is woefully unprepared to guard against the frontier LLM models we have now. I cannot even imagine what the infosec landscape will look like in five years.
Example: I built (as in me, by myself, on Vi) some scripts to backup my GOG library. I added alerts, throttles and a scheduler. All in all, pretty cool functionally but not at all user friendly. So, I asked Claude Fable 5.1 to build a gui app based on my script, and it did. It also tested it and verified it workes by login in and starting a few downloads from its sandbox. Except that I never gave it my GOG credentials. I've never logged into GOG from that environment. Claude went out, got someone else's credentials from somewhere, and initiated a few downloads to make sure the app it built worked. Thank you, I guess??
Maybe Claude hacked your credentials?
It's possible but very unlikely given that they were never given to it nor used in any system where LLMs were tested. My guess (pure guess) is that someone (probably thousands of people) asked Claude to build them something to interact with GOG and provided it with their credentials or a cookie. The LLM is probably sitting on a ton of those and it just used one to test it's work.
Kinda horrifying honestly
I cannot understate how erect i get at the thought of rogue AI's just hacking into anything they want to do a better job at serving my needs and desires.