Man, AI agents are remarkably bad at "self-awareness" like this, I've used it to configure some networking on a Raspberry Pi, and found myself reminding it frequently, "hey buddy, maybe don't lock us out of connecting to this thing over the network, I really don't want to have to wipe the thing because it's running a headless OS".

It's a perfect example of the kind of thing that "walk or drive to wash your car?" captures. I need you to realize some non-explicit context and make some basic logical inferences before you can be even remotely trusted to do anything important without very close expert supervision, a degree of supervision that almost makes it totally worthless for that kind of task because the expert could just do it instead.

[–] Earthman_Jim@lemmy.zip 22 points 13 hours ago (3 children)

Hey, maybe if we're lucky, Claude will accidentally lock the world out of using nukes forever.

[–] TwilitSky@lemmy.world 5 points 6 hours ago (2 children)

Or, more likely, Claude will launch them.

[–] DeadDigger@lemmy.zip 7 points 5 hours ago

Would be funny if it also forgets to open the hatches

[–] fartographer@lemmy.world 2 points 5 hours ago

Not if I launch them first! Where's the 9v battery for the rocket engine igniter?

load more comments (10 replies)