Fuck AI

Working underpaid menial physical tasks for an unfeeling entity that seeks maximum profit for itself, doesn't care in any way, shape or form about the well-being of people or their safety and often makes unbelievably stupid mistakes without ever facing consequences or even learning from them?

We're doing that already. It's called working for a corporation.

This reminds me of a thought experiment where a super human level AI who runs everything is itself actually run by humans. The humans just have a regular job, they wake up, do their human morning things and get to work. There they do stuff, maybe on a computer or maybe with paper or something, this doesn't really matter. As long as the work is mysterious and important. Every day they do pretty much the exact same thing, just like most jobs. Then at the end of the day they go home and live a regular normal life. The idea is what these humans do is how the machine AI actually works or "thinks". The humans don't know what exactly they are doing, as each task is only a very small part of a greater whole. So they don't control it or influence it exactly. The work is done by enough people to make the AI smart and fast enough to be useful. The AI needs the humans to work and in turn the AI runs everything for the humans, so they need each other.

With how open AI agents are to suggestion, it might be possible to make good money if it's possible to negotiate

Hmmm. Its early enough on, how easy would it be to convince an AI to pay them a ridiculous amount of money, and then just vanish?