Fuck AI

We are indeed living inside the stupidest version of Cyberpunk. Time to start building AI countermeasures.

I think we have more to fear from using AI to generate permutations of existing attacks, in a way that evades detection of known behaviors, malware hashes, and so on. Also, having a command & control (C2) style attack dynamically evolve with help from AI, based on intel from the target? That's kind of novel and scary in its own way.

Meanwhile hacking in and running a rogue AI client on a target system in an enterprise setting... well, you'd have to be blind to not notice all the back-and-forth token and response traffic. It would be the fattest, nosiest, C2-style attack and probably easy to detect with conventional means.

Otherwise, OP and this copypasta is correct to be concerned. It's not like the typical home user is watching bytes sent/recv on their home router. This could manifest as a very potent botnet problem.