Idk, it might seem obvious, but I noticed something interesting when I was thinking about the "it's not x, it's y". Like a friend told me some time ago, "it looks like infomercials, it's like it's trying to sell you something".
When someone says something like that, what comes to mind? My first thought is always "but I didn't think, say or reason that it would be x, nor you". Why cite a different thing before the real argument before the argument then?
To me this just seems some way to add "weight" to an argument, like a high school essay, taking attention out of the real argument, stylistically giving the illusion of strength. It looks like and acts like the high school essay "padding" that we all know and used (or at least people used at my school) "that guy was not important, he was essential", "the trouser gave the world one of the most important and necessary inventions", "the onion is one of the most notorious newspapers in existence, being cited in nature and by the bible itself".
I'm giving some bullshit texts with no meaning examples, but just substituting this with the "guy" we needed to talk about in 4 pages would add shit up and give a little bit of the semblance of weight to the text.
Now looking at any Claude or ChatGPT text, this is exactly the thing that's happening, a high schooler is padding their essay with things saying that "yeah, my argument is very strong, because it's not x, it's y", and people are falling for it. Every topic on the signs of LLM wikipedia page is this.
Idk, but this seems to be the same case of the "warmth" that bootlicking LLMs have, the person looking at the screen gets manipulated (by the companies that train the LLMs to act this way) and likes it, even if not directly or knowingly.
Imagine what could be done with this kind of "fallacies"? Just changing peoples mind on topics they want to be changed? Instead of refusing to say anything, the LLM trainers could just train certain argument tactics and try to change peoples mind, if people is falling for the bullshit padding, why not tactically engineer peoples opinions?
If it's not doing this already.
That "it’s not x, it’s y" - thing is really weird. I think it's even discouraged to talk like that for humans, as it frequently gets people confused. And if the audience is not paying attention 100% they might as well memorize X instead of Y. It's also mostly unnecessary. If something is Y, just say so. (There's a few limited use-cases. Like when debunking proper myths. Or in an Ikea instruction leaflet with the wrong screw crossed out and the correct one depicted next to it. And other textbook examples like that. But it shouldn't be a normal speech pattern, by any means.)
I had ChatGPT do other weird things. I tried writing Python code with ChatGPT. And on more than one occasion it gave me a full screen of text, or computer code... Which I read and copy-pasted... Just to scroll down and find out it goes on to say ChatGPT does NOT recommend doing it that way, and instead do...
And that really made me feel mocked. Why waste 3mins of my life then, by making me read and copy-paste the wrong "solution"? I don't know why that is, though. Maybe it came to some "realization" while writing it, and since it's writing left to right, it can't make corrections. So it'll decide to weasel itself out and portray it as some "lesson" in what not to do.
Feels somehow like the "it’s not x, it’s y". Just with the "not" coming after explaining X. And everything is stretched out over 5 pages.
Or maybe it'a just because they made it prefer longer answers, or give several options/perspectives... And adding wrong ones contributes to length.