Technology

But it isn't encoding knowledge, it's encoding word correlations.

I'm saying that humans do this a lot, too. Qualitatively, it's different, in that this particular batch of frontier LLMs will get things wrong in ways that most human brains wouldn't, but as a category of error it's not unique to LLMs.

I know a ton of facts that I learned only through reading, and have no actual firsthand knowledge/experience or ability to test it: Jupiter is larger than Saturn, the atmosphere during the Carboniferous period was high in oxygen, cigarettes cause cancer, Thomas Jefferson owned slaves, the capital of Norway is Oslo. At best, I can cross reference other sources and see that things are consistent with each other. Is my belief in those facts "knowledge," or is it merely recognizing from my training data that those particular words can validly be presented in that order?

If you ask average people on the street whether FAT32 is a good filesystem for a 64GB removable drive, most of them won't know, but there are a handful of bullshitters who might confidently parrot back things they can Google but not understand. That's part of the human condition, too.

I'm by no means an AI booster/enthusiast. I suspect LLMs/transformers are actually a dead end, and expect the upcoming crash to be economically and financially devastating to the tech and financial sectors. But I also have a pretty dim view of human intelligence, too, and see way too many parallels in LLMs as bullshit artists to humans as bullshit artists, too.