532
this post was submitted on 15 Sep 2026
532 points (98.2% liked)
Fuck AI
8229 readers
912 users here now
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
This person very clearly does not understand, even at a basic level, how llms work. They are completely incapable of "learning" outside of training, and training cannot happen while you are using them - in no small part because they require MILLIONS of tokens of data (minimum) to be able to recognize patterns.
Meanwhile that theoretical stupid kid she thinks should be paying her masters to have her teach them can learn instantaneously from even a single datapoint.
She's wrong about the advantages of AI, but she's right about how companies are going to see it. It's true that they'd rather deal with AI than an intern, and it's true then that internships will fall onto the universities to coordinate like rotations as part of an expensive education system.
It's also true that kids are learning how to use AI on their own, and are losing their ability to learn from any datapoints, because why do that if the magic box will give you an answer that sounds correct enough? Ultimately, it doesn't matter how useless AI is if everyone acts as if it's as useful as this lady thinks it is.
This was already a thing when I went to university over a decade ago. Partnerships with local businesses where university students basically do work placements, completing low-stakes corporate projects as assignments.
I know this goes against expectation but it was actually really good. I think the difference was that the business actually cared about us, taught us things, and even extended a job offer to people who did well.
I dunno if that program is still good today, probably not.
Yeah, local students do projects for local manufacturing plants to find efficiency improvements. I have second hand knowledge (girlfriend)
Well, no. Models are not "self-learning". This is unrelated to size of the corpus. In fact, it was proven times and times again, that poisoning a model is not a costly proces.
I'm told that agents these days keep a scratchpad of notes to themselves, so an agent can (kind of) learn about things by making a note about it which it will still have access to later, even outside the formal context window.
It's going to be far more limited than the kind of training you're talking about, but it is something which an individual agent can do
That is sort of true, but it still isn't learning. Basically, it can store information usable by other sessions. But to do so, it just processes that information the same way as a prompt. So less like learning and more like just tacking something on to its context window. So, it never "learns" that information, it just tells itself that thing every time it processes any prompt.
(don't downvote MartianSands' comment - it's a good point. Shame on you)
50 First Prompts
Explain the distinction in more detail, and why it matters.
Say you had a version of chat GPT that hadn't been trained on a corpus that included enough information about the number of Rs the word "strawberry" contains, didn't have workarounds for character counting, and has context sharing.
LLMs, at their core, use models that have computed lexical & semantic similarity to predict words (really they perform contextualized vector transformations, but let'snot get into that). When they "learn", they just do this more effectively using more text or using the same text more efficiently. Next word prediction is not the same thing as learning concepts, like the concept of numbers, non-numerical things being ascribed numerical values (ironically, since LLMs function by turning words into numerical vectors), or counting.
When it responds "Strawberry has 3 Rs" it isn't because it knows what that means or how to gain that knowledge about other words. It is merely parroting back what you've told it because "Strawberry has 3 Rs" in its shared context has very close lexical similarity to your query "How many Rs does Strawberry have?". It also parses Strawberry as ["straw", "berry"] encoded into numerical values representing their relationship in the corpus' vector space (e.g. {[ .420, -.67, .67], [.420, .69, -.69]}) - so unless it has instructions to further break those tokens into characters, it does not have the capability to count. You can actually do some weird math using these vectors and their relationships (their distance apart in the 3D vector space, angles between vectors, etc actually relate to the semantic content of the tokens), but you lose granularity such as the number of letters in a token when you look at words this way.
Learning is more than just computing lexical similarity. Modern LLMs mimic reasoning by expanding the query with intermediary tokens, basically creating a temporary scratch pad of related words on the same vector, but don't actually reason (this is known as chain of thought). Learning is partly computing lexical similarity, but it is also about extrapolating concepts from facts and inferences, applying those concepts to novel simuli, etc. LLMs really don't do reasoning, which is part of learning.
I would also like to add another key difference between human learning and machine learning. When I, as a person, read a book, I can conceptually understand the story. A machine would digest that same book by, for example, counting the numbers of times each word comes after another and assigning probabilities to occurrences or words or basic sentences. The major differences are still being explored. We don’t fully understand how humans learn, fundamentally. We know somewhat mechanisticly that neurons share and process chemical signals, but we don’t know how that information is interpreted or stored. We don’t know the equivalent of a computer language of the neurons (or other tissue types that may also learn). We know that we require input to learn, as do computers, the similarities kind of end there. That said, we definitely learn differently from how LLMs “learn.”
That's not true at all. We know how learning and memory work?
Counterpoint:
“Teach” the LLM the proper procedure for counting the Rs and tell it to commit that to a file (or as a tool call). Then from there it will suceed 100% of the time.
It doesn’t matter if it’s true learning or not. When you teach an employee to do something you give them the procedure and have them do it. If AI can be fed a procedure too, it will do it just fine even if it’s not in its true training. Does it matter if it’s consuming the procedure as prompt context everytime? Hell a lot of humans do read the procedure every time when it’s complex enough.
What the example shows is that you cannot "teach" the llm how to count the letter R, because LLMs don't work that way.
The AI company would have to solve the problem another way, let's say by recognizing that the user is asking for letter-counting, and pass that off to a different kind of algorithm that can count letters.
Yes, and that’s called a harness, which everyone uses these days. The harness increases perceived intelligence (or accuracy) by absurd amounts. You can “teach” (or the equivalent of) LLMs anything with a custom harness.
How is it teaching? Sounds like programming to me.
Well the idea is you don’t need to code it yourself, you can have it do it for you. Sure it’ll have bugs the first few times, but humans also make mistakes the until they get the hang of it.
The difference i guess is that humans are capable of learning and producing better code as they become more expert at it.
The big tech companies are surely trying to improve AI with these "harnesses" as you call them, and you can try vibe coding them yourself.
But it seems to me like spending a lot of time adding features to a technology to try to catch edge cases, but the edge cases will never end, and you'll never be able to use it for anything except rough approximations or bullshit
yes lets have the llm be a blurry frontend for a bunch of invisible proprietary harness programs. that surely is a valuable human endeavor
Because right now LLMs (and potentially other forms of AI) are the only technology capable of doing it. Humans can too, but are much slower and more expensive.
As you converse with it and it responds, part of the input is just your entire conversation up to that point, and its responses, plus your new prompt. That’s it that’s how the memory works.
And for this you pay an increasing amount for tokens.
Michael Reeves did an interesting piece on it: https://youtube.com/shorts/WP5_XJY_P0Q
Yeah I understand that much, my question was how does that personal record differ from what it already 'knows'.
In my experience, it seems to be limited to previous conversations, and it may be limited in length.
I pretty regularly delete old conversations, partly with the guide of privacy which I am sure does not actually delete or exist, but also for my own sanity of looking back on projects. And I don't even use AI that much, it's all free tier, mostly Claude.
That said, it will often get "lobotomized" and "forget" things after I delete the source conversation.
It does remember basic things. Like I use it a lot to help set up Docker containers, so it knows I have a Linux machine and a webserver and a Windows Machine. It knows the Linux box runs Docker and is named Squall. It knows these sort of, generic things, even across conversations. I can open a new chat and say "I want to set up a Wordpress sandbox" and it will ask like "Would you like to use Docker on Squall or run it outside a container with Apache."
Most harnesses that people use to interact with llms have memory and skills.
You can "teach" them a skill which is sort of like a macro so you can say "deploy to dev" and it'll look up that skill and execute it without any extra input needed.
For memory it'll keep files on your local of what it's seen before and can pull from the information in those files if necessary. So if you always ask it to label reports in a certain directory "Midwest expansion project" the harness will add that to its memory files and start to label it automatically.