Technology

“The raw output of ChatGPT’s proof was actually quite poor. So it required an expert to kind of sift through and actually understand what it was trying to say,” Lichtman says. But now he and Tao have shortened the proof so that it better distills the LLM’s key insight.

This tracks with what I have seen regarding AI. It looks superficially awesome, but when you start to analyze its output it has a lot of holes that require someone trained in the art to fix. You know, someone with years of experience, and who got that experience without the benefit of AI shortcuts.

What happens 10 or 15 years from now, when all the current crop of experts are retired and all the experts who could have curated the AI output had to spend all that time as baristas instead because the AI took all of their entry level jobs?

This reminds me of a story my graph theory professor told me (long before LLMs). One of their grad students discovered that a subset of graphs that are of type A and B at once has fantastic properties, such as fast searching, and a few others, useful in communication networks etc.

Excited about their potential thesis, student asked the professor to take a look. After calculating which graphs actually are types A and B at the same time, professor found that the intersection of such graph types is a null set. So the theoretically nice graphs the student "discovered" simply do not exist.

Easy to be surprised when you don't know how the magic box works. Basically magic.