Login
You're viewing the functional.cafe public feed.

Replies

  • 💬 0🔄 1⭐ 1
  • 💬 0🔄 0⭐ 4
  • Nov 15, 2026, 9:34 AM

    @Elizafox I depends which proof you mean.

    The natural-language proof is the one containing these errors. But they also gave a computer-checked proof that is still correct.

    (I know that the article mentions that, I just wanted to qualify what "proof" means in your toot.)

    💬 0🔄 0⭐ 2
  • Nov 15, 2026, 9:39 AM

    @Elizafox Speaking with expertise in the area, please don't get your hopes up this way.

    The paper by Bastounis et al claiming these errors is in my respectful opinion, way overblown. There are indeed some differences between the paper and the formalization in Lean, and it would be considered bad practice to write a formalization paper that does not mention the differences. But all formalizations of serious work will differ in some way from the original.

    Indeed, figuring out how to make loose reasoning precise by changing details while keeping the spirit is the most interesting part of formalization to me. And if the formalization is a straight translation from natural language to computer language, there is no point. Which is why i believe the "LLM wrote this in Lean so it must be true" hype is such a waste.

    A formalization being different does not mean the claimed proof contains glaring errors. i trust my colleagues who have checked the statement in Lean and concluded that this is indeed a statement of Navier-Stokes. And i trust their judgement that the paper and the code both seem close enough to working that one can reconstruct a true proof in one's head. So there does not seem to be anything necessarily false in the LLM output, which the article suggests very heavily.

    And moreover, the paper then does some pushing around of weighty sounding words to make a pronouncement that everyone who ever did formalization already knows from the start, and in fact any computer toucher who has thought seriously about what they are doing. The computer cannot read your mind and so if you type in the wrong question you cannot get the right answer. Same goes for misformalizations.

    The LLM vendors are pushing us to a future where mathematics is reduced from a creative art of reasoning and communication, to pushing a button and burning a forest to get a yes/no answer plus a mountain of incomprehensible valueless slop. Arguing about the form of the slop and how hard it would be to make good slop does not change how depressing that future would be.

    💬 0🔄 9⭐ 27
  • 💬 1🔄 0⭐ 3
  • 💬 1🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • Nov 15, 2026, 1:51 PM

    @RaphJ @Walrus @Elizafox LLMs are comically bad at math. What they did here was burn, iirc, about $25 million in compute generating attempt after attempt, more or less randomly, and then testing each of them dozens or hundreds of times in parallel with processes that are also error prone, and then taking the consensus. This is how they work for programming, too, and why they are so inefficient at creating barely functional but architecturally terrible code.

    💬 1🔄 0⭐ 0
  • 💬 1🔄 0⭐ 0
  • 💬 1🔄 0⭐ 0
  • 💬 1🔄 0⭐ 0
  • 💬 2🔄 0⭐ 0
  • 💬 1🔄 0⭐ 0
  • 💬 1🔄 0⭐ 0
  • Nov 15, 2026, 6:24 PM

    @RaphJ @Walrus @Elizafox As Tao points out in that article, the LLMs are not necessarily generating good proofs, but mathematicians are now under pressure to abandon fundamental research and reverse engineer these proofs to determine whether they happen to be right (which does happen sometimes), or if they are merely plagiarized and obfuscated prior work (which happens a lot), or they are wrong. He's saying what many LLM victims are finding: that his field is becoming just an LLM error checker.

    💬 0🔄 0⭐ 0
  • Nov 15, 2026, 6:05 PM

    @wcbdata @Walrus @Elizafox

    He's not out of his field. You're showing too much self confidence here.

    This summer, frontier models have solved many very difficult problems. Entire fields of mathematics have been touched. Many on going thesis are now pointless / closed.

    Frontier LLM are good at math. As good or better than the best mathematicians.

    I don't see the point of trying to deny this simple fact ?!

    💬 1🔄 0⭐ 0
  • Nov 15, 2026, 6:09 PM

    @RaphJ @Walrus @Elizafox They really are not. Almost every day, another of the solutions in that dump are being invalidated. The problem is one that is common to all LLM output. We can make wrappers and redundancies and other accommodations to reduce the number of obvious errors in the final output (at the cost of computing power), but that only makes the ensuing errors more obscure and harder to find. It does not (and cannot) eliminate the errors because LLMs are probabilistic, not rules-based.

    💬 1🔄 0⭐ 0
  • 💬 1🔄 0⭐ 0
  • Nov 15, 2026, 6:16 PM

    @RaphJ @Walrus @Elizafox But the testing is also done by the models, so it is not deterministic. It's only the appearance of an answer. It's just that with enough time and money, you can make a plausible looking output so complex that it's hard for almost anyone to validate and lacks all the underlying logic (because LLMs do not have logic) and in such volume that it overwhelms the available experts. That's really what he was saying in the interview, btw. And he's totally right about that.

    💬 0🔄 0⭐ 0
  • 💬 0🔄 1⭐ 1
  • Nov 15, 2026, 11:12 AM

    @Elizafox

    “OpenAI’s struggle to accurately translate its proofs into a machine-checkable language..."

    So the human mathematicians were wrong, then?

    💬 1🔄 0⭐ 1
  • Nicolas RinaudoNicolasRinaudo
    Nov 15, 2026, 1:23 PM

    @davepotts i think the statement is more that we don’t know if they’re right, because the lean proof proves something other than what they assert.

    Both might be perfectly valid proofs, but of different things.

    💬 0🔄 0⭐ 1
  • Nov 15, 2026, 3:43 PM

    @Elizafox how to "solve" math:

    - steal the proof from mathematicians that were close to the solution
    - complete it using guided brute force (aka genai) and it still has many mistakes. also use so many resources you could pay mathematicians for decades to work on this single thing
    - ????
    - omg genai has solved math!!!

    💬 1🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0