@samth @shriramk The revolution bit is obvious. But humanity has spent the best part of the last 2000 years, wondering about some of the things that LLM’s suddenly claim to do. Not hundred years ago, Gödels theorems seemed to impose serious limitations on what symbolic reasoning could do. Now, all of a sudden, folks claim all of this has changed.
Asking “why” seems a legitimate question…
Sep 22, 2026, 9:54 PM@sandorspruit @shriramk I agree, "why does this all work" is an excellent question that I don't think we know the answers to. But I don't think we in PL are well placed to answer it, and we don't have to in order to take stock of the revolution.
@sandorspruit @samth
Agreed. I struggle to understand why a high-enough dimensional space is enough for a rather dumb algorithm to produce remarkable output. But I've been surprised by volume and dimensionality before (eg, Alexei Efros's work on computational photography). ↵
@sandorspruit @samth
The way I see it, we're where the mechanical sciences were in the 1700s: people are inventing gadgets without really understanding how they work (and sometimes —because of that— they blow up!).
We're starting to get to the 1800s (Dalton, Carnot, Maxwell), with foundational sciences cropping up to explain and guide.
Then we'll get to the 1900s, 2000s, and so on.
Except in this domain we'll speedrun that explanation and engineering cycle; it'll take a decade at most.
@sandorspruit @samth
Think about steam engines.
People like Newcomen and Watt performed observations, had intuitions, and did experiments, and make significant progress. They made tremendously useful engines, even if the science was poor and the engines well short of optimal.
It took ~50 years before before Carnot explained the idealized thermodynamic model, and people could improve the engines from first principles.
We can think of current GenAI systems as being Watt engines.
@sandorspruit @samth
The steam engines created an entire industrial revolution even before the science was established.
A tremendous amount of useful and effective math was done long before we understood the foundations and limits (with new insights every century: epsilon-delta, Galois theory, Gödel, etc.).
This seems to be the nature of things? When the tool become useful enough, theoreticians rush in to finally bring order.
@shriramk @sandorspruit @samth its not merely enough dimensionality to avoid local minima. its also… absolutely absurd levels of compute. so it is a combination of sufficiently fast chips with sufficient memory… and subsidization.
Sep 23, 2026, 1:36 AM@gershom @shriramk @sandorspruit I mean, I recognize all of that. But it's still a big pile of matrix multiplication that can solve famous math problems.
@shriramk @sandorspruit @samth i think lately of the “its all so stupid” speech from knives out 2: glass onion.
@sandorspruit @samth
There are certainly enough people on here who question how high that dimensionality is. (-:
@sandorspruit @samth
Most of my research these days is in applied cognitive science, actually.