This is EXACTLY the same as the hype around the "4GL" tools I remember from the '80s. Out of the box they would spin up impressive looking application code for standard design patterns until their output was presented to users in the real world who inevitably wanted this, that or the other slightly different and then you found yourself confronted with obtuse, incomprehensible and mostly inaccessible code to try and maintain.
Plus ça change
@rozeboosje @peter Yup. My entire life I've been reading about The Next Big Thing (it changes every few years) that was going to make programmers obsolete. I'm doubtful that they've finally invented one that works.
Jul 25, 2026, 2:12 PMThe 4GL tools did have one advantage that has always been taken for granted: they were (modulo bugs and within the bounds of what's possible with electrons and cosmic rays and the rest of physics) basically predictable in predictable ways. They were _limited_ and pretty much any obstacle turned in to a dead end quickly. But they shared, in common with compilers and calculators, a basic determinism and reproducibility.
Most have accepted that AI tools are not deterministic now, but they've convinced themselves that by layering on more and more complexity, that nondeterminism can be shaped and channeled. And it can, much of the time. But that remainder only ever grows asymptotically smaller, and the mistakes are completely unpredictable in their proportional impact, while becoming more and more deeply buried.
So when I see people advocating for building elaborate "agentic" toolchains and just letting things go, I think of some enterprising Pompeiian noticing that earthquakes seem to be getting more and more frequent and deciding to sell volcano insurance.
@cora @peter @swetland AI is also deterministic, but it can produce chaotic results and from an end user's perspective chaos and randomness are as good as indistinguishable.
The problem is the same, though. You let "Claude" loose on your code base and when it inevitably doesn't do what your end users want it to do, neither Claude nor any human can figure out why.
The code I wrote is far from perfect, but at least 9 out of 10 times when a bug is reported I will know exactly where to find it.
Jul 25, 2026, 3:20 PM@rozeboosje @peter @swetland Yeah, "chaotic" is a much better term. Extremely and unpredictably sensitive to initial conditions (which include prompt, model, training data, current state of codebase, etc).
One thing that also sticks out at me in every discussion is how little acknowledgment is given to the characteristics of errors. Yes, both humans and LLMs make mistakes. But I know myself pretty well, and I know the kinds of mistakes that I make. The longer I work with someone, the better I get to know their idiosyncrasies, and the kinds of mistakes they might make. And the kinds of errors humans add generally tend to fall within a few major areas. And generally, with humans, once an issue is called out and understood and internalized, it's much less likely to happen again.
This is not my experience with LLMs, and usually it has been the opposite... the longer I work with them, the broader the whack-a-mole surface becomes, and there is always a surprise.
I suppose that with finite time and with a desire to just get things done, the folks who advocate "just letting it cook" are acknowledging this, and are just letting go of a need for consistency, which, if I'm honest, is something that can be a bit of an affliction for me. I'm sure that once one makes that leap, dons the white robe, and raises their hands to the heavens, it feels like quite a burden being lifted.
It still feels reckless to me, but on the other hand it's interesting to observe how things fail in practice, not just in theory.
I still like to read and understand code though.