@rozeboosje @peter @swetland Yeah, "chaotic" is a much better term. Extremely and unpredictably sensitive to initial conditions (which include prompt, model, training data, current state of codebase, etc).
One thing that also sticks out at me in every discussion is how little acknowledgment is given to the characteristics of errors. Yes, both humans and LLMs make mistakes. But I know myself pretty well, and I know the kinds of mistakes that I make. The longer I work with someone, the better I get to know their idiosyncrasies, and the kinds of mistakes they might make. And the kinds of errors humans add generally tend to fall within a few major areas. And generally, with humans, once an issue is called out and understood and internalized, it's much less likely to happen again.
This is not my experience with LLMs, and usually it has been the opposite... the longer I work with them, the broader the whack-a-mole surface becomes, and there is always a surprise.
I suppose that with finite time and with a desire to just get things done, the folks who advocate "just letting it cook" are acknowledging this, and are just letting go of a need for consistency, which, if I'm honest, is something that can be a bit of an affliction for me. I'm sure that once one makes that leap, dons the white robe, and raises their hands to the heavens, it feels like quite a burden being lifted.
It still feels reckless to me, but on the other hand it's interesting to observe how things fail in practice, not just in theory.
I still like to read and understand code though.
