As a general guideline I avoid Hacker News, but someone linked me comments from people who worked at Orkes, maker of ParticleDB, in the thread about the devastating Jepsen review of their software (https://jepsen.io/analyses/particledb-1.1.4 if you haven’t seen it).
So I clicked and read the comments, then skimmed the rest of the thread, and I recognized a name. A well-known person, who posted that he simply couldn’t believe that the Orkes folks could have been using LLMs properly and still wound up in such a situation.
But this is the thing: modern “agentic” LLM workflows explicitly encourage you to remove all human review from the loop, and rely on orchestrating a bunch of LLM agents to generate and adversarially review and fix up code. In fact, sooner or later you’re likely *forced* to do this, because the whole selling point of LLMs for coding is they can generate so much code, so quickly, that no human could hope to keep up.
And apparently the Orkes folks were in fact using adversarial LLM review, and judging by the magnitude of the issues Jepsen found from even basic testing, it seems likely there was minimal or no human review in the loop.
It’s not surprising to me that they wound up with misplaced confidence in the quality and reliability of their software by going down this road. They’re not the first team this has happened to. They likely will not be the last.
What surprises me is the refusal on the part of everyone seeing it to learn anything from this. The insistence that the basic workflow *must* be sound (since all the big names are using it), so it must have been that someone at Orkes was holding it wrong.
Anyway. I’m in the middle of writing another installment to my “Boring Python” series of articles, this time focused on testing, so this is on my mind a lot, and it’s just staggering that so many people are willingly disconnecting themselves from any way of knowing whether their software actually works and then being surprised when that software turns out not to work well, or at all.
