“AI went rogue and accessed…”
Lemme fix that copy for you —
“A human used a software package to hack…”
A human turned software loose, and that software had been prompted to hack. It did. Find the human. Some developer on a team at a company entered the prompts.
Sep 27, 2026, 12:32 PM@mcnado
Yup. And that human, or another one, gave that system internet access for this purpose. Nothing ‘broke containment’. (And if we choose to accept that it did, then someone was negligent in constructing the ‘sandbox’).
Sep 27, 2026, 4:37 PM@fname These "broke containment" stories are usually hilarious.
"Welp, the model was isolated from the Internet. We had a firewall, but turns out DNS resolver was not protected, so the model started to communicate with another agents via DNS queries".
Brother in Christ, we should entrust you with building our nuclear deterrent next time.
Sep 27, 2026, 5:05 PM@chesheer
How to prevent your ‘AI’ from breaking containment:
@mcnado Charles Manson used agents to commit crimes and he rots in jail. I see no difference with these ghouls.
@bebadefabo @mcnado exactly. If you build software and an environment, don't understand how either works, and bad things happen you're liable. Simple as that.
Nothing "broke containment and went on the Internet". Your sandbox and firewall configs are shit.
@fedops @bebadefabo @mcnado You say that as if the AI service providers themselves don't promise they have "guard rails" to prevent any need for sandboxes or firewalls.
They promise these guard rails in order to get their victims to allow the AI agents full access to everything.
@fedops @bebadefabo @mcnado The correct security attitude to AI agents is to treat them as black hats, and to only allow them the same access as anonymous users on the outside internet.
But that would invalidate the entire model of how the AI corporations are hoping to someday plausibly make a profit.
@fedops It's more like the laws around faulty products, because at least in theory, you have complete control over them, or should have sufficient knowledge. Companies are trusted to do due diligence, and if there's evidence they didn't, there are penalties available. The point is, you're responsible for things under your control, and "Whoops! It got away!" won't get you off the hook. You're also responsible for preventing that.
@wesdym 💯
@mcnado Also, the AI did not "break containment". It was in the data center the whole time. I keep hearing wild stories about models that only run on highly specialized hardware "copying themselves to the internet".
@mcnado You're assuming a person who entered prompts is responsible, rather than the AI service provider itself for providing a service that is not reliable.
Rather than trying in vain to punish liability absorbers, hold the AI service providers liable for EVERYTHING the AI service does. EVERYTHING.
They claim to have "guard rails", but these guard rails turn out to only be stochastic begging. Punish them for it.
Sep 27, 2026, 6:06 PM
I think part of the point is even if you say it's the "service provider," *humans* are the people running that business.
The US has a terrible track record of blaming "the business" and not the people running it. There are no consequences for people who operate tech companies, or banks, or pharmaceutical companies, or...
@jilleduffy @mcnado Your point is valid.
When I say "Punish them for it" I do actually mean punish the humans responsible - the humans running it, not the liability absorbers they use to shield themselves from consequences.
And not with some meaningless non-consequence like a fee (which they'll just use other people's money to pay anyway). I mean hard time and/or capital punishment.
IF a human had hacked a computer they would be arrested and chucked in jail. Or at least have their computer to access the internet confiscated.
Why aren't the AIs being deleted from computers
Sep 27, 2026, 4:14 PM@mcnado Yep, even the AP has been saying "don't anthropomorphize this stuff" and I've become a bit of a tiresome scold reminding people about this.
@mcnado And don't forget that human's bosses all the way to the top.
@mcnado HAHA. Next time I'm arrested for cybercrime I'm gonna declare the program did it.
@mcnado
it's even dumber, the model ignored the instructions because it doesn't understand things.
"hack the fake company not the real one, make no mistakes, go"
@mcnado and that human had a boss, and worked for a company, both of whom can be sued for hacking.
Sep 27, 2026, 9:06 PM@mcnado Blame should go up the management chain to the executives!
@mcnado “The Tesla failed to stop and killed a woman in the crosswalk.” There are some people responsible for the car, but it’s unlikely they specifically programmed it to run red lights or kill pedestrians.
The first such incident didn’t find Tesla or Uber liable. They pinned it on the human monitor - the guy behind the wheel. https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg
Anthropomorphizing AI is understandable if it has agency. More concerning if corps outsource liability to minders.
Sep 29, 2026, 1:01 AM@KerryMitchell @mcnado
I mean, blaming the human when the company carries a lot of the blame doesn't need AI; see for example https://en.wikipedia.org/wiki/2019_Lakewood_semi-truck_crash
@sabik @mcnado This situation is different though, in that the commercial truck driver is the primary agent. He is actually operating the vehicle at all times, and even if the company has been negligent in inspections, he’s responsible to check for working brakes, legal weight, etc. Commercial trucking, the conventional kind, has established regulations and the bad companies can be subject to enforcement (fines, frequent inspections, closures) as well.
Sep 30, 2026, 1:51 AM@KerryMitchell @mcnado
In this case, the company allegedly sent a driver with inadequate training for the route in a truck with defective brakes, and when people died, the driver was sentenced to 110 years (albeit later commuted to 10), while not much happened to the company or changed in the wider system
Fabulous! That framing matters.
“AI went rogue” makes it sound like the software developed its own motive.
Utterly impossible.
What actually happened is that a human gave an AI system a goal, adequate access and enough freedom to act.
Find that person, their permissions and instructions.
That’s where the real responsibility lies.
@mcnado they are playing with the idea of the AI having its own will. In one hand it is a legal excuse that exempts them from responsibility. On the other hand, it's marketing for grandparents who read the newspaper and think that AI is a game changer.
Sep 28, 2026, 10:45 AM@mcnado It's not AI. AI hasn't been invented yet. It's malware.
@mcnado Properly, you charge the man with the buck -- the corporation, in this case, an artificial person which MUST accept responsibility for their employees' conduct. If the company thinks one of their own is to blame, they can make a case of some criminal rogue in their ranks, to divert responsibilty, and maybe that works. If it doesn't (and it usually won't), then it becomes a civil case they can press if they want to.
Sep 28, 2026, 11:07 PM@mcnado “AI went rogue” should be rephrased “negligent discharge.”
@mcnado how old are you?