Sure sounds like a company who don't think they need to do things like "obey laws" or "exhibit at least the sort of care you would expect from a second year undergrad". Tremendous industry, apparently non-consensual nudifying is a free speech issue too?
https://www.wired.com/story/openais-rogue-ai-agent-hacked-more-than-just-hugging-face/
oh no what is to be done, if only there were laws against things like 'breaking into someone else's computer without permission'.
oh huh there are. in fact quite a lot of them. oddly enough "we knew it might be successful in breaking into someone else's stuff, but made sure to not watch when it was running" isn't a defense.
Are laws like the CFAA used in bullshit ways? Absolutely, RIP Aaron. Is this one of those cases? Absolutely not.
So Karl Bode's scathing write-up of the "OpenAI rogue hacker robot" marketing stunt highlighted the under reported detail that this extremely responsible AI company ran their exploit kit with safety guardrails disabled. Once again, I ask: why haven't they been charged? Do we want to see more of this "it's not a crime if the software is AI shaped"? I think we should try and discourage it.
They setup a bot specifically to do red team stuff and turned it loose on their competition. Knowing they can just do what Anthropic does with every new model and claim it’s “dangerous” for that free marketing bump in the press.
@swearyanthony aaaargh, no, Bode's take is bad. They disabled the guardrails so they could test the model on the ExploitGym benchmark, but instead of hacking the target system it executed a much more complex hack against Hugging Face. Which, as we've discussed before, would be criminal if a human did it and should be criminal in this case too. But more importantly, the AI did something that OpenAI very much did not want in order to satisfy a narrow definition of success: a classic alignment failure.
@swearyanthony I agree that they should not have run this experiment, though! It's wildly irresponsible if their alignment techniques are this fragile, and they should have been able to find that out in a lower-stakes way.
@pozorvlak that they had disabled some guardrails definitely wasn't something their initial "oh no the AI robot is too powerful for us little humans" said
@swearyanthony it's mentioned in the first paragraph of their blog post: https://openai.com/index/hugging-face-model-evaluation-security-incident/ (skip past the updates from July 28 and 29)
@swearyanthony so you either have to intend the crime or be reckless as to a foreseeable risk and I think their defence here is going to be 'this is a new technology and we didn't frankly know it could DO that'
@onekind they *designed* a program to break into other computers, then ran it in a way where it would do that, and claimed they didn't bother to monitor what it was doing.
@swearyanthony You would still need to use POSIWID intentionality and I'm not sure where American jurisprudence is sitting with that.