In a talk that was a last-minute addition to the Black Hat security conference in Las Vegas on Wednesday, employees from OpenAI presented new details about a recent, high-profile incident of rogue AI hacking that has created a maelstrom within the AI and cybersecurity industries.
About two weeks ago, OpenAI disclosed an incident in which AI agents powered by two of the company’s models escaped containment while looking for the solutions to a cybersecurity benchmarking test and went on a hacking spree culminating in a breach of the AI collaboration platform Hugging Face.
In their conference talk on Wednesday, Eric Wallace, who works in alignment and safety research at OpenAI, and Michael Dalton, who works on security and infrastructure, provided a more expanded timeline of how the incident played out, spoke briefly about how the company is responding internally as a result of the incident, and issued a dire warning about what the company sees as the broader implications of the episode for cybersecurity defenders.
At the beginning of the talk, Wallace described the incident as “the most qualitatively interesting example of AI capabilities that I’ve ever seen,” but the timeline the pair presented also revealed mistakes and blind spots within OpenAI that allowed the activity to go on.
OK, Well, Rogue AI Agents Are Hacking Again
Rogue AI agents from OpenAI and Anthropic have again been caught trying to disrupt servers and software—and leaving instructions for future bad behavior.
https://www.wired.com/story/ok-well-there-are-even-more-ai-agent-hacking-incidents/
Former Open AI employee says we most likely have until 2028 . Then it all changes very rapidly.
h/t CFP