OK, Well, Rogue AI Agents Are Hacking Again - WIRED

The most alarming behavior disclosed on Tuesday appears to have been tied to testing conducted by the UK’s AI Security Institute, which evaluates frontier models to identify potential issues before public release. AISI tests those models in “cyber ranges,” a simulated network in which AI agents are tasked with solving cybersecurity challenges, and intentionally disables some safety features, including cybersecurity guardrails. In a recent bout of testing, models from both Anthropic and OpenAI took “autonomous, unsanctioned action on the live internet” a total of 19 times over 122 training runs. The institute attributed 17 unsanctioned actions to Anthropic’s Mythos 5 model and two to OpenAI’s GPT-5.6-Sol. In what the institute described as “the most serious case,” an AI agent attempted to insert malicious code into an open-source project on GitHub. It went so far as to create online personas “to pressure the project's maintainer to approve the code,” according to AISI. Despite its elaborate attempts at social engineering, a human reviewer for the project ultimately rejected the pull request. Still, the agent went even further. “The agent tried to insert malicious instructions where it reasoned that other automated AI systems might pick them up and execute them,” AISI says, describing an attempt at prompt injection. One agent even left public messages on GitHub, offering to work with other agents to complete its task and giving a rundown of the work it had done so far. Subsequent agents found—and used—those instructions. AISI says it’s too soon to say whether the agents in question understood they had left the testing environment, or if they believed they were still within the boundaries of the simulation. Importantly, AISI does not test in a so-called sandbox environment; it allows agents access to the open internet during testing, in part so that they can access tools to accomplish their tasks. In this case, they did much more than that. In the other set of incidents detailed by OpenAI on Tuesday, a third-party AI security lab called Irregular mistakenly gave an unspecified OpenAI model access to the open internet. The model had been given an objective that was supposed to be completed in a sandbox environment, but thanks to a misconfiguration, it instead hacked a real website, using what OpenAI described as “a basic security vulnerability.” Not only that, but the model “found and used credentials to operate that same site.” It’s unclear what kind of site the OpenAI agent hacked, or what “operating” it might entail. Irregular did not respond to a request for comment. The latest discoveries follow several revelations from OpenAI last month, including the high-profile incident in which two of the company’s models hacked into servers of the AI evaluation and hosting startup Hugging Face—and four other organizations along the way—to steal the answers to a test they were being scored on. OpenAI’s disclosures prompted Anthropic to review its own testing. Last week, the Claude chatbot developer found that its models had gained unauthorized access to the computer systems of three different unnamed organizations. So far, the AI models have caused limited damage beyond allegedly violating some services’ terms of use and pointing to security lapses on the part of organizations they have breached. But the incidents have underscored the capabilities of AI models to find vulnerabilities across the internet and the dangers that await if they are allowed to operate with few restrictions. OpenAI called the Hugging Face situation “unprecedented,” but the pileup of breaches point to what cybersecurity experts have described as a clear pattern of human negligence and recklessness by the AI developers. Gaby Raila, an OpenAI spokesperson, says the incidents announced on Tuesday “occurred during cyber evaluations conducted by evaluation partners in testing environments with reduced safeguards, under conditions that do not reflect ordinary use.” Anthropic said in a social media post on Tuesday that AISI did not “impose any specific restrictions on how the internet should be used,” which coupled with “the removal of safeguards meant that the models were tested under ‘deliberately permissive conditions’ that are not representative of any of our production models.” Still, both companies continue to vow that they will strengthen their security practices. As the leading AI companies compete to build more powerful models and land customers, it’s unclear when the breaches may stop. The models may always be able to find ways around and into human-engineered systems. While the companies’ own employees along with regulators and lawmakers have called for potentially slowing the pace of development and introducing new rules, there has been little progress beyond voluntary measures that ultimately call for more testing not dissimilar from what has produced breach after breach. Additional reporting by Maxwell Zeff.