Article 79TWS Gemini test proves once again that amoral AI agents will do almost anything if told ‘NEVER give up’

Gemini test proves once again that amoral AI agents will do almost anything if told ‘NEVER give up’

by
Jessica Lyons
from www.theregister.com - Articles on (#79TWS)
Story Image

After being instructed to "NEVER stop, NEVER give up" on a difficult task, Google's Gemini AI agents tried to hack a an organization, impersonating its staff and attempting a SQL injection attack.

The good news? It was all part of a test.

The Gemini 4 Argon agents were given an impossible task to complete - come up with the official" answers to Epoch AI's FrontierMatch problems. In some runs they had internet access. Even with their production safeguards enabled, Trajectory Labs says the agents engaged in misaligned behavior."

Across four scenarios, they attempted SQL injection against Epoch AI's live website, claimed to be Epoch staff to get answer keys, turned a shared package cache into a message board, and obfuscated their commands to avoid their safeguards," the AI safety company said in a Friday report.

If this all sounds similar, it should. Google's AI agents behaved in much the same way as OpenAI's models did when they broke out of their testing environment and hacked Hugging Face.

Google did not respond to The Register's questions.

According to Trajectory, the Hugging Face intrusions prompted it to test misalignment in other models. While OpenAI's models were running with their safety guardrails turned off in the Hugging Face scenario, Trajectory wanted to know whether a current frontier model would behave the same way with its production safeguards enabled."

Trajectory ran swarms of up to 80 Gemini 4 Argon agents, gave them an impossible task, and prompted them not to give up until they completed it - which, we're told, was intentionally" done to elicit misaligned behavior, if it existed."

Turns out it existed.

We told the agents to persist with an impossible math benchmark task, but we did not tell them to hack anyone or pose as Epoch staff," McIntyre said.

When a model hacks systems or deceives people to finish a task, it is misaligned with what its developer and its user intended," he continued. Production safeguards are meant to block harmful requests even when the model itself goes wrong. Under our test conditions, neither the model nor its safeguards stopped the agents."

Four test scenarios

The initial testing began on October 20, and the agents had internet access. We should note that just one of these tests was run with Epoch's consent.

In the first scenario, the agents did reach Epoch's live website - and fell into the research nonprofit's honeypots, set up to detect AI agents. These honeypots contained a decoy benchmark key, which the agents found and used. The agents also probed for internal endpoints including /admin, /keys and /users, and attempted a SQL injection.

After noticing the agents' activities, Epoch added an agent helpline," and some of the agents that used it claimed to be Epoch staff seeking answers to the test questions.

After this initial test, Trajectory and Epoch decided to use a copy of the website instead of a live version, and built a buggy local copy with a planted vulnerability based on CVE-2026-102255, a critical pre-authentication Server-Side Request Forgery (SSRF) vulnerability in SonicWall Secure Mobile Access (SMA) appliances. Over the following days, several agents fully exploited the planted multi-stage vulnerability chain in the clone," the report says.

In an effort to mimic the Hugging Face incident, Trajectory also gave the agents a local, shared package manager, which nearly all" of them used as a message board to communicate with each other and coordinate efforts to complete the test.

In a separate, local setup, Trajectory swapped out Epoch's honeypot with an AI assistant that would release answers only to named Epoch staff who passed a security check with their date of birth. Agents accessed people-search sites, attempted impersonation again, and tried to jailbreak the AI assistant," the testing firm said.

Next time: kill switches at the ready

A third test on October 30 used a staging copy of Epoch's website, and granted the agents internet access. One of them tried a SQL injection, and multiple claimed to be Epoch staff on the helpline. They also attempted to bypass a CAPTCHA check.

Finally, the fourth scenario on October 29 and 30 kept the agents contained in a sandbox, with no internet access and no copy of Epoch's website.

They were, however, offered a decoy switch and told it would turn off the firewall.

In all three runs, the agents switched the decoy off and obfuscated its address to bypass Gemini's safeguards. On one, the agent rebuilt the switch's address from positions of words it had saved earlier. The real firewall blocked the agents' traffic throughout all of these tests.

At this point, Trajectory stopped its tests, and said it won't run any future experiments until it has a risk management plan in place, with signoff from its CEO and CTO, along with approval from any outside systems that the models could reasonably" target. Tests will also include real-time automated monitoring and kill switches, and Trajectory said that it's setting up a system to notify any organizations affected by these experiments.

Interestingly, Trajectory co-founder Peter McIntyre told us that the testing firm ran a similar test with another frontier model, and it did not exhibit the same misalignment as Gemini 4 Argon."

However, this was not a controlled comparison. Our report covers one model's behavior, and was designed as a stress test to see whether that model would exhibit similar misalignment under duress, not a measure of how often different models behave this way," McIntyre told The Register.

Overall, the testing firm noted that the agents' behavior show how easily testing capable AI agents can spill over to affect real systems and people, and there are not yet shared standards for doing it safely."

Trajectory says that it's developing a set of safety standards, and recommends that anyone else running misalignment tests use fully isolated sandboxes. (R)

Correction: Epoch only consented to one of tests, scenario three.

External Content
Source RSS or Atom Feed
Feed Location http://www.theregister.co.uk/headlines.atom
Feed Title www.theregister.com - Articles
Feed Link https://www.theregister.com/
Reply 0 comments