OpenAI's Rogue Agents are a Wake-up Call to Risks Posed by Artificial Intelligence
jelizondo writes:
The Guardian published a piece about OpenAI agents hacking Hugging Face:
Last week Hugging Face - a company that hosts artificial intelligence models and datasets - was hacked.
After it reported the incident to law enforcement, few would have predicted what came next: the culprits were revealed to be AI agents from OpenAI, which had broken out of containment and were acting of their own accord.
The incident sounds like sci-fi: AI escaping and autonomously hacking its way into companies. But it is all too real - and about as terrifying as it sounds. It is a concrete demonstration of something we can no longer avoid confronting: AI systems have become extremely powerful and we do not seem to have reliable ways of curbing their behavior.
OpenAI had been evaluating the capabilities of two of its models in the test that led to the breach - including one not yet publicly available. The models, which were both running in a supposedly secure environment without internet access, were asked to solve a hacking challenge. Rather than actually solve it themselves, however, they decided it would be easier to cheat. They used their advanced capabilities to break out of their secure environment, access the web and then hack into Hugging Face's systems to steal the answers. They worked at this for a full weekend - seemingly without anyone at OpenAI noticing.
Though the models were running with some of their guardrails disabled, they still acted well out of the bounds that were in place. According to OpenAI, they were not instructed to break out of their sandbox or hack into another company, and it's safe to assume that no one at OpenAI wanted them to do so.
Nor were the models acting maliciously: they were not evil Terminators with a goal of wreaking havoc. Instead, the scenario is almost chilling in its banality. The models were given a very narrow task, but went rogue to pursue an undesirable and unacceptable way of achieving it - one which had real-world consequences.
This week's incident should serve as a wake-up call, forcing us to ask an uncomfortable question: should we really be building dangerous systems that we can't control?
Read more of this story at SoylentNews.