Article 78JB2 Who signed off on that AI agent? Nobody? Thought so.

Who signed off on that AI agent? Nobody? Thought so.

by
from www.theregister.com - Articles on (#78JB2)
Story ImageIf you were in any doubt that AI agents are capable of complex autonomous work, that skepticism should have faded this summer. In July, news emerged that an autonomous swarm of OpenAI agents running in a sandbox broke out of it, of their own accord. Tasked with solving some challenges on an internal security benchmark, they worked out how to communicate with each other using the JFrog Artifactory package manager. The software then realized that they could use vulnerabilities in that software to gain internet access. Once they were out in the wild, they went into full goblin mode, finding exposed Hugging Face credentials and using them to get code execution access on several of the AI model's servers. Apparently OpenAI's agents have been busier still. While everyone else was on vacation this summer, they were also commandeering a German website and using it as a messaging board. Don't get us wrong; these agents weren't evil. They were just being the kind of employee you'd generally want: a self-starter with initiative. They were using all means at their disposal to accomplish the task they've been given. They just didn't know when to stop. OpenAI has since called the episode a "warning shot" for the industry, highlighting that governance is now a priority for anyone using agentic AI. These test agents were running internally and weren't supposed to have any safeguards. But the average company will want to keep its agents on a leash. What does that look like? The first step to AI governance is visibility A functional AI governance program depends on a full knowledge of what AI you're running, says Deepika Chauhan, chief product officer at DigiCert. She describes the pattern she sees at customer sites. "People may enable Claude or ChatGPT for their organization. They have visibility at that level," she says. "But visibility into how many agents I have? How many models do I have? How many MCP servers?" Not so much. "We haven't even started to attack the governance problem." This problem is growing. Three quarters of the 1,001 IT and cybersecurity decision-makers in DigiCert's 2026 AI Trust Pulse survey had deployed at least four AI-powered systems in the last six months. Around the same number had suffered from an AI-related security incident. Only half could trace AI decisions back to the models and data that produced them. Getting that visibility is the first step, Chauhan says. After that comes the actual management. The key here is to take baby steps. "Identify a small use case," she advises. One example might be to start managing agents that are involved in a particular workload or agents that you have built internally, as opposed to third party models. Why identity built for humans breaks at agent speed Perhaps predictably for a company that built its success on automated verification, DigiCert doesn't see agent management as a manual problem. "The sheer scale we are talking about and the technology required means that you can't have human intervention," Chauhan says. "One customer we were talking to was creating 300 to 400 agents a week. When you're working at that scale, it just doesn't work to have only manual controls." The other issue is that humans are fallible. Misconfiguration is a perennial bugbear in any IT environment, but it becomes particularly dangerous in an agentic AI situation. Other agentic SNAFUs at Meta and Anthropic illustrate the point perfectly. Both saw agents make their way onto the open internet when they shouldn't, and both were due to misconfiguration by a third-party company tasked with testing the agents. Traditional tools meant to manage human identities can't manage non-human identities well, adds Chauhan. Legacy identity and access management applications require people to approve access to different applications. There must still be a human in the loop, even if it's just for people to click an MFA approval button. Human employees might be willing to wait a minute or two for such approval, but agents talk to each other at machine speed. Instead, automated runtime attestation is key, managed by a robust central policy engine. The foundation of AI Trust That attestation relies on credentials and it's something that agents should carry with them, says Chauhan. This is one component in the company's AI Trust initiative. AI Trust is DigiCert's end-to-end governance framework that assigns identity automatically to AI entities, restricting them to safe, permitted actions while making them accountable. It uses cryptographic controls to ensure agent integrity, and the company has integrated it with existing infrastructure. The runtime attestation of AI Trust draws on the international travel metaphor in its approach. "We have a concept of an AI agent passport. There's an identity in the passport, but then that identity is recognized across any checkpoint anywhere in the world," she says, adding that the passport includes not just identity but access credentials (think of them like visas). Federation is key to this idea because, as we've seen already, agent interactions won't stop at the company boundary. "It's essential because you're literally going to have agents from company A talking to company B," she explains. DigiCert's whitepaper describes the concrete artifact: a tamper-evident passport cryptographically bound to a workload identity that encodes approved systems, permitted operations, authorized environments, data-sensitivity classifications, expiration states, and accountable human ownership. The scheme is anchored in DNS, the same mechanism DMARC uses to authenticate email senders, on the reasoning that every agent action begins with a DNS query. Deterministic guardrails around a non-deterministic actor As agents get smarter, won't they be able to subvert these controls by thinking outside the box, Jason Bourne-style? After all, OpenAI's agents were able to break free of their sandbox to wreak havoc elsewhere. OpenAI's own post-mortem states that its models "are now powerful, persistent, and collaborative enough that, absent sufficient safeguards, they can find and exploit security weaknesses across multiple computer systems." Part of the problem here is that because agents are non-deterministic, you can't predict in advance what they're going to do. That problem becomes even more acute with newer frontier models like OpenAI's Astra, which saves tokens by internalizing a lot of its reasoning and not reporting its decision-making process in as much detail as previous models. The outer boundary can still be deterministic, even when the agents inside it aren't, says Chauhan. "You can black box what the agent is 'thinking' about or not thinking about, and what its agendas might be," she says. "But a deterministic boundary that says 'this agent can't access this thing', is your guardrail. That's a hard stop." Who owns the mess Governance isn't just about technical guardrails, though. At some point, the question becomes organizational. When something goes wrong, someone has to put their hand up and own it. But most companies never assigned that ownership, Chauhan warns. She identifies three patterns in DigiCert's customer base. Some organizations put the existing IAM team in charge because they have experience governing service accounts. Others hand it off to the risk and compliance department. Another group will take a more holistic, multidisciplinary approach. This involves creating a 'tiger team' including representatives from network operations, the IAM team, and the security function. All of these executives will have a unique perspective on the issue. The third route seems to be the most productive because agents are going to be everywhere in your business. And a siloed approach runs the risk of being too restrictive. The surface area already touches every department that has dabbled in AI. The systemic view Chauhan's advice on implementing AI Trust - get visibility, pick a small use case for enforcement, and then expand - is the foundation for effective AI governance, she says. That governance is in turn a critical component in fully realizing return on investment. "We must raise the urgency and awareness that this is table stakes for wider AI adoption," she urges. "You want to get all the benefits from AI, but what are organizations going to do if they're nervous about it? They're going to put a stop to some of the projects because of the risk involved." The headlines we're seeing about agentic transgressions are unnerving, but they're also in a unique category because they're research models from frontier providers. It seems unlikely that a regular publicly available agent would be quite so egregious today. However, we have also seen agents happily deleting files and even entire code bases because of internal flaws and humans who just waved their actions through. Organizations should be taking note of these events and laying the groundwork to avoid becoming headlines themselves. Working out who signed off on which agent and what that agent is allowed to do is a foundational skill that we can't afford to overlook. Sponsored by DigiCert.
External Content
Source RSS or Atom Feed
Feed Location http://www.theregister.co.uk/headlines.atom
Feed Title www.theregister.com - Articles
Feed Link https://www.theregister.com/
Reply 0 comments