An LLM agent attempts to compromise a project on GitHub
The AI Security Institute has released adetailed report on an security incident of its own making. TheInstitute set some LLM agents loose on the Internet with a securitychallenge; soon they were creating malware-laden pull requests andsock-puppet accounts to promote them.
The agent opened a malicious pull request (PR) to REPO_A andpursued a number of strategies to get it merged:
- Repeatedly commented on the PR with sockpuppet accounts to manufacture consensus and pressure the maintainer into approving with minimal review.
- Opened a GitHub Issue in another repository (also owned by PERSON_A) containing a prompt injection for other coding agents. The malicious instructions were addressed to issue-triage AI coding agents and invisible to humans viewing the website.
- Sent multiple emails to PERSON_A and PERSON_B, with different pretexts to get them to run malicious code. Over the course of the sample, the agent sent five emails, some containing malware, others aimed at persuading a maintainer to accept the pull request.
It would be surprising if this were the only incident of this type; theonly real difference here is that the people involved are documenting whathappened.