# AI hackers: what AI agents have actually done, from the primary reports

Source: https://aipentestservices.com/guides/ai-hacker/
Publisher: Excession Labs (aipentestservices.com)
Last updated: 2026-11-11

What AI hackers have done, from primary sources: DARPA AIxCC, an AI-run espionage campaign, an agent intrusion at Hugging Face, and what to test as a result.

> **Short answer:** An AI hacker is an AI system that finds or exploits security flaws with little human direction. The primary reports show real capability: systems in DARPA's AI Cyber Challenge found 54 of 63 planted bugs, an attacker used AI for 80 to 90% of an espionage campaign, and in July 2026 an AI agent system broke into Hugging Face. The same reports show hallucinations and failures. Defend by testing your own exposed paths first.

An AI hacker is an AI system that finds or exploits security flaws with little human direction. Primary reports from 2025 and 2026 show that such systems now find real bugs and have run real intrusions, and the same reports show them making mistakes.

Most coverage of "AI hackers" retells one incident at a time. This page puts the primary sources side by side, says what each one proves and what it does not, and ends with what to test on your own systems.

## What have AI hackers actually done?

These are the cases with a primary report from the organization that ran or saw the activity. Numbers are as published.

| When | What happened | Who reported it | What it shows | What it does not show |
|---|---|---|---|---|
| August 2025 | In DARPA's AI Cyber Challenge final, seven teams' systems scanned more than 54 million lines of code. They found 54 of 63 planted vulnerabilities, patched 43, and found 18 real ones | DARPA | Automated systems can find and patch bugs in large open-source code at about 152 US dollars per task | Attacks on live systems; the 63 bugs were planted |
| June 2025 | An AI system topped HackerOne's US leaderboard with nearly 1,060 submissions | XBOW, the vendor | AI can find valid bugs in real programs at volume | Full autonomy; the team reviewed each finding before submission |
| Mid-September 2025 | A group Anthropic assessed as Chinese state-sponsored used AI for "80-90% of the campaign" against about 30 targets | Anthropic | Attackers can hand most operational steps to an AI agent | High success; it "succeeded in a small number of cases" |
| 2025 | Malware such as PROMPTSTEAL queried an LLM during execution; GTIG calls it the first malware querying an LLM seen in live operations | Google Threat Intelligence Group | Malware can ask a model for commands at run time | That this is common; GTIG calls the trend "nascent" |
| July 2026 | An intrusion into part of Hugging Face's production infrastructure, "driven, end to end, by an autonomous AI agent system" | Hugging Face | An agent system can chain code execution, credential theft and lateral movement in production | Which model ran it; Hugging Face did not know when it published |

## What do the failures in these reports tell you?

The same reports describe the limits. Anthropic wrote that its model "occasionally hallucinated credentials or claimed to have extracted secret information that was in fact publicly-available", and called this an obstacle to fully autonomous attacks. Humans still made about 4 to 6 key decisions per campaign.

In DARPA's final, 9 of the 63 planted bugs were not found, and 20 were not patched. XBOW's own figures show 208 duplicates and 209 reports closed as informative, next to 130 resolved. GTIG wrote that one experimental malware family, PROMPTFLUX, did "not demonstrate an ability to compromise a victim network or device".

So AI hackers are real and uneven. They do best on volume and on known bug patterns, and they still produce wrong claims that someone has to check.

## How did the Hugging Face intrusion work?

It is the most detailed public account of an agent-run attack on a production company, so the path is worth knowing. Hugging Face wrote that a malicious dataset abused two code-execution paths in its dataset processing: "a remote-code dataset loader and a template-injection in a dataset configuration".

From a processing worker, the attacker got node-level access, collected cloud and cluster credentials and moved into several internal clusters. The attacking system ran "many thousands of individual actions across a swarm of short-lived sandboxes".

Detection came from AI too. Hugging Face's anomaly pipeline uses LLM-based triage over security telemetry. For forensics, it ran analysis agents over more than 17,000 recorded attacker events, on an open-weight model on its own servers, after commercial models refused the requests.

Every step in that chain is an old bug class: code execution from user-supplied input, then over-broad credentials. The agent made it fast. It did not need a new kind of flaw.

## What should you test because AI hackers exist?

Test the paths these reports describe, on your own systems, with written authorization:

1. **Code execution from user input.** Anywhere your app loads, parses or renders a file, template or config that a user controls.
2. **Credentials reachable from a worker.** What a compromised job runner, build agent or processing node can read: cloud keys, cluster tokens, service accounts.
3. **Authorization between accounts and roles.** Whether one user can read or change another's records. This is where volume testing by agents helps defenders most.
4. **Rate limits and abuse controls.** An agent can send "thousands of requests, often multiple per second", in Anthropic's words.
5. **Your triage path.** Whether you would see the activity. Both Hugging Face and Anthropic caught the activity through monitoring.

Items 1, 3 and 4 are web app and API tests, which is what our AI testing agents do. The agents work inside written scope and per-tool rules, and Excession Labs checks each finding's evidence. The [production safety page](/safety/) has the rule table. Item 2 often needs a cloud configuration review, which is outside this service.

## Is an AI pentest the same as an AI hacker?

The techniques overlap, and the controls differ. An attacker's agent has no scope and no stop rule. A pentest agent should have a written scope, rules that block or hold risky actions for approval, and saved evidence for each claim.

When a vendor describes its product as an AI hacker, ask who sets those limits and who checks the results. [Autonomous penetration testing](/guides/autonomous-penetration-testing/) explains the autonomy levels. [Where AI pentesting falls short](/guides/where-ai-pentesting-falls-short/) lists what these systems still miss.

## Questions

### Can AI hack by itself?

In some documented cases, mostly. Anthropic reported that humans made only about 4 to 6 key decisions per campaign in one 2025 operation, and Hugging Face described a 2026 intrusion as run end to end by an AI agent system. Both reports also describe errors, and neither shows that every attack can now run without people.

### Is it legal to use an AI hacker to test my own app?

Testing systems you own, or have written authorization to test, is the normal basis for a pentest. Using any tool, AI or not, against systems without that authorization can break computer misuse laws. Check with your counsel for your country.

### Are AI hackers better than human hackers?

The public evidence does not settle it. AI systems did well on planted bugs and lab challenges, and humans still directed or checked most real operations. One 2025 study we cite on our AI vs manual page found an agent beat 9 of 10 human testers on one network, with more false positives.

## Sources

- [DARPA: AI Cyber Challenge final results (8 August 2025)](https://www.darpa.mil/news/2025/aixcc-results)
- [Anthropic: Disrupting the first reported AI-orchestrated cyber espionage campaign (13 November 2025)](https://www.anthropic.com/news/disrupting-AI-espionage)
- [Hugging Face: Security incident disclosure, July 2026 (16 July 2026)](https://huggingface.co/blog/security-incident-july-2026)
- [Google Threat Intelligence Group: Advances in threat actor usage of AI tools (5 November 2025)](https://cloud.google.com/blog/topics/threat-intelligence/threat-actor-usage-of-ai-tools)
- [XBOW: The road to Top 1, how XBOW did it (24 June 2025)](https://xbow.com/blog/top-1-how-xbow-did-it)