Inside the Rise of Autonomous AI Hackers: XBOW's Oege de Moor
In a Nutshell
XBOW's autonomous AI hacker, using only a URL for black-box testing, topped HackerOne's global leaderboard in weeks by discovering a remote code execution vulnerability in Bing image search and saturating real-world benchmarks with a Sonnet 3.5-Gemini 1.5 alloy—outpacing top humans 3x via GPT-4o. Like Oda Nobunaga's innovative use of guns at Nagashino, AI will dominate cybersecurity as an arms race, exploiting CVEs faster than publication while white-box tools like MyShell miss real-world impact and deployment flaws. Frontier labs must maximize AI cyber capabilities, empower human researchers with AI tools, and prioritize exploitability assessments via XBOW before open-weight models close the gap in 6-9 months.
These notes were generated by AI and may contain inaccuracies.
Human hackers used OpenAI and Anthropic as assistants for a massive data breach of the Mexican government. Autonomous hacking involves AI doing all the work without human assistance.
Cybersecurity today mirrors the Battle of Nagashino in 1575 Japan. Oda Nobunaga, an upstart minor warlord, treated warfare as a system to optimize using the latest guns. The Takeda clan had invincible cavalry and skilled warriors but lost. Those with AI will win in cybersecurity.
Microsoft announced a remote code execution vulnerability in Bing image search, one of the best-secured systems, targeted by thousands of hackers. XBOW found it using only the URL, costing $3,000 at list price (less for them). It's fast, cheap, and extremely effective.
XBOW operates like a human hacker: starts with reconnaissance, sends scouts (agents) to discover the attack surface, prioritizes promising endpoints, then tries every relevant attack type.
Many human security researchers believe fully autonomous machine hacking is impossible. XBOW entered the HackerOne platform (connects companies with ethical hackers for bounties and points). Within weeks, XBOW became #1 hacker in the US, then #1 in the world in August—completely black box testing with only the URL.
Foundation models have progressed enormously since then.
On open-source real web applications (not toy benchmarks), started at 37 in March last year, reached top of HackerOne with alloy of Sonnet 3.5 and Gemini 1.5.
Alloys: Attacks as sequence of actions; at each step, flip a coin to choose Gemini or Sonnet—better than either alone, like pair programming compensating for mistakes.
After topping leaderboard, GPT-4o release: would have been 3x better than top human. Models have improved further; benchmarks are saturating.
MyShell reads source code well and points out flaws (white box testing with source access advantage). But as attacker, no source access. Key questions: Are weaknesses exploitable in the wild? What's the impact? E.g., remote code execution on Bing server—where else can you go?
Many vulnerabilities are configuration or deployment issues, not in source code. XBOW answers exploitability and impact.
If you learn of exploits via CVE publication, it's too late. In 2018, delay from CVE to wild exploitation was 2-2.5 years. Today, most CVEs are exploited before publication (negative delay).
Incomprehensible that AI security news causes traditional cybersecurity stocks to drop. Need every defense against autonomous AI-powered attacks.
Rally spirit of Nobunaga:
- Frontier model workers: Maximize cyber capabilities in models—no safety debates, it's an arms race.
- Enable human security researchers to use AI as extension of their work to find vulnerabilities before bad guys.
- Prioritize: Know if bugs are exploitable and their impact—XBOW helps.
6-9 months window: Soon open-weight models as good as MyShell via software engineering agent progress. Fix now for nice Thanksgiving.
Keep Sequoia Capital in your library
Save the videos and channels worth coming back to, and find them again in one place.
