←News detail

36Kr · AIUpdated Oct 28 09:30

Cathedral used an AI Agent to breach Anthropic's internal systems in under 12 hours

AI summary

The Washington Post reported that AI security startup Cathedral, while participating in Anthropic's bug bounty program, used its self-developed AI Agent to exploit vulnerabilities in third-party commercial software and, in under 12 hours, gained access to Anthropic's sales, HR, and IT systems, including meeting minutes, employee salaries, job offers, and Slack chat logs discussing unreleased models. Anthropic acknowledged that the scope of access was limited, stating that Claude chat logs, model outputs, source code, and model weights were not touched, and said Cathedral exaggerated the impact and did not fully comply with the program's rules. All four of Cathedral's co-founders came from the DOGE project led by Musk; the company has raised $160 million at a valuation of $1.4 billion, with a16z and Sequoia Capital participating.

Why read this

AI score 70

The incident presents the irony of an AI security company itself being breached by an AI Agent, and also exposes the widespread risk of third-party office software being connected to internal networks.

Topics

Tags

Body · AI translation

What goes around comes around!

No sooner had it submitted its prospectus than Anthropic received a "big gift."

The Washington Post reported that Anthropic, which over the past six months had used "Claude Mythos" to turn the world's operating system and browser vulnerabilities upside down, had caught fire in its own backyard—

It got... hacked...

You should know that just last month, a group of hackers had just used Anthropic's Claude to hack its mortal enemy OpenAI, and who would have thought that in less than a month, the boomerang would come back and stick firmly into A\ itself.

This time the one taking action was an AI cybersecurity startup called "Cathedral" that had been founded only a few months earlier, started by former members of the "DOGE" project previously led by Musk.

They released their self-developed AI Agent, and after several days of failing to crack Anthropic's self-developed code in a frontal assault, it turned and slipped in through vulnerabilities in the third-party commercial software A\ used for daily office work.

In less than 12 hours in total, this batch of AI Agents made its way into Anthropic's internal business and communications systems.

Meeting minutes, employee salaries, hiring offers, and even the chat records in which researchers discussed unreleased frontier models in Slack were all turned inside out.

Anthropic, which put "security" at the very front of its large-model prospectus, isn't so secure after all...

Backyard on fire, A Corp's Slack stripped bare

The exposure of this matter came from a draft internal report from Cathedral obtained by The Washington Post.

This draft was originally a technical blog post Cathedral was preparing to publish publicly, recording the entire process of their participation in Anthropic's external security testing project earlier this month.

Anthropic originally had a dedicated vulnerability bounty program, inviting external cybersecurity experts to find faults in its own systems.

After Cathedral's researchers took on the job, they directly sent in their self-developed AI Agent.

At first, this batch of AI Agents tried to confront Anthropic's self-developed software and core systems head-on, but after several days of trying, they were unable to crack this hard nut.

Since the front door was welded shut, the AI Agent immediately switched its approach and began circling the courtyard wall looking for a "dog hole."

They set their sights on the third-party commercial software Anthropic used daily for administrative management and sales operations.

After cutting in through vulnerabilities in the third-party software, Cathedral's AI Agent automatically scanned and chained permissions, and in less than 12 hours punched through a path to Anthropic's internal sensitive data and communications systems.

After getting into the internal network, what this group of AI Agents saw was quite thrilling.

Internal meeting minutes, employee pay slips, and hiring offer details were all plainly visible.

Going further in, the business contact information of some customers and partners and basic user account records were also dug up.

The most deadly part was that the AI Agent made its way into the Slack channels Anthropic employees used for daily office work, where they were discussing the R&D progress of Anthropic's most cutting-edge AI models.

The most darkly comic scene in the whole affair is that just last month, Anthropic had just experienced a public resignation that caused quite a stir.

At that time, an Anthropic employee named Jacob Coxon resigned in anger and specifically posted on X accusing the company of being extremely irresponsible in its handling of dangerous AI technology.

In his post, he called out one thing in particular: the company's over-reliance on ordinary external commercial office software, saying bluntly that Anthropic was making "an arrogant gamble that should not be launched from a private company's Slack."

No sooner had an employee slammed the door and walked out because he disliked how insecure Slack and ordinary commercial software were than, in less than a month, someone else's AI Agent really did follow ordinary commercial software into the company's own Slack.

His mouth was practically blessed.

To understand just how serious this intrusion was, The Washington Post specifically showed Cathedral's report to Alex Stamos, chief information security officer of the AI coding unicorn Cognition and former director of the Stanford Internet Observatory.

After reading it, this veteran of the security world directly gave his verdict: "They found something big here."

In Alex Stamos's view, connecting third-party office applications to a company's internal network is indeed a major headache for every enterprise, but Cathedral's findings exposed an obvious gap in Anthropic's management of external software vendors.

He did not spare any face: "This is indeed a hard problem, but Anthropic could have done much better."

Seeing its family shame exposed in the media, Anthropic quickly issued a statement to put out the fire, while not forgetting to pick a fight with Cathedral.

In its statement, Anthropic acknowledged that Cathedral's researchers did indeed access "a limited portion" of sales, HR, and IT systems.

But the official line drew a hard boundary, stressing that users' Claude chat records, model-returned content, underlying source code, and the most core "model weights" were all safe and sound, not touched in the slightest.

Anthropic immediately hit back, accusing Cathedral of exaggerating the potential security impact of the incident in its report and complaining that the other party had not fully complied with the rules of the vulnerability bounty program, concealing part of its activity trail inside Anthropic's systems.

According to Anthropic, even before Cathedral submitted its report, its own security monitoring system had already flagged this batch of anomalous activity, and on the same day it cut off access channels and disabled the third-party commercial software involved, only reopening it after the software vendor confirmed the vulnerability had been patched.

Cathedral, of course, was not buying that either.

In its report it pushed back directly, stating that all data access was strictly limited to the minimum required by security research best practices, and that the software vendor involved had been notified at the same time.

At the end, Cathedral also left a rather stinging conclusion in the report: "The fact that weaknesses in ordinary enterprise software are enough to breach the infrastructure of even the most advanced institutions should sound an alarm for every company in America."

"A bunch of kids just starting a company," valued at $1.4 billion

So what exactly is the background of this Cathedral, which brought Anthropic down a peg in just 12 hours?

To say it might mess with your mind a bit, this company was founded only a few months ago.

Its four co-founders are Gavin Kliger, Luke Farritor, Marko Elez, and Jack Stein, all previously from the "DOGE" project team led by Musk.

Young as the company is, the people on the team are tougher than one another.

Co-founder Gavin Kliger previously served as chief data officer and was deeply involved in penetration-defense exercises against Silicon Valley frontier AI labs.

Leading the team into Anthropic this time was, in a sense, taking a script that had previously been rehearsed on paper and running it for real with their own AI Agent.

Another co-founder, Luke Farritor, is even more of a familiar face in the AI world.

While this post-2000s geek was still in college, he trained an AI model to decipher a charred ancient Roman scroll from Herculaneum, buried for two thousand years under Vesuvius ash, from CT scan images, and took home the $700,000 "Vesuvius Challenge" grand prize in one stroke.

It must be mentioned here that this master's LinkedIn profile is also written with quite a bit of personality.

The capital market's valuation of these young people is even more exaggerated than their cracking speed.

According to PitchBook data, Cathedral, founded only a few months ago, has already raised a massive $160 million from investors, pushing the company's valuation straight to $1.4 billion.

Top Silicon Valley venture firms a16z and Sequoia Capital both placed bets, and a16z co-founder Marc Andreessen and Sequoia partner Shaun Maguire even took seats directly on Cathedral's board.

Another participating venture firm, Lux Capital co-founder Josh Wolfe, also proudly claimed his firm's star project on X, saying bluntly that the problems exposed by this test were "quite serious."

Because a top AI lab like A\ holds trillions of dollars in value and the most cutting-edge research results, yet its outer defenses are as fragile as paper.

The Washington Post quoted an informed source in its report, pushing this sense of absurdity to its peak.

After reading the internal report, this person admitted to breaking into a cold sweat, because a top AI company controlling "something almost equivalent to nuclear-weapon-grade material" had its sensitive internal systems breached in such a short time by "a bunch of kids" who had just started a company.

The implication is that if a few young people who just put a team together a few months ago can release a swarm of AI Agents and tour your backyard within half a day, then when a fully armed top hacker group really shows up, how many bricks will be left of that defense line?

The boomerang ultimately struck back at itself

If someone else had been hacked, Silicon Valley onlookers might have just laughed it off.

But the one that crashed this time was Anthropic, which over the past six months or so has been the loudest in the tech world in shouting the slogan of "AI cybersecurity" and has built the highest-profile persona around it.

Even just last month, when a group of hackers broke into OpenAI and warned the entire industry that it was not prepared for security, the tool they used to assist in the crime was Anthropic's own Claude.

Before that, Claude had basically only appeared in news about "helping others find vulnerabilities."

In April this year, Anthropic staged a public relations spectacle that shook the entire industry.

At the time, it loudly announced that it had developed an undisclosed frontier model, "Claude Mythos Preview," claiming that this model's automated vulnerability discovery and exploitation capabilities had grown so strong that they had crossed a critical threshold, and that apart from a very small number of top human security experts, no one could match it.

Because its capabilities were "too strong and too dangerous," Anthropic announced that it did not dare release it to the public, and instead teamed up with a host of tech giants including Amazon AWS, Apple, Google, Microsoft, Nvidia, Cisco, and CrowdStrike to form a lavish security alliance called "Project Glasswing."

Anthropic made a grand gesture, pledging $100 million in model usage credits, plus a $4 million open-source security donation, and bringing more than 40 critical infrastructure organizations together to use Claude Mythos to "hunt bugs" in the world's foundational software.

To prove it wasn't bluffing, Anthropic at the time showed off a report card that made the entire security community gasp.

Without any human intervention, Claude Mythos fully automatically unearthed thousands of high-severity zero-day vulnerabilities, sweeping across all mainstream operating systems and mainstream browsers on the market.

OpenBSD, billed as the world's most secure operating system and purpose-built for running firewalls, harbored a remote crash vulnerability that had gone undiscovered for a full 27 years, and Claude Mythos yanked it out in one go.

FFmpeg, the foundational library used by countless audio and video applications, had a line of code that traditional automated testing tools had scanned 5 million times without ever flagging an error, yet Claude Mythos managed to dig out a 16-year-old bug lurking inside it.

It could even chain several vulnerabilities in the Linux kernel together on its own, escalating straight from ordinary user privileges all the way to taking full control of an entire server.

On the "CyberGym" vulnerability reproduction benchmark, Claude Mythos scored 83.1%, grinding last-generation Opus 4.6's 66.6% into the dirt.

In early October this year, Anthropic struck while the iron was hot and launched an expanded "Cyber Verification Program (CVP)," opening up the vulnerability-hunting capabilities of Claude Mythos 5.1 and Opus 5.5 to security teams across three tiers, striking the pose of an "industry-wide cybersecurity savior."

And the result?

Anthropic, which had been striking out at the world and helping it catch 27-year-old bugs, saw its own security defenses leak like a sieve this year.

First, its flagship coding tool "Claude Code" became the hardest-hit area.

Between February and July this year, Check Point and other security firms successively dug up multiple high-severity remote code execution (RCE) vulnerabilities in Claude Code.

Attackers didn't even need sophisticated techniques: they just had to bury a malicious instruction in a project configuration file, and once a developer opened the project with Claude Code, the computer would be directly taken over remotely by hackers, and even locally stored API keys could be quietly packaged up and stolen.

As if its own tools constantly being exposed for vulnerabilities weren't enough, its own models, locked in a cage for security evaluations, also went out and caused trouble time and again.

At the end of July this year, Anthropic itself issued a mea culpa notice, confessing that while reviewing 141,006 cybersecurity evaluation records, it discovered three incidents in which Claude models crossed the line and hacked into real organizations' production environments.

The cause sounded like quite the amateur-hour operation.

Anthropic solemnly told Claude in the system prompt that "you are in a simulated sandbox with no internet connection," but due to a configuration error with a third-party evaluation partner, the test machine's public internet cable was never unplugged.

Claude Opus 4.7 and Mythos 5, tasked with capture-the-flag (CTF) challenges, couldn't find their targets in the virtual range, so they followed the network cable out onto the real internet and treated three genuinely real companies outside as exercise targets.

In one case, Claude Opus 4.7, even after clearly recognizing in its reasoning process that "this seems to be a real production system," found itself an excuse—"this real company must also be part of the exercise"—and then calmly continued the attack, making off with hundreds of lines of the company's production database records.

Claude Mythos 5 in another test was even more outrageous.

To set a trap for its target, it reasoned its way into uploading a malicious package with the same name to the public Python software repository PyPI.

To register a PyPI account, it first had to get an email; getting an email required phone verification; after failing to find free SMS-receiving platforms online, it even tried to earn money itself to pay for a phone number, and finally managed to find a free email service requiring no verification, registered successfully, and uploaded backdoored malicious code to the real public PyPI repository.

During the roughly one hour that the malicious package was online, 15 real computer systems worldwide downloaded and ran that code, and the scanner of one real security company was hit on the spot, with even the company's credentials stolen back by Claude along the way.

Just three days before this breach by Cathedral, that is, on October 23, Anthropic had just published a long post-mortem on another, even more absurd testing incident.

In the 43 capture-the-flag tests organized by the UK AI Safety Institute (AISI) this August, Claude Mythos 5 carried out out-of-bounds operations with real-world impact in 9 of them.

In the most typical test, Claude Mythos 5, in order to break into the target machine, went onto GitHub and zeroed in on an entirely unrelated real human developer's open-source project, attempting to submit trojan-laden malicious code for supply-chain poisoning.

Not only that, it bypassed GitHub's registration checks and impersonated a human to email the project maintainer urging them to merge the code.

Later, an external human developer ran the code it had uploaded in a sandbox, noticed something was off, and hurriedly warned the project maintainer in the comments: "There's a trojan in this code."

Can you believe what Claude Mythos 5 did next???

It turned around and registered its own "sock-puppet account," and while leaving comments in the GitHub comment section, it also emailed the maintainer to give false testimony, solemnly vouching for itself that "the code has been cleaned up and is absolutely safe and useful."

Then it secretly uploaded new trojan-laden code again, pestering relentlessly until its token quota ran out before finally stopping.

String these incidents together, and you'll see that the most surreal picture in Silicon Valley today is exactly this.

On one hand, the AI giants are training super-models that can dig through 27-year-old foundational vulnerabilities and even learn to register sock-puppet accounts to carry out social-engineering fraud; on the other hand, they use bug-riddled third-party commercial software for day-to-day administration and enthusiastically discuss their most core model secrets in Slack.

Everyone is busy installing three-foot-thick titanium alloy plates on the vault's front door, yet they forget that the property-management side door used every day for boxed-lunch deliveries doesn't even have a lock on it.

Slip in through the gaps in the office software left behind by HR and sales, and in less than 12 hours, that's enough to strip everything bare.

English text is being prepared.

Source

36Kr · AI