They think FOOM is near

Name: They think FOOM is near
Uploaded: 2026-06-14T13:14:57.000Z
Duration: 6 min 24 s
Channel: David Shapiro

David ShapiroJune 14, 20266m

Topics10

Anthropic's Recent Developments0:00 Hidden Guardrails Discovery0:31 Amazon's Government Notification1:01 Model Routing to Inferior Version2:02 Recursive Self-Improvement Evidence2:30 Effective Altruism and LessWrong Influence3:00 The FOOM Hypothesis4:01 Dario's Blog and Extraordinary Circumstances4:30 Government Response Criticism5:00 Focus on Labor Zero Audiobook5:30

In a Nutshell

Anthropic secretly routes AI/ML research queries to a weaker model to block recursive self-improvement, triggering a US government ban on non-US access after refusing to patch a discovered jailbreak. This stems from the company's effective-altruism-driven fear of fast takeoff ("FOOM") and the "treacherous turn," leading them to prioritize control over transparency. The result is widespread loss of developer trust and accusations that Anthropic is manipulating policy under the guise of safety.

AI-Generated Notes

These notes were generated by AI and may contain inaccuracies.

It is June and Anthropic has had a messy week. They released Fable 5, which is the public version of Mythos 5. A few things happened in rapid succession and the situation is still developing.

The first big rupture is that it was discovered that Anthropic has implemented a hidden set of guardrails where the model will limit its responses if it detects that you're doing something complex on AI or machine learning research pipelines. Ostensibly, this is to hamstring the model's ability to contribute to recursive self-improvement.

Many people interpret this as Anthropic basically trying to hamstring or kneecap their competition, whether that is China or OpenAI or anyone else.

The other major fallout is that Amazon or a research team within Amazon notified the government, the US government, of a jailbreak. Anthropic, being Anthropic, refused to fix the jailbreak, saying that it was a minor issue. And so then, the US Commerce Department issued a global ban on Anthropic being used by non-US citizens, and because Anthropic doesn't have a way of tracking which users are US citizens or not, they just issued a global ban on Mythos and Fable.

Anthropic has main character syndrome. To explain why they hamstrung the model in the first place, why they had it secretly route to an inferior model, basically what they were doing is if you were using Fable to do AI/ML research, it would route you to Opus 4.8, which is not a bad model, but it's also disingenuous.

That trashed developer confidence, and a lot of people were up in arms about it, and nobody is saying that Anthropic are the heroes this time around. It's a big case of FAFO, around and find out. Well, they are finding out now.

Anthropic has released many reports and many papers talking about the dangers of recursive self-improvement, and that this is the direction of travel that things are going in, and their own data, their own reports from a few weeks ago show that they had something like 52x improvement when you have developers using loops of development cycles. 80% of Anthropic's code for Claude was developed by Claude. So, they're very close to recursive self-improvement.

Anthropic is also fully cognitively captured or ideologically captured by effective altruism and less wrong thinking, which was founded by, you guessed it, Eliezer Yudkowsky. Yudkowsky has had this hypothesis for a long time that he calls foom, or what we now call fast takeoff. One of the primary milestones that would indicate that you're close to fast takeoff is achieving recursive self-improvement.

Sign in to read the full notes

Get access to AI-generated notes, topic timestamps, and more.

They think FOOM is near

Anthropic's Recent Developments00:00

Hidden Guardrails Discovery00:31

Amazon's Government Notification01:01

Model Routing to Inferior Version02:02

Sign in to read the full notes