Anthropic just proved AI isn't getting better
In a Nutshell
Anthropic's latest model (Opus 5) exemplifies a fundamental AI limitation: it excels at quick, surface-level code edits via simple pattern matching but lacks architectural understanding of codebases, making decisions based on minimal context rather than accumulated experience. This stems from intrinsic properties of how LLMs function—compression losses, human feedback training artifacts, and inability to truly comprehend system-wide implications—meaning these problems won't be solved by future model iterations. The speaker uses this to argue AI won't reach AGI or replace human engineers, while promoting their course on post-development skills like marketing and storytelling.
These notes were generated by AI and may contain inaccuracies.
Everybody hates Opus 5 right now, but this issue extends far beyond any single model and represents a fundamental limitation that no model will ever fix. The speaker explains that while Opus 5 performs trivial tasks like renaming variables extremely well, it then generates unnecessary commentary such as "one thing worth knowing" before understanding what the actual thing is. This behavior mirrors an annoying business casual person in meetings who must say something to buy time while thinking, exemplified by the phrase "So here's the thing, guys" followed by obvious statements like "Renaming things can affect other places where that thing is named."
This behavior stems from human feedback during post-training, where annotators in third-world countries kept giving thumbs up to responses containing "it's worth noting that," permanently imprinting this personality trait onto the model. The problem is that these caveats sometimes contain valuable information, so users cannot simply ignore them and must read every single one.
The speaker has been building a project using AI extensively over several weeks to observe the technology firsthand. While AI is fast at implementing features once it understands the intent, there exists a deep, intrinsic limitation that won't be solved by future models like Fable 6, 7, or 10.
When given serious tasks that would take a human a week to reason through and build, the AI thinks for approximately eight seconds before beginning to edit files. The critical issue is that half of software engineering involves determining where and what to edit, which is called architecture. The AI selects files through simple grep searches, reading minimal files to complete changes quickly rather than understanding the broader codebase implications.
Sign in to read the full notes
Get access to AI-generated notes, topic timestamps, and more.