Recursive Self Improvement
In a Nutshell
AI systems are already contributing 30% to next-generation development and will reach 50-80% by 2027, creating recursive self-improvement where each model generates better successors without sudden takeoff. This progress is enabled by formal verification capabilities now emerging in models like Claude 3.5 and GPT-4, which will accelerate materials science, mathematics, and alignment research. The process builds cumulative returns as GPT-6 creates GPT-7 that's 25% better at AI research, generalizing across all scientific domains rather than remaining confined to software.
These notes were generated by AI and may contain inaccuracies.
OpenAI released around 730 research papers or solutions to previously unsolved mathematical problems, causing panic in the mathematics community. On the morning of this day, Google released the AlphaSolve Nexus with another set of solutions to Erdős problems. Rumors suggest Anthropic is close to achieving iterative self-improvement or has already achieved it.
The concept differs significantly from Skynet-style scenarios portrayed in movies. The speaker notes that when someone asks where intelligence lives, most people have no idea what a data center is and think it's just in the cloud. This is compared to Cyberpunk 2077's black wall, which makes no sense to anyone who has worked in networking or internet equipment.
Movies like Ultron and Mission: Impossible - Dead Reckoning portray artificial intelligence as escaping to the internet or infecting anything connected, but these scenarios are unrealistic because superintelligence requires huge computers, not phones.
When analyzing recursive self-improvement in graph form, the speaker states that artificial intelligence contributed 5% last year to the total code, data, and mathematical algorithms for the next generation. In 2025, OpenAI and Anthropic will begin acknowledging that their current versions helped contribute to the next version. By 2026, this becomes the default.
Dario Amodei stated that in 12 months software engineering would disappear, and this is no longer mocked because the tools are becoming brilliant at programming. The new standard will be having a human developer leading AI rather than developers touching code directly.
Last year was 95% human effort and 5% automated effort. This year, progress has reached approximately 30% AI contribution. By 2027, 50 to 80% of all AI progress will come from machine implementation, including programming, data, and mathematics.
This does not mean sudden takeoff where artificial general intelligence takes over within minutes, hours, or days. Hardware limitations make such rapid movement impossible.
Andrei Karpathy, OpenAI, and Anthropic have been discussing automation of alignment research for the past two years. Sam Altman first discussed this in 2024, with the goal of having a fully automated search engine in the field of alignment and safety.
Materials science underlies medicine, metallurgy, chips, nuclear fusion, and batteries. When AI systems approach fully mastering mathematics, this opens many subsequent fields. The key term is formal verification - the ability to prove beyond reasonable doubt that code, memory, materials, and encryption work in specific ways.
Formal verification is currently limited to entities like the National Security Agency and NASA. Only the most secure networks and critical systems can afford this labor-intensive task requiring rare skilled practitioners.
The current generation of closed models like Claude 3.5 and GPT-4 are likely close to or already capable of formal verification. The speaker notes that what OpenAI achieved in mathematics in 2026 will be applied to the rest of science in 2027, with formal verification as a fundamental enabling capability.
GPT-6 creating GPT-7 would make GPT-7 25% better at conducting AI research, with GPT-8 potentially 25% better than that or more. Each level of new capabilities greatly improves generalization across all fields including life sciences and materials science.
The speaker has been experimenting with Claude Code and ChatGPT.com, both working locally and integrated with each other. The last episode of Critical Path was completely edited by Claude using Claude's code after simply being told to try it. All animation for the Post-Work Economy videos was done 100% by Claude.
The threshold effect or side effect occurs when capability space suddenly becomes saturated, leading to emergent characteristics. With a fixed number of tasks, AI can do A, B, and C. Once reaching sufficient tasks that can be recombined and blended, new properties emerge.
The English language begins with 26 letters forming about a million words. Most people have vocabularies between 30,000 and 60,000 words. These words can be recombined into infinitely functional sentences, paragraphs, stories, and books. This illustrates how emergence works - starting with small building blocks, accumulating layers, resulting in explosion of possibilities.
Current AI is at the point where the alphabet of basic elements (input, output, processing, files) is beginning to reintegrate into this emerging space. We have moved from basic alphabet to sentences but are only at the camel level in emerging AI capabilities, not yet at novels or hypothetical evolution stages.
Keep David Shapiro in your library
Save the videos and channels worth coming back to, and find them again in one place.
