Building the Automated AGI Lab: Core Automation's Jerry Tworek and Rohan Anil
In a Nutshell
Core Automation was founded because current transformer-based systems cannot learn or adapt at test time on new real-world tasks, and the founders believe the architecture itself—not just scaling—is now the bottleneck to AGI. Their mission is to automate the entire AI research lab, particularly kernel optimization, to enable rapid architecture experimentation at scale. The key insight is that models need new learning algorithms capable of meta-learning and continual adaptation, rather than relying on pre-training plus RL alone.
These notes were generated by AI and may contain inaccuracies.
Football learning resembles reinforcement learning through repeated attempts with incremental adjustments and self-reinforcement based on outcomes. Mathematical learning operates differently, involving reading complex concepts and deep internal thinking until connections form. Both represent learning from experience but through fundamentally different mechanisms.
Current AI systems likely allocate more compute to learning from experience than ever before, though reinforcement learning represents only one approach. Better methods for utilizing experiential data will emerge from research in coming years.
Jerry Tworek previously served as VP at OpenAI, leading the strawberry and reasoning teams. Rohan Anil held one of four pre-training lead positions at Gemini and conducted fundamental AI research at Google Brain, serving as the fix-it person across Google and later at Anthropic.
Jerry's statement that "the first step to replacing transformers is appreciating deeply how far they were able to carry us" emphasizes understanding both capabilities and limitations. Appreciating transformers requires recognizing what they solve effectively before addressing their weaknesses.
Current architectural work focuses predominantly on making transformers cheaper and more efficient rather than more powerful or expressive. Understanding the shape of transformers—both strengths and weaknesses—reveals the same underlying analysis.
Sign in to read the full notes
Get access to AI-generated notes, topic timestamps, and more.