ReadTube
Back to Sequoia Capital

Rich Sutton and Khurram Javed: Why AI Models Stop Learning, and How to Start It Again

Sequoia CapitalAugust 18, 202653m
Topics44
Learning as Continual Process0:00Rich Sutton Background1:01Conviction Behind Reinforcement Learning2:01Personal Health Crisis2:31Ordinary vs. Radical Thinking4:02Student Selection5:30The Bitter Lesson Origin7:01Essence of The Bitter Lesson8:31Large Language Models Assessment9:30Synthetic Data Critique11:00Physical World Complexity13:01Big World Hypothesis15:30Simulation Limitations16:31Agent-Generated Models17:30Human Knowledge and Learning18:01Prior Knowledge Relationship18:31Current AI Paradigm Critique20:02Future Robot Learning21:01Experiential Learning Systems22:01Current Assistant Limitations23:01Weight Updates vs. Context24:00Personalization Challenge24:31Context in Weights vs State25:04Inspiration from Biological Learning26:00School as Non-Essential to Intelligence28:30Unified World Models30:30Paradigm Shifts and Planning33:00The Alberta Plan36:00Abstractions in Elite Performance38:00Algorithmic Requirements for Continual Learning39:00Step Size Optimization and Generate-and-Test40:00Continual Backprop Algorithm41:00Application to Foundation Models42:00Research Scope and Ambition43:00Energy Efficiency Targets45:00Local Minimum Problem47:30Company Vision49:00Large Language Models and Their Limitations50:04The Scope of Intelligence Beyond Language50:30Single Design vs. Multiple Minds51:00The Big World Hypothesis and System Limitations51:30Hiring Approach for New AI Development52:01Alignment and Productivity Goals52:31Conclusion of Discussion53:00
In a Nutshell

Rich Sutton argues that AI systems fundamentally need to keep learning after deployment through ongoing weight updates rather than relying on static models like current LLMs. The core problem is that existing approaches depend on finite human-generated data and human-designed simulations, which creates an insurmountable bottleneck because the world is infinitely complex and requires continual learning from direct experience. The solution involves developing new algorithms like continual backprop that enable systems to learn from single streams of experience without catastrophic forgetting, allowing them to form abstractions and plan with self-discovered knowledge.

AI-Generated Notes

These notes were generated by AI and may contain inaccuracies.

Rich Sutton argues that the concept of "continual learning" is only necessary because the field has become weird. In normal times, learning would always be understood as continual. The ordinary way of thinking is that we always act and learn simultaneously.

Rich Sutton invented reinforcement learning, wrote the seminal textbook, and mentored key figures like Dave Silver. He authored "The Bitter Lesson," considered the Bible of the field. He co-founded Oak Lab with Khurram Javed, his former student from University of Alberta.

Sutton chose reinforcement learning because learning and having goals are central parts of the mind and intelligence. He was doubling down on what he had always been thinking about.

In 2003, Sutton was dying of cancer when he decided to take a job at University of Alberta. He continued working on research during this period because of habit. He references Benjamin Franklin's observation that people do things either out of habit or vanity.

Sutton maintains he is not radical or weird. The field is weird for needing to specify "continual learning" when all learning is naturally continual. He notes that a decade ago, learning was understood as important, goals were central, and perception mattered. Before AI craziness, no one would need to specify continual learning.

Sign in to read the full notes

Get access to AI-generated notes, topic timestamps, and more.