How to Build the Future: Demis Hassabis
In a Nutshell
Demis Hassabis outlines unsolved AGI challenges like continual learning, long-term reasoning, and memory, predicting AGI around 2030 via scaling RL/search (e.g., AlphaGo ideas), agent systems, model distillation into efficient open models like Gemma, and multimodal Gemini for robotics/world models. DeepMind's breakthroughs like AlphaFold (now expanding to virtual cells and drug discovery at Isomorphic Labs) exemplify AI's potential to solve massive combinatorial problems in science, with similar transformations imminent in materials, math, and climate. Advice for builders: pursue deep tech intersections with AI, leverage 10-year journeys accounting for midway AGI, and target clear objectives with simulators for Alpha-style wins.
These notes were generated by AI and may contain inaccuracies.
Continual learning, long-term reasoning, some aspects of memory remain unsolved and are required for AGI. AGI timeline around 2030 means deep tech journeys must account for AGI appearing midway. Active systems that solve problems are needed to reach AGI; agents are the path and just getting started.
Demis Hassabis's career: chess prodigy, designed first hit video game Theme Park at 17, PhD in cognitive neuroscience, published foundational work on memory and imagination in the brain, co-founded DeepMind in 2010 to solve intelligence. Achievements: AlphaGo beat world champion at Go, AlphaFold cracked protein structure prediction (50-year grand challenge in biology, given away free to every scientist, won Nobel Prize in chemistry last year). Now leads Google DeepMind building Gemini toward AGI.
Components like large-scale pre-training, RLHF, chain of thought will be part of final AGI architecture; they've proven capabilities and won't be dead ends. Missing: continual learning, long-term reasoning, some memory aspects, system consistency. These may require scaling existing techniques with innovation or one or two big ideas (50/50 chance). Google DeepMind works on both.
Current continual learning uses duct tape like dream cycles. Brain integrates new knowledge via hippocampus during sleep, especially REM, replaying important episodes. DeepMind's first Atari program DQN (2013) used experience replay from neuroscience, replaying successful trajectories. Shoving everything into context window is unsatisfying; even with millions/tens of millions token context (perfect recall), lookup cost for relevant info is non-trivial.
Million-token context is plenty big for most uses but brute force stores unimportant/wrong info. For live video, 1M tokens is only 20 minutes; need more for understanding life over a month or two. Context window equates to working memory (humans: average 7 digits, few dozen max).
DeepMind historically leaned into RL and search (AlphaGo, AlphaZero, MuZero). RL potentially underrated, comes in waves. DeepMind worked on agents from beginning (Atari, AlphaGo, StarCraft with AlphaStar), generalizing to world models/language. Today's thinking modes/chain of thought pioneered by AlphaGo. Relooking at old ideas at scale: Monte Carlo search, augmenting RL on foundation models. AlphaGo/AlphaZero ideas relevant to advances next few years.
Build biggest models for frontier capabilities, then distill into smaller/faster ones (DeepMind invented distillation process). Strengths: packing power into smaller models quickly for Google products (search AI overviews, Gemini app, Maps, YouTube) serving billions of users with low latency/cost. Gemma models use distillation, amazing power for size (40 million downloads in 2.5 weeks).
No known informational limits to distillation yet; assumption: half a year to year after frontier model release, capabilities in tiny/edge models. Small models enable 500-1000x engineer productivity (e.g., thousandx work of 2000s Google engineer). Benefits: cost, iteration speed (90-95% capability gains back more via speed), edge deployment for efficiency/privacy/security (e.g., robots in house process audio/visual locally, delegate to cloud frontier models when needed).
Lack of continual learning holds back agents from full tasks; currently useful for aspects, patched together, but don't adapt to context. Need to learn specific context for fire-and-forget agents and full general intelligence.
Much innovation left in thinking paradigms; current approaches simplistic/brute force. Scope for monitoring chain of thought, interjecting midway (systems overthink, loop, e.g., in chess vs. Gemini: considers blunder, can't find better, plays it anyway). Huge gaps cause jagged intelligence: solves IMO gold medal problems but makes elementary math/reasoning errors. Missing introspection on own thought process; may need only one or two tweaks.
Agents just beginning; active systems essential for AGI. Experimenting to integrate into workflows for fundamental value (last couple months technology good enough). Running dozens of agents for 40 hours may not yet justify input. No AAA game topping app store charts fully agent-coded yet; prototypes fast (e.g., theme park prototype in half hour vs. 6 months at 17), but needs human craft/soul/taste. No kid-made hit game selling 10M copies yet; something missing (process/tools?). Expect results in 6-12 months: first 1000x humans/builders, then bestselling apps/games, then more automation.
AlphaGo's Move 37 inspired AlphaFold (started day after Seoul, 10 years ago; celebrating 10-year anniversary in Korea). Not enough to find novel moves; need systems that invent Go from high-level description (learn rules in 5 min, lifetimes to master, beautiful, playable in afternoon). Today's systems can't; maybe missing something, or just usage/creative person at one with tools.
Proponents of open source/open science (AlphaFold free, publish in top journals). Gemma: world-leading models for sizes, committed path (competitive vs. Chinese open models). Strategic: edge/nano models for Android/glasses/robotics best as open (vulnerable on surfaces anyway). Hope many build on it.
Gemini multimodal from start (harder initially than text-only, but long-term gains). Key for world models (Genie), robotics (Gemini Robotics), devices/assistants understanding physical world/intuitive physics/context (strongest on these). Used in Waymo.
The cost of inference is dropping fast. Inference will never be essentially free due to Jevons paradox; people will use all available compute. Imagine millions of agents or swarms working together, or single agents thinking in multiple directions and ensembling results. Experiments are ongoing with these approaches, all consuming available inference. One day energy costs could approach zero with fusion, superconductors, optimal batteries, or material science advances, but physical chip creation will remain a bottleneck for decades, requiring efficient inference use.
Smaller models are getting smarter.
AlphaFold 3 expanded beyond proteins to a broad spectrum of biomolecules. Isomorphic Labs, spun out from DeepMind after AlphaFold 2, is building beyond AlphaFold in drug discovery, including adjacent biochemistry and chemistry to design compounds with right properties. Big announcements coming soon.
Eventually, aim for a full virtual cell: a working simulation you can perturb, with outputs close enough to experiments to be useful, skipping search steps and generating synthetic data to train models predicting real cells. About 10 years away from a full virtual cell. Starting with a virtual nucleus on DeepMind's science side as it's relatively self-contained.
Key challenge: pick a slice of complexity at the right detail level that's self-contained, approximating inputs/outputs. Data is insufficient. Imaging a live cell at nanometer resolution without killing it would be game-changing, turning it into a vision problem. Current static electron microscope images are detailed but not dynamic. Solutions: hardware/data-driven or better learned simulators of dynamical systems.
All domains exciting: material science, drug discovery, climate modeling, mathematics. AI as ultimate tool for science to advance understanding, discovery, medicine, universe. Original DeepMind mission: step 1 solve intelligence (build AGI), step 2 use it to solve everything else (root node problems unlocking new branches).
AlphaFold prototypical: over 3 million researchers use it, nearly every biology researcher; almost every future drug will use it somewhere in discovery.
We're at an AlphaFold 1 moment in mentioned areas: promising results but grand challenges unsolved. Lots to announce in next couple years, including materials and mathematics.
Promethean capability, but be careful with use and misuse.
Advice for AI-Science Startups
Intercept AI tech trajectory. Combine with deep technology areas like materials or medicine involving the world of atoms—defensible from foundation model updates. Interdisciplinary teams essential. Nothing long-lasting is easy; drawn to deep tech.
AI was deep tech in 2010; dismissed by investors/academia. Need belief in why it's different this time, plus expertise in ML and target area (or founding team with it). Work on genuine passions; would have continued AI regardless.
Lessons from AlphaGo and AlphaFold: ideal for massive combinatorial search spaces (e.g., Go moves, protein configurations exceed atoms in universe) unsolvable by brute force/special algorithms.
Requirements:
- Clear objective function (e.g., minimize free energy, win Go) for hill-climbing.
- Enough data and/or simulator for in-distribution synthetic data.
With these, today's methods find needles in haystacks. Drug discovery same: find compound solving disease (no side effects, physics-allowed) efficiently. AlphaFold showed systems can find perfect moves.
Close to AI doing genuine scientific reasoning beyond pattern matching. Systems like co-scientist and alpha evolve go beyond basic Gemini; all frontier labs experimenting. Yet no massive discoveries (e.g., harder math than IMO).
Needs creativity, beyond known bounds: analogical reasoning, not pattern matching/extrapolation. Can it hypothesize interestingly? Solving Riemann hypothesis or Millennium Prize problems amazing (maybe 2 years out); harder: invent new worthy Millennium problems.
Not magical; missing one or two things. Einstein test: train on physics to 1901, see if it invents 1905 special relativity, photoelectric effect (annus mirabilis). Once passed, systems invent truly novel things.
Going after hard, deep problems no more difficult than shallow ones—just differently difficult. Life's short: put life force into world-changing work.
Interdisciplinary work rising, easier with AI; find field connections.
AGI timeline ~2030; deep tech is 10-year journey. Account for AGI midway: leverage it? General systems like Gemini/Claude will use specialized tools like AlphaFold (not in one brain—regression hurts generality; info efficiency). Train tool-using models with separate specialized systems.
Build useful in that world: physical factories, finance systems. Imagine AGI world and build accordingly.
De[m]is Hassabis everyone
Keep Y Combinator in your library
Save the videos and channels worth coming back to, and find them again in one place.
