Building AlphaGo from scratch – Eric Jang

Name: Building AlphaGo from scratch – Eric Jang
Uploaded: 2026-05-15T16:20:46.000Z
Duration: 2 h 37 min 17 s
Channel: Dwarkesh Patel

Dwarkesh PatelMay 15, 20262h 37m

Topics27

Introduction to Eric Jang and AlphaGo Project0:00 How Go Works2:04 Game End and Scoring5:01 AlphaGo Overview and Search Intuition8:04 Monte Carlo Tree Search (MCTS) Basics12:00 Backup and Value Propagation22:00 MCTS in Practice28:06 Flashcard Generation Pipeline30:47 Value and Policy Networks32:03 Network Architecture Details37:32 Neural-Enhanced MCTS43:05 MCTS as Policy Improvement58:36 Jane Street Data Centers59:00 RL Self-Play1:00:30 MCTS Policy Convergence1:01:01 AlphaGo Self-Training via Distillation1:02:32 Analogy to DAgger in Robotics1:06:00 Profundity of AlphaGo1:16:04 MCTS vs Naive RL1:24:36 Variance of the Gradient in Multi-Step RL1:30:24 Neural Fictitious Self-Play1:37:30 Scaling Laws and Test-Time Compute1:51:03 Auxiliary Supervision and Initialization in KataGo1:59:45 Outer Loop Signals and Discovering Deep Learning2:29:20 Bitter Lesson, Compute Scaling, and Heuristics2:31:03 Positive Transfer from Games to Broader AI2:33:32 Resources and Closing Thoughts2:35:32

In a Nutshell

Eric Jang rebuilt AlphaGo from scratch using modern compute, achieving strong Go bots for ~$10K via neural-enhanced MCTS—combining policy/value networks with PUCT-guided search for efficient policy improvement through self-play and distillation, far cheaper than DeepMind's original millions. MCTS elegantly sidesteps RL's high variance by iteratively relabeling states with search-improved soft targets, enabling stable supervised learning that hill-climbs win rates without exploration failures or long-horizon credit assignment issues plaguing naive policy gradients in LLMs. Key takeaways: initialize with expert/small-board data, prioritize bug-free baselines before scaling laws, and leverage LLMs for hyperparameter tuning/experiments while humans supply research taste for track selection.

AI-Generated Notes

These notes were generated by AI and may contain inaccuracies.

Building AlphaGo from Scratch – Eric Jang

Eric Jang was most recently vice president of AI at 1X Technologies, and before that, senior research scientist at what is now Google DeepMind Robotics. He's been on sabbatical for the last few months, rebuilding, improving, and hacking on AlphaGo. The discussion covers building AlphaGo from scratch and what it tells us about the future of AI research and development.

Why AlphaGo is interesting: AlphaGo got Eric into the field. Early breakthroughs in 2014, 2015, 2016 showed how smart AI systems could become and the computational complexity class they could tackle with deep learning. Go was long understood to be intractable for search, yet solved through deep learning, which was mysterious.

Eric's training is in deep neural nets for robotics, where decisions are intuitive. But AlphaGo involves very deep search, mysterious how a ten-layer network amortizes simulation of something deep in the game tree.

Plot of compute for strong Go bots: In 2020, open-source KataGo by David Wu from Jane Street achieved 40x reduction in compute to train a strong Go bot tabula rasa. Not certain if stronger than AlphaGo Zero, AlphaZero, or MuZero, but very strong; most Go practitioners train against it today.

Thanks to LLM coding, what took DeepMind team, millions of dollars, research and compute, now done for few thousand dollars of rented compute.

Go is simple game, easy to implement on computer. Objective: put down black and white stones to occupy as much territory as possible.

Black always goes first. To capture opponent's stones: surround all four neighbors of an intersection; stone cut off from oxygen, dead.

Slight variations: Chinese, Japanese, Tromp-Taylor rules. Tromp-Taylor designed completely unambiguous; all Go AIs train and resolve against it.

In typical Go, humans can't put stone in instant suicide position. In Tromp-Taylor, fine; immediately resolves to death, same outcome.

Example play: Random play to surround stones. Move exposes empty neighbor, akin to check in chess. Must respond or lose stone. Surrounded on three sides, threat of capture.

Putting stone forces response, otherwise opponent gets two-block. Responding elsewhere leads to capture of three stones.

Sign in to read the full notes

Get access to AI-generated notes, topic timestamps, and more.

Building AlphaGo from scratch – Eric Jang

Building AlphaGo from Scratch – Eric Jang

Introduction to Eric Jang and AlphaGo Project00:00

How Go Works02:04

Sign in to read the full notes