5 Papers That Show Where AI Research Is Heading Right Now
In a Nutshell
The video argues that AI progress hinges on maximizing intelligence per sample and per watt, rejecting the idea that scaling test-time compute or self-improvement on human data alone will reach the full solution space. Key evidence comes from protein language models that follow clean neural scaling laws once data volume reaches billions of sequences, enabling near-AlphaFold performance without handcrafted MSAs, plus mechanistic interpretability showing the models spontaneously learn real biological features. Parallel advances in self-play RL, streaming voice RAG, Lean-based formal verification, and RTS-style agent orchestration show the same pattern: general scaling beats domain-specific engineering when data, search, and verification can be made cheap and automatic.
These notes were generated by AI and may contain inaccuracies.
Thank you guys so much for coming. This one will have a much more applied bent based on the feedback. We have a bunch of really cool people that will be introduced. The session covers AI for biology by Yas Beg. Luke out of Tatsu's lab will talk about self-play, Alpha Zero style self-play for LLMs. Arnob will present on stream rag, a super different application thinking about real-time voice agents. Robert George is working on lean for science. Luke Worthwine is the AI token maxer.
A call for presentations is made to inspire interest and encourage others to present. Memory has been the hot topic for at least the last year and a half. There have been many papers from mem zero to recursive language models, cartridges out of the lab, hnet, and dynamic chunking stuff. There are many different ideas in this area.
A Nome Brown podcast launched a couple weeks ago where the view is that the human-generated subspace H is still viable—if we train on that we can test-time compute our way out of it and recursively self-improve all the way to F minus H. There is a struggle with this view and it does not seem probable that we will sample all of that. It is not that it will not be possible, but it is just not probable that we will sample all of it.
The left side is AlphaGo, the right side is AlphaZero. AlphaZero, unbiased by humans meandering, is the way to get to much more intelligent systems, maybe even AGI. If the full solution space F is F, training on known human solutions will limit you to some typical set H despite any feasible amount of test-time compute or recursive self-improvement. You won't feasibly sample F minus H, especially all of it. If there is infinite recursive self-improvement and infinite test compute, maybe, but we do not have infinite.
Life's a POMDP and this is a finite horizon MDP. Intelligence per sample is discussed.
The two major problems left are intelligence per sample and intelligence per watt. Intelligence per sample is thought about as getting one new sample and doing continuous learning. The goal is to maximize performance conditioned upon that N. Most people's answer in practice is ICL. As the number of samples in ICL increases, it is not monotonically improving in performance. It starts to bob and weave. It gets worse, gets better sometimes, and then hits a cliff at the context length that the model was trained on and literally just stops. It clearly cannot go on forever and does not monotonically improve.
Experiments were done with LoRA. LoRA at lower ranks for lower amounts of sample size actually does impressively well. Then it has a kind of arc. They both peter out pretty quickly as the number of samples increases all the way until doing SFT group O all the way at the end. ICL is the optimal thing to do in the beginning and then training the whole thing. If one new sample is received, the desire is to retrain with LoRA at some rank on N plus one samples just to get that little bump in performance. There is a different optimal thing to do all along the way as streaming occurs and more samples are received. That is not how we currently operate.
Humans are monotonically improving. The more chess games Magnus Carlsen plays, he just keeps getting better. The 10,000 hour rule applies—we just keep getting better. It is the same algo. There must exist some learning procedure that has a much higher intelligence per sample.
Intelligence per watt comes out of the lab with Ivonica and John. Smaller models are sometimes actually better from an intelligence per watt perspective. Alternatives to backprop are of interest. There is very little evidence that the brain is taking the transpose of the weight matrix and there must exist some other learning procedure. SPSA is of high interest. Recommendations for alternatives are welcome.
Novel breakthroughs are of interest. Yaso is one of the favorite AI researchers, mostly focused on bio, and always sends biopapers. It is super interesting whether it is about how birds navigate the world via iron in their liver, crazy robotics, speech, or other things. Unhinge founder hacks are very interesting as well.
Call for ideas on ways to make the club better. Better ways to meet if it is a lightning round or not. Some people talked about AI benchmarks that could actually be launched together. Club challenges to challenge each other. Any open source ideas to hack on together for this club would be interesting.
Labmates for about two years. France brings very creative and out-of-distribution ideas to the group. The talk is on bioAI things, presenting on a paper that came out just last week from Biohub folks in California, not too far in the Bay Area. They just moved to the city. A second year PhD student co-advised by Steve Quake over at Stanford. Steve has done a lot of work in bioengineering and was director of the biohub where this work came out until recently.
Most of the audience is probably more AI/ML types. The talk covers how ideas from language modeling and AI have recently been translating into biology with a focus on this recent paper. The paper does an excellent job of interrogating how scale, which has been the fundamental primitive in terms of assumptions about how to make things better, has been playing out for biological problems, particularly protein biology. There will not be much bio in the talk, focusing more on the ML.
The talk is titled "The Bitter Lesson Comes from Biology." The paper title is below that. A quick refresher on Richard Sutton's famous article: across the past 70 years of AI, methods that win are methods that are general that exploit fundamentals of scaling compute and data as opposed to methods that hand-engineer human domain knowledge. Sutton cites work in AlphaGo and AlphaGo Zero that inordinately scaled compute. For a long time they were far worse than expert systems until they eventually overtook and then exponentially improved past them. Knowledge systems win at first but then eventually these big large dumber models will win in the long run.
This is also true for a lot of biological AI problems. The bet behind this paper is that the same pattern basically also holds in protein biology. The question is to what extent we can study this. Scaling law analysis comes from the famous neural scaling laws paper and translates them for problems we care about for designing a drug or understanding how a cell works. On the left is something trusted like a language model with nice smooth log-linear scaling laws that predictively fall as a function of compute and data. The question on the right is whether this curve will exist for bio, specifically proteins. Does our LLM recipe transfer or does biology, being out of distributional domain relative to language, break it.
Three vignettes from this paper interrogate to what extent this is true.
This is all the biology needed. In the body there are broadly three major classes of macromolecules: lipids, carbohydrates, and proteins. A protein is just a string of amino acids, a special type of biomolecule. There are 20 varieties of amino acids. If put together into a sequence, there can be a virtually infinite number of possible molecules that then fold. Every single protein is a 20-letter alphabet. That string specifically determines a unique 3D shape. By virtue of the shape of that protein, what job it does in the cell—presents, catalyzes a reaction, keeps pathogens out, etc.
The work done trains ESMC, the third or fourth major iteration in a long series of models from the evolutionary scale group, originally at Meta, then their own company, now Biohub. The cell is very similar to language. We take hundreds of millions of years of evolved sequences found across biology both in humans but also across bacteria and in the environment and train a big masked language model on them. Nowadays we mostly train NTP models, but the pitch is that if a protein is represented as strings of 20 tokens, hide a few, and train a really big BERT-style transformer to predict masked positions as a function of the other ones nearby. The crucial part is that nothing is told about the protein beyond just the sequence. All access is just this string and it is asked to learn things about the grammar of that protein as a function of which amino acids tend to co-occur.
There is an old saying in natural language processing that you will know a word by the company it keeps. Here the idea is that you will know a protein by the amino acids it keeps. The bet is that if this is done at scale just on the simple sequence task, we will eventually get all these other properties of proteins like structure that we do care about.
There has been a lot of prior work on this largely from evolutionary scale but a few other groups working on this. This table will be a map for the rest of the talk. Every row is a concept already known from natural language and then analogized onto the protein context. Tokens become amino acids. The internet becomes all evolution sequence databases, all the proteins we can actually go out and measure. Masked token prediction stays as masked token prediction. Emerging capabilities—we talk about language models having emerging structure and function within basically understanding of a protein. There is also really fun stuff at the bottom. Recently there have been a lot of advancements in interpretability toolkits from the MechInterp folks, things like sparse autoencoders, some of the earliest work trying to interrogate using the toolkit the language modeling community has built to understand language models now in a protein language model setting.
The right-hand column will be filled in with evidence and three questions: do these models learn with scale, can they basically substitute for a lot of these handbuilt features, does the bitter lesson hold, and what do these representations actually encode interpretably.
Do scaling laws even hold in the protein context in the way we see them in the language context. What we measure when talking about emergent properties—how do we actually study the model to see if it is learning anything. We need a proxy for does the model understand protein structure. The one the authors use is that they look at the internal or take the model representations during training and use this to predict long-distance protein contacts. Proteins have a one-dimensional sequence but they fold into complex three-dimensional shapes. If the model has understood something complex about the protein structure, it should be able to predict contacts that occur over long distances. Nearby contacts are rather kind of obvious and this is a really challenging object to get just de novo purely from sequences alone. They call this P at L, long contact precision at some given length. It is a clean unsupervised readout of structural knowledge inbuilt in the model learned during this language modeling objective.
On the right the performance is plotted against training compute for this new model family called the ESM Cambrian at 300 million, 600 million, 6 billion parameter scales. There was a fit line which is basically this predicted compute optimality curve estimated just from low-end training runs. They find it actually extrapolates very cleanly to real model training runs. The answer is yes, these models do scale. There is a nice log-linear curve. If more and more compute is invested, training more and more protein data with larger and larger models, the same exact broad qualitative shape as the LM scaling setting appears and it transfers cleanly. Without any kind of predisposed part of the model taught to look at protein structure, even without getting any protein structures, it does a good job of picking these out just from sequence co-occurrence patterns.
Previous models, the prior generation ESM2 models shown in purple, had actually not shown the same behavior. They hit a wall where they kept adding more parameters and got diminishing returns and there was flattening out of the scaling curve. This ESMC or ESM Cambrian model, the green line, keeps climbing with no plateau. The fix was not really a clever inductive bias in the architecture. There is a lot of excellent engineering work in this paper, but really it was just data scaling. There were about 50 million training samples in the original ESM2 paper and here they pushed that to 2.8 billion by pulling largely in metagenomic data. Protein sequences found from sequencing DNA out in dirt and oceans and human guts from organisms that nobody has really ever cultured or elucidated. The conclusion is that more data ends up being really important and keeps justifying the cost for increasing compute. It is the protein version of the LM data wall conversation. Evolution has been generating this training data for four billion years and not humans in the past 30 or so. Compared to tokens in natural language, we have only sampled less than 1% of all known protein sequence diversity and that is only currently at this moment in time, let alone all of the sequence diversity evolution has sampled since the beginning of life on Earth.
Question two is the most bitter lesson part. They really try to evaluate to what extent their model trained purely on masked language modeling objectives can compete against a structure-based model with hand-tuned inductive components. AlphaFold won the Nobel Prize a few years ago, a landmark moment in bio showing computational methods have a lot of value. AlphaFold is brilliant but its power comes from building hand-built inputs, a manual feature curation called a multiple sequence alignment or MSA. To fold a protein it goes and finds hundreds of evolutionary cousins of that protein and stacks them up. These patterns of covariation across a family encode the information needed to get structure. This is a beautiful domain engineering application and the sort of really good human crafting objective bias that the bitter lesson claims should eventually lose. Think like HOG features in CV and compared to what we used to do before. This is actually far more bitter lesson than building a whole physics simulator for a protein, but it is also really slow. We need to build these huge databases. The sequence alignment takes time and it is absent precisely where you often want it, for instance the antibody design task.
ESM just throws this away. It just takes input sequence instead of an alignment and feeds the model's representations as the input to their structure predictor. These are just per-residue embeddings. Take the input sequence, get a set of per amino acid numerical representation, and train the specialized module to do predicting the large protein structure. This folding network is a projection into 3D coordinate space. Same target, same output, no hand-built features. The question becomes can this general model representation match the specialist model in getting that MSA value.
One interesting architectural note: in their projection networks for the part that converts representation to structure, there is actually one really interesting feature that builds off work from lab alum Dan Fu. They have a looped model. There is a lot of excitement about these recently for parameter sharing. They are cool algorithmically for a number of reasons. This gives a lever by which they can scale inference-time compute. There is a model that predicts structures and a procedure by which representations can be fed through a series of layers and refine their structure predictions without necessarily retraining or any kind of fine tuning. This is test-time compute access, something like diffusion steps or test-time sampling.
The headline figure shows that the technique works really well. DOCQ pass rate is a metric for how good the structure prediction was. It is a measure of the fraction of test cases where the predicted shape of two proteins that stick close together is close enough to be really useful in realistic settings. There are two groups in each panel: one is for single sequence with no MSA and the other is a single sequence plus an optional MSA that can also be fed to the model or is required for the competitor model.
For general protein-protein complexes, ESM Fold 2, the new projection model from a single sequence with no MSA, lands within about three points of AlphaFold 3 which does take these handcrafted features. Near par is achieved without the crutch. For antibody applications on the right, single sequence ESM Fold 2 does actually beat AlphaFold 3, 50 versus 47 on this really specific design task that people really care about. Biologically this makes a lot of sense. Compared to other classes of proteins, the amount of sequence variation sampled in the space of all known antibodies relative to structure is considerably smaller considering their enormous diversity.
The headline is not that MSAs are dead yet. Hand-built features only help where it is abundant and where drug designers really need it, it often does go away. This general method still saves a lot from pre-training across all known evolutionary contacts. We are not quite there yet. A second point: give it MSA. We can also scale the amount of test-time compute, how many loops we run in this recursive model in order to improve performance. We do see returns on this, meaning that the bitter lesson at least at inference time also seems to broadly hold. It is also just much faster. MSA construction takes a lot of classical computational biology time. If throughput or latency is the concern, with this single representation quicker results can be obtained. The wall clock times are well within what would be considered pretty good to start off with.
The last bit is a really interesting analysis of mechanistic interpretability—what these models are actually learning and can we find features that are interpretable as humans in the same way that language modeling folks in the MechInterp community have found in language models.
Researchers applied sparse coding analysis tools from Anthropic to examine activations in protein models. The analysis found that models trained purely with fill-in-the-blank pre-training decompose their latent space into clean features corresponding to real biological concepts. These concepts were annotated by language model agents and organized in a hierarchy: individual amino acids at the bottom, followed by structural motifs, whole protein domains, functional sites, and whole protein roles. None of this organization was supervised—the model learned to structure its latent space purely through masked language modeling.
One example feature activation corresponds to the nucleophilic elbow, a well-known protein motif used in enzyme catalysis. This motif has evolved multiple times in unrelated proteins. The model identified this motif across four structurally diverse proteins with both evolutionary distance and different surrounding protein contexts, demonstrating it learned to recognize the motif rather than memorizing broad sequence similarity.
Analysis of the model's full activation space revealed structures corresponding to known aspects of biology. The organization scales to all of life. Researchers built a protein atlas by folding and analyzing up to seven billion proteins, creating the largest atlas in alpha protein structure databases—larger than AlphaFold's database. They predicted on the order of a billion proteins and laid them out in SAPE space representation, producing a protein space family map. Clear families are visible, including CRISPR-Cas9 enzymes. The map functions as a Google Maps equivalent for proteins, revealing evolutionary and functional relationships discovered entirely de novo.
Data scaling continues to improve performance. While some analysis still requires handcrafted features and is not fully competitive yet, models learn enormous amounts of biology from relatively simple pre-training objectives and large data volumes, which can be reverse-interrogated afterward. Biology is recommended as a strong area for ML work because models remain young and data is increasing exponentially each year with an accelerating rate of increase, meaning the field is not data-limited.
Models were also applied to inverse design, developing potential protein drugs validated in wet lab settings. These proteins were designed using the model primarily in sequence space, with one structure head at the end. An example is a PD-L1 binder, a medication representing a major success in immunotherapy that has helped cancer patients in previously untreatable ways. Developing medications targeting this protein was historically challenging, and reducing future drug development costs would have enormous human impact.
Current training stacks for large language models consist of pre-training on web text followed by post-training. A significant portion of compute is now spent on large-scale, long-running reinforcement learning during post-training. This RL involves collecting numerous tasks (coding, mathematics, software interaction), having the agent take actions, receiving rewards, and updating the model by upweighting good rollouts and downweighting bad rollouts. RL post-training compute is now approaching or surpassing pre-training compute.
Increasing the number of RL tasks and compute during post-training produces better downstream performance. A plot from the Composer 2 technical report shows that when models see each task only once, increasing training steps (each introducing compute and a new RL task) produces reliable improvement on both an evaluation set and a downstream coding benchmark.
The challenge is that RL tasks must currently be collected by hand, and the x-axis on the relevant plot uses log scale. Additionally, the goal is for models to eventually surpass any problems humans can provide them.
Self-play addresses automatic generation of new RL tasks. In traditional RL, a predefined task and environment are used for training. In self-play, the model both generates RL tasks and attempts to solve them, with training improving performance on both generation and solving.
Symmetric self-play (as in AlphaGo/AlphaZero) involves the model taking on the same role twice—for example, an older version of the agent serves as opponent in Go. Asymmetric self-play uses a conjecturer model to generate entire RL tasks for a solver model to operate in. The conjecturer might create a coding problem with unit tests, after which the solver performs rollouts, receives reward, and trains on that data.
In principle, self-play removes bounds on learning. Training on human demonstrations limits the model to demonstration quality. Standard RL has two failure modes: acing all environments or receiving no reward in them. Self-play generates new learning signals continuously through new tasks. Two-player games like Go demonstrated this, with models improving beyond human performance indefinitely.
In practice, running self-play for language models causes the model to plateau and stop improving, matching the plateau seen in standard RL.
The baseline algorithm samples synthetic tasks from the conjecturer, has the solver attempt them, verifies correctness via reward signal (such as unit tests written by the conjecturer), updates the solver on correct rollouts, and updates the conjecturer using a reward of zero if the solver could not solve the problem and one minus the solver success rate otherwise. This reward incentivizes the conjecturer to produce problems at the frontier of the solver's capability.
An experiment used 3,000 formal math problems written in Lean. A strong RL baseline plateaued at approximately 60% of problems solved. Standard self-play generated increasing numbers of synthetic tasks at the solver's capability frontier but produced no improvement over the RL baseline.
The conjecturer learned to produce extremely complicated, overly complex, and inelegant problem statements in Lean. The reward for producing tricky problems was satisfied by creating messy, artificially complex problems equivalent to three-page high school calculus problems that cause solvers to make minor errors—useless for learning transferable mathematical skills.
The SGS algorithm addresses the problem with two components. For each unsolved target problem from the 3,000-problem set, the conjecturer generates a related synthetic problem, grounding the synthetic data distribution in known good problems. A third model role called the guide judges whether the synthetic problem is actually related to the target and is not overly complex. This guide score is multiplied with the solver-rate reward when updating the conjecturer.
Using a 7 billion parameter model, SGS outperformed both the RL baseline and standard self-play. With eight times the compute spent on self-play, the 7B model reached the pass rate of a 670B parameter model. However, the method did not reach 100% problem coverage, leaving substantial unsolved problems for future work.
Voice AI applications require conversational, low-latency interactions. Standard RAG introduces unacceptable latency for natural conversation. The approach is to begin analyzing spoken words and run RAG while the question is still being spoken, rather than waiting for the question to complete.
Fixed interval streaming RAG divides audio into blocks and runs RAG after each block arrives. The key challenge is determining which intermediate block is sufficiently complete to warrant running the full RAG pipeline. One method compares top documents retrieved from partial queries against those from the complete query. If the top documents match, the intermediate query is considered sufficient and the full RAG pipeline proceeds on it. Another approach involves fine-tuning a model to decide when to trigger RAG.
An alternative to running RAG on every single chunk involves fine-tuning a model to trigger only when a chunk contains critical new information. The model evaluates whether the partial query warrants generating a new query or if existing queries from previous chunks suffice for answering the question. This approach avoids computationally wasteful calls on every chunk.
The paper describes a post-training pipeline where pseudo queries are generated from partial spoken questions using an LLM, followed by RAG execution. Retrieved documents are analyzed to determine whether the partial query introduces something new or whether sufficient material already exists.
Alternative Assessment Methods
Decision-making can be based on retrieval quality of the partial question. Alternative approaches include examining the semantics of the question so far to determine whether the partial question alone provides enough information to answer without running the full RAG pipeline. Research is needed on determining when a portion of a spoken question is sufficient to proceed with retrieval rather than waiting until completion.
Results from the Paper
The paper, approximately one year old, evaluated smaller open source models on RAG benchmarks converted to audio. Latency decreased by 0.5 seconds on synthetic datasets and by nearly 1.5 seconds on human spoken datasets. Accuracy remained comparable between RAG after the final query and streaming RAG approaches.
The key takeaway is that solving small problems in this domain can lead to substantial production gains.
Robert George, a third-year PhD student at Caltech working on AI for math and science, presented on Lean and the emerging era of verified intelligence.
Recent Breakthroughs in AI for Mathematics
Multiple breakthroughs occurred within the preceding couple of weeks. Two years prior, the IMO remained an open challenge. DeepMind achieved a gold medal at the 2024 IMO. A prominent problem list for solving previously unsolved IMO problems continues to grow with new models from OpenAI, DeepMind, and others. OpenAI recently claimed to solve an 80-year-old IMO problem. DeepMind released work solving problems across multiple fields, incorporating formal verification in the loop.
Informal vs. Formal Mathematics
Informal mathematics, as encountered in high school calculus and undergraduate courses, is flexible. Professors sometimes use proof by QED or proof by intimidation, with many steps left unwritten. Formal mathematics requires full explicitness.
History and Capabilities of Lean
Formal mathematics has been pursued for hundreds of years. Lean has emerged as a well-designed language that has gained significant adoption. Lean offers two core advantages: proofs are easily verified as correct, and the system cannot be fooled; it is scalable despite existing challenges.
Automatic vs. Interactive Theorem Provers
Automatic theorem provers, including SMT solvers from the 1990s through the 2020s, require minimal human effort but have limited expressivity. Interactive theorem provers such as Lean, Coq, and Isabelle employ stricter expressive logic systems based on dependent type theory, requiring greater human effort to write proofs.
The Mathlib library in Lean represents a major formalized mathematics collection with at least one million lines of high-quality mathematics spanning topology, algebraic geometry, and other domains.
Lean as Both Theorem Prover and Programming Language
Lean functions as both a theorem prover and a functional programming language. It supports compile-time checking, meta-programming, macros, custom automation, and even game creation. Lean provides a foreign function interface for external library bindings including CUDA.
Lean Code Structure
In VS Code, an info goal view displays current subgoals during proof development. Each proof line is called a tactic. Proof search operates over tactic space. Two axes exist: foolproof generation and tactic search.
Lean has a steep learning curve but is more approachable than C++ for some learners. Users experience satisfaction from fully proving theorems with no remaining assumptions or handwaving.
Formalization Breakthroughs
The first generative language model for automatic theorem proving was released in 2020 by Ilya and Stan from OpenAI, called GPDF. Progress on miniF2F, an Olympiad-level benchmark, has been exponential, involving open source models and major players across China, the US, and Canada.
At the previous IMO, DeepMind claimed not to have used Lean, though OpenAI solutions employed a DSL of Lean-like constructs. The Chinese system known as STP prover also achieved IMO gold. Axiomatic improver and Harmonic AI recently solved all 12 problems in the Putnam competition and most IMO problems, with solutions formally verified using systems like Aristotle from Harmonic. Fields Medal work from Math Inc and Google DeepMind contributions represent additional notable developments.
Program Verification and AI for Science
Beyond mathematics, program verification addresses the trillion-dollar industry of bugs in code. AI for science involves reproducibility concerns. LLMs can write code, but proving correctness remains challenging. Scale of generated code, presence of bugs, and capturing human intent in verification languages are open considerations.
Program verification involves human specifications that code must satisfy. A proof demonstrates that code meets its specification.
Bridge and Verifiable Coding
The Bridge work uses Lean as a functional programming language to elicit LLMs to prove code correctness. Max Tegmark's group advocates shifting from vibe coding to verifiable coding. CSL, started from Clark Barrett's group at Stanford with contributions from DeepMind researchers, provides a platform for contributing to computer science concepts.
Torch: Neural Networks in Lean
Torch represents the first unified framework for writing neural networks in Lean. It provides a PyTorch-style tensor system compiling to a shared intermediate representation. Properties of specifications can be proven, including verified floating-point arithmetic and neural network verification such as certified robustness.
Flash Attention can be shown equal to standard attention at the specification level. Attention mechanisms are permutation invariant without positional encodings. A GPT-2 style model was trained natively in Torch Lean, with properties proven about the system.
Non-Determinism Formalization
Thinking Machine Lab released work on non-determinism in LLM inference even at temperature zero. The entire system was formalized in Torch Lean down to near-GPU kernel verification at a small CUDA level. The work demonstrated that tiny floating-point arithmetic differences can flip final argmax results in batch processing.
The goal is a future where science and code can be formally verified through accumulated building blocks.
Luke Orthwein, who holds a CS degree from Harvard and previously ran growth at WeChat from 2012 to 2015, presented on maximizing advantages of agentic programming assistance.
Channel AI operates as a consumer entertainment AI business focused on automating software and content development through end-to-end pure AI systems.
Chess vs. Real-Time Strategy Analogy
Traditional programming resembled chess: linear, predictive, focused on thoughtful robust system design. Agentic coding resembles real-time strategy games where multiple elements must be balanced simultaneously—economy, production, unit activity, and engagement. Maximally parallelizing both systems and attention enables corrective feedback as new information emerges.
Linear Work Trees and Parallel Development
Linear work trees combine git work trees, task management software, portable work, and autonomous agents. Multiple repositories run development in parallel, compiling separately without interference.
Orchestrator and Worker Agent Workflow
An orchestrator agent, typically running Claude or Codex 2, initiates work from an idea with minimal keystrokes. Status tracking functions like watching a minimap. Workers receive instructions to proceed as far as possible with low premium on their own time and high premium on human time. Workers push forward even if potentially incorrect, enabling many agents to run in parallel despite token waste.
Workers complete work to PR stage including summaries. Results from completed workers feed back into the system for learning with minimal human input.
Core Practices
All scripts and operations run from cloud instances. Portability enables moving work between machines for overnight execution or additional compute. Dangerously skip permissions mode runs whenever possible; sandboxes are created when needed to avoid regular human feedback loops that slow progress.
Workers do not rigidly adhere to given specifications but learn and adapt, making assumptions that can be corrected later. For front-end development, workers boot local dev servers, run tests, and prepare environments so humans can test by opening browser tabs at the correct port.
Baked-in Knowledge and Documentation
Known agent weaknesses are identified and baked into cloud MD files and broader linked wiki-style knowledge base graphs. Claude systematically underestimates task duration because training data reflects human timelines rather than current system capabilities.
Code serves as an expensive source of truth for context retrieval. Aggressively documenting in structured linked knowledge base files benefits future agents, improves human visibility, and enhances auditability.
Macro by Default, Micro When It Counts
RTS principles apply: winning requires producing units rather than only moving existing ones. Deep dives into critical tickets are balanced with spawning additional parallel work that consumes less cognitive bandwidth. Items can wait days for return with minimal cost.
Macro is necessary while micro is useful, but you can win in RTS games and in programming by doing enough things and quickly identifying problems to solve them. The high visibility setup allows immediate auditing and correction of agent streams without digging through interfaces. Agents make mistakes and go in wrong directions, so catching errors early saves significant time and value. You should be looking around between different agents, monitoring them while running as many as possible in parallel.
Audio cues are valuable for managing many agents across tasks. Sound design from Warcraft and Starcraft provides memorable, attention-catching notifications. Each individual agent session maps to different Warcraft and Starcraft units that are color-coded and themed based on ticket type, playing the actual sound effects from those games. This allows immediate visual identification of which tab needs attention without reading.
Gaming has optimized pneumatic devices and cues, with sound design that catches attention in different ways. Color, icons, and anything that enables quicker reading and processing matter significantly when trying to aggressively complete work. The sky is the limit in how you can implement these attention mechanisms.
An APM tracker monitors productivity using tool calls rather than clicks. The tracker shows actions per minute, per five minutes, per hour, per day, and per seven days. No one is good at RTS games without high APM, though maximizing APM alone does not make you the best player. The mental rubric asks whether you are thinking and typing slowly, and whether you would be the best if this were a competition.
Taking lots of little micro decisions moves toward the right goals. High APM means agents are actively making tool calls rather than sitting idle. Low APM likely indicates you are not being as productive as possible.
Using tokens more effectively means doing many things in parallel, having different agents work simultaneously, and running different tasks with the same agent. Complex tasks usually produce better outcomes when executed in parallel rather than sequentially. Never have Claude tokens sitting unused, as that represents inefficient economy. Use all available tokens every hour.
The presentation was created by going to Claude, pasting the requested topic, and instructing it to look at the existing knowledge base and how things are done. The system generated the PowerPoint based on philosophies embedded in the knowledge base and prior advice. Approximately 15 edits were needed to reach the final version.
The completed presentation was fed back into the knowledge base with instructions to learn everything said, all advice given, and all corrections made. The knowledge base uses linked docs because they are much faster and more diverse for LLMs. Claude and Codex are really good at coming up with features when they have enough knowledge about the business. Building the knowledge base in an automated way is super useful.
Everybody should be full stack all the time. People should come up with their own tickets because if something could be done, it should be done. Be reactive. Even if an agent does something way worse or slower than you would, it is still better to have it done. It is easy to change things when they are screwed up.
Satisficing is doing things enough but not perfect, and this is a really key principle for everything. Mix different ticket sizes at the same time. The team achieved 3.5x output in PRs per engineer per month. Both the use of LLMs and the broad adoption of these methods in the last month contributed to another 60% growth in PRs per engineer per month.
You are not going to get a lot smarter, but you can train yourself on how to act like people who are good at doing these kinds of things, like RTS pro players. The goal is to be optimal in this domain and learn the methods of doing it, just like programming like an RTS pro.
The session concluded with thanks to attendees. A feedback form will be sent out for thoughts and reviews. Calls for presentations and ideas remain open. More paper discussions are scheduled, with the next one in two weeks already fully slated and the first one in July still open for presenters.
Keep Y Combinator in your library
Save the videos and channels worth coming back to, and find them again in one place.
