5 Papers That Show Where AI Research Is Heading Right Now

Name: 5 Papers That Show Where AI Research Is Heading Right Now
Uploaded: 2026-06-12T14:00:20.000Z
Duration: 1 h 16 min 55 s
Channel: Y Combinator

Y CombinatorJune 12, 20261h 16m

Topics49

Introduction and Overview0:08 Call for Presentations0:31 Views on Human-Generated Subspace1:00 AlphaGo vs AlphaZero and the Solution Space1:30 Intelligence Per Sample and Per Watt2:32 Experiments with LoRA and Training Approaches3:01 Human Learning vs Current AI Approaches4:01 Intelligence Per Watt and Learning Procedures4:30 Interests in Novel Breakthroughs and Biology5:01 Club Improvement Ideas5:30 BioAI Talk Introduction6:00 High-Level Pitch6:31 The Bitter Lesson and Biology7:01 Scaling Laws in Protein Biology7:31 Three Vignettes8:30 Biology Background on Proteins9:02 ESMC Model Training10:01 Learning Protein Grammar10:30 Prior Work and Table Overview11:00 Three Questions12:01 Question One: Do Scaling Laws Hold12:01 Performance Against Training Compute13:00 Previous Models and Data Scaling14:01 Question Two: Bitter Lesson Evaluation15:30 ESM Approach Without MSA16:30 Looped Model Architecture17:00 Headline Results18:00 Performance on Protein Complexes and Antibodies18:32 MSA Status and Test-Time Compute19:30 Mechanistic Interpretability Analysis20:30 Protein Model Latent Space Analysis20:35 Protein Atlas Construction22:31 Scaling the Bitter Lesson to Biology23:01 Post-Training Reinforcement Learning for Large Language Models25:31 Self-Play for Language Models28:00 The Promise and Reality of Self-Play30:00 Baseline Self-Play Algorithm and Its Failure Mode32:00 Self-Guided Self-Play (SGS)35:00 SGS Experimental Results36:02 Streaming RAG for Voice AI37:32 Optimizing RAG for Streaming Audio Queries44:03 Lean and the Era of Verified Intelligence47:30 Rethinking Software Engineering with Agentic Systems58:32 Managing Multiple Agents with RTS-Inspired Workflows1:08:40 APM Tracking for Agent Productivity1:12:01 Token Efficiency and Parallel Execution1:13:30 Building and Maintaining Knowledge Bases1:14:02 Team Principles and Output Metrics1:15:00 Closing Remarks1:16:06

In a Nutshell

The video argues that AI progress hinges on maximizing intelligence per sample and per watt, rejecting the idea that scaling test-time compute or self-improvement on human data alone will reach the full solution space. Key evidence comes from protein language models that follow clean neural scaling laws once data volume reaches billions of sequences, enabling near-AlphaFold performance without handcrafted MSAs, plus mechanistic interpretability showing the models spontaneously learn real biological features. Parallel advances in self-play RL, streaming voice RAG, Lean-based formal verification, and RTS-style agent orchestration show the same pattern: general scaling beats domain-specific engineering when data, search, and verification can be made cheap and automatic.

AI-Generated Notes

These notes were generated by AI and may contain inaccuracies.

Thank you guys so much for coming. This one will have a much more applied bent based on the feedback. We have a bunch of really cool people that will be introduced. The session covers AI for biology by Yas Beg. Luke out of Tatsu's lab will talk about self-play, Alpha Zero style self-play for LLMs. Arnob will present on stream rag, a super different application thinking about real-time voice agents. Robert George is working on lean for science. Luke Worthwine is the AI token maxer.

A call for presentations is made to inspire interest and encourage others to present. Memory has been the hot topic for at least the last year and a half. There have been many papers from mem zero to recursive language models, cartridges out of the lab, hnet, and dynamic chunking stuff. There are many different ideas in this area.

A Nome Brown podcast launched a couple weeks ago where the view is that the human-generated subspace H is still viable—if we train on that we can test-time compute our way out of it and recursively self-improve all the way to F minus H. There is a struggle with this view and it does not seem probable that we will sample all of that. It is not that it will not be possible, but it is just not probable that we will sample all of it.

The left side is AlphaGo, the right side is AlphaZero. AlphaZero, unbiased by humans meandering, is the way to get to much more intelligent systems, maybe even AGI. If the full solution space F is F, training on known human solutions will limit you to some typical set H despite any feasible amount of test-time compute or recursive self-improvement. You won't feasibly sample F minus H, especially all of it. If there is infinite recursive self-improvement and infinite test compute, maybe, but we do not have infinite.

Life's a POMDP and this is a finite horizon MDP. Intelligence per sample is discussed.

The two major problems left are intelligence per sample and intelligence per watt. Intelligence per sample is thought about as getting one new sample and doing continuous learning. The goal is to maximize performance conditioned upon that N. Most people's answer in practice is ICL. As the number of samples in ICL increases, it is not monotonically improving in performance. It starts to bob and weave. It gets worse, gets better sometimes, and then hits a cliff at the context length that the model was trained on and literally just stops. It clearly cannot go on forever and does not monotonically improve.

Experiments were done with LoRA. LoRA at lower ranks for lower amounts of sample size actually does impressively well. Then it has a kind of arc. They both peter out pretty quickly as the number of samples increases all the way until doing SFT group O all the way at the end. ICL is the optimal thing to do in the beginning and then training the whole thing. If one new sample is received, the desire is to retrain with LoRA at some rank on N plus one samples just to get that little bump in performance. There is a different optimal thing to do all along the way as streaming occurs and more samples are received. That is not how we currently operate.

Sign in to read the full notes

Get access to AI-generated notes, topic timestamps, and more.

5 Papers That Show Where AI Research Is Heading Right Now

Introduction and Overview00:08

Call for Presentations00:31

Views on Human-Generated Subspace01:00

AlphaGo vs AlphaZero and the Solution Space01:30

Sign in to read the full notes