Beyond Bigger Models: Recursion As The Next Scaling Law In AI
In a Nutshell
Hierarchical Reasoning Models (HRM) and Tiny Recursive Models (TRM) enable small models (7-28M parameters) to outperform massive LLMs like o3 on ARC Prize (up to 87% on ARC-1) by using recursion for deep reasoning at inference time, bypassing transformer layer limits and backprop-through-time issues via truncated BPTT (t=1), fixed-point iteration, and EM-style latent state updates. Unlike one-shot LLMs or hacks like Chain-of-Thought/tool use (bounded by human data), these RNN-inspired models discover efficient algorithms from scratch on incompressible tasks like Sudoku/mazes, compressing reasoning into hidden states with outer refinement loops. Recursion scales compute without parameter bloat, suggesting hybrids of tiny recursive heads atop giant LLMs as the next AI frontier.
These notes were generated by AI and may contain inaccuracies.
Welcome back to another episode of Decoded. Today with YC visiting partner Francois Shaard discussing recursion to improve model's reasoning performance at inference time rather than making models bigger. Two papers from 2025: Hierarchical Reasoning Models (HRM) and Tiny Recursive Models (TRM).
Francois, thanks for joining. Tell us about these two models and what was interesting.
To set foundation, RNN is a model recursively called on itself. Belief that RNNs required for AGI. Peak RNN use until 2016 with Alex Graves' Neural Turing Machines keynote and adaptive computation time work. This was 10 years ago in era of LSTMs with attention, before attention invented depending on professors.
Limiting step on RNNs: backprop through time. Roll out model, approximate gradient, step back. As model bigger and more steps, accumulation of errors, noisier gradients, stops working. Vanishing or exploding gradients: input with 20 steps multiplies matrices 20 times. For context length million or billion, worse. Retain activations every step: need million copies of brain. Tricks like gradient checkpointing trade memory for wall clock time and compute.
Contrast with LLMs: at training, one-shot feedforward for every input. Transformer block takes all inputs in parallel, not iteratively. No storing tons of activations or giant vanishing gradients.
All happens in one shot with causal mask, lower triangle trick. Forward and backward in one shot, amazing for training wall clock despite high FLOPs and memory. Tradeoff: lose latent reasoning and compression in time direction. No compression in LM: retain entire Shakespeare novel to decode bit. RNNs compress in hidden state.
Sign in to read the full notes
Get access to AI-generated notes, topic timestamps, and more.