Continual Learning: How AI Agents Get Better With Every Use | Arjun Karanam, Trajectory
In a Nutshell
Trajectory's core thesis is that AI agents improve by learning from the interaction data currently being discarded, creating an experience axis orthogonal to intelligence that compounds over time. Their four-pillar approach—traceability of full action trees, specification-based evaluation, reinforcement learning on traces via SDPO, and intelligent harness updates—transforms discarded tokens into training signals that make agents faster, cheaper, and more reliable with continued use. The key practical insight is that feedback must be elicited and categorized: corrective actions like edits and retries provide actionable rewards, while globally relevant patterns train models and user-specific information stays in context.
These notes were generated by AI and may contain inaccuracies.
Arjun Karanam and Ronak are co-founders of Trajectory, a company building a platform for continual learning. Arjun's background includes work with Ronak at One Surf and Train Speed 1, while Michael worked on robotics at DeepMind. The company's mission centers on closing the experience gap in AI systems.
Current AI models excel on the IQ axis, becoming smarter over time, but lack experience. Using the analogy of having Terence Tao in your pocket but him being a poor accountant on day one, the speakers emphasize that experience compounds over time. This creates an orthogonal axis to intelligence that is equally important.
Experience comes from hundreds of millions of tokens generated by agents that are currently discarded. These represent real work being performed with user interactions, yet the data gets thrown away. Trajectory's core thesis is that this interaction data should serve as learning signals, mirroring how humans improve through experience.
Today's agents remain slow, expensive, and error-prone over time, with implementations that don't improve with usage. The vision involves learning agents that become better, faster, and cheaper with continued use, ultimately creating systems that compound in capability over time.
The approach follows four sequential steps:
- Traceability: capturing interactions and experience currently being discarded
Sign in to read the full notes
Get access to AI-generated notes, topic timestamps, and more.