Baseten CEO Tuhin Srivastava on Custom Models, and Building the Inference Cloud
In a Nutshell
Baseten, the AI Inference Cloud, has scaled 30X in a year to over $1B in projected revenue, powering 95% custom models for AI natives like Abridged and Cursor amid severe compute shortages managed via 90 clusters across 18 clouds. Inference dominates as the "last market" due to Jevons paradox—cheaper intelligence drives explosive demand for agentic workflows—while open-source models (including top Chinese ones like DeepSeek at 20% cost) enable companies to own specialized post-trained models using proprietary user signals. Key challenges include capacity crunches requiring multi-year GPU contracts, sticky software layers for 400% NDR, and building an inference-training loop with sandboxes and runtime optimizations for long-horizon agents.
These notes were generated by AI and may contain inaccuracies.
Elad and host interview Tuhin Srivastava, founder and CEO of Baseten, the AI Inference Cloud. Topics include capacity constraints for AI compute, why inference is the last market, changing workloads, open-source and multi-chip future, and 30X scale in a year.
Baseten grown 30X over the last year, expecting more than a billion dollars in revenue this year. Over last 24 months, realization that AI can be put everywhere. Options from closed-source to open-source models. Open-source models crossed chasm in baseline capability. RL techniques and post-training for specialized models mainstream with examples. Customers realizing they can own their inference.
Long tail of models coming through, customers in-housing intelligence. Application layer growing massively; Baseten indexes on that demand.
Existential question: does independent application layer exist versus labs? Application layer exists because value to companies is user signal only they can gather, encoded in workflows rather than models.
Example: Abridged, ambient scribe used by physicians in almost all US hospitals. Deep integration into clinician workflows and EMR. Frontier model companies lack access to that user signal. Companies with signal can post-train models on reward signal, build long-horizon agentic models.
Another example: support tasks at companies like Baseten involve 1-20 actions per ticket, allowing specialized models.
Sign in to read the full notes
Get access to AI-generated notes, topic timestamps, and more.