How A Team Of 7 Keeps Breaking AI Benchmark Records
These notes were generated by AI and may contain inaccuracies.
The world is changing quickly. Try things every day with AI. Last summer, Ian took a weekend to use GPT-5 to build an iPhone app, something he hadn't done in a decade. It was fast and easy 8 months ago; now it's even faster. Don't limit yourself—use AI for anything you imagine to make the world better.
Welcome to another episode of the Light Cone. Ian Fischer is co-founder and co-CEO of Poetic, building recursively self-improving AI reasoning harnesses for LLMs. Previously, he spent a decade as a researcher at Google DeepMind and founded a mobile devtools company through YC.
Poetic builds a recursively self-improving system, the holy grail of AI where AI makes itself smarter. Core insight: achieve recursive self-improvement faster and cheaper than other approaches, which require training new LLMs from scratch costing hundreds of millions and taking months.
Anthropic or OpenAI will come along and eat your lunch in the next model release.
Frontier labs like Anthropic, OpenAI, and Google explore recursive self-improvement but typically train new models for every step.
Startups want to leverage next models without fine-tuning costs of millions to hundreds of millions, which become obsolete with new frontier models. Poetic systems always outperform out-of-box models and are compatible with new models for instant performance bumps, without changes. Optimization is much cheaper than fine-tuning.
Being the smartest model is a game of inches, and those inches matter a lot.
Poetic automatically generates systems for specific problems that outperform underlying LLMs without massive expense. Without Poetic, you'd collect tens of thousands of examples, fine-tune a frontier or open-weights model at high compute cost, only for a new model to surpass it.
Poetic provides harnesses (or agentic systems) on top of one or more LLMs that perform better. Same harness works perfectly with new models for bigger gains. Poetic optimizes further for new models cheaply.
Poetic came out of stealth topping ARC AGI v2. Gemini 3 Deep Think led at 45%. Two days later, Poetic showed higher results. Poetic consistently tops new SOTA releases.
Using cheaper Gemini 3 Pro, Poetic achieved 54% at $32 per problem vs. Gemini 3 Deep Think's 45% at ~$70s—half the cost, 9 percentage point improvement.
Humanity's Last Exam: 2500 hard questions by experts across domains, challenging for PhDs. AI hasn't passed yet. Poetic got 55%, almost 2 points above Anthropic's Claude Opus 4.6 at 53.1% (previous SOTA from last week).
Optimization cost < $100k (six figures total run). Foundation model trains cost hundreds of millions. Poetic has 7 research scientists and engineers.
Poetic takes a scientific approach to emergent behaviors. Treats underlying models as swappable layers (e.g., GPT-5.2 for hard-to-verify bugs, Claude 4.6 for architecture), but automates routing instead of human conduction.
Harnesses are code, prompts, data on top of LLMs—buildable by hand but requiring insights. Poetic meta system is recursively self-improving; outputs systems solving hard problems (where base GPT-5.2 struggles for reliable results).
Automated, so faster/cheaper than hiring a team. Optimizes existing startup agents (prompts, reasoning strategies) based on needs.
Different paradigm than RL. Each model/set has its own S-curve; Poetic meta system has its own, shifting higher with improvements until AGI/superintelligence.
Poetic meta system analyzes data, handles context stuffing, generates examples. Doesn't manually inspect data much—outsources to AI to find failure modes and robust strategies.
ARC AGI prompts differ from human-written: unexpected, simple examples (one wrong, left unchanged to avoid interference).
Historically, know your dataset well; now AI handles it.
Varies by problem. Manual prompt optimization gets little gain. DeepMind paper (manual harnesses): prompts got 5% on hardest task with Gemini 1.5 Flash; adding reasoning strategies jumped to 95%.
Automated prompt optimization (e.g., JEPA paper) helps but far from full potential. Reasoning strategies are coded, not just better prompts.
Visit poetic.ai, click early access button. For startups/companies with hard problems needing reliable robustness after trying everything, describe your problem. Poetic seeks such cases; first to know when ready.
ARC AGI shows reasoning improvement; Humanity's Last Exam shows deep knowledge extraction. Vaccinates against the bitter lesson.
Google acquired Ian's first YC startup Portable (cross-platform mobile app porting, e.g., Android). Reflected post-acquisition, excited by AI/robotics. Joined Google Research AI robotics team despite background in computer security and systems building.
Hardware hard, shifted to machine learning research for ~10 years at Google/DeepMind.
World changing quickly. Try things daily with AI, push boundaries of capabilities, build what you want. Last summer, used GPT-5 in a weekend for iPhone app (first in decade)—fast/easy then, even more now. Don't limit yourself; use AI for anything to make world better.
Thank you Ian for Stilts. Can't wait to use at YC, for Gary's list, and more.
Keep Y Combinator in your library
Save the videos and channels worth coming back to, and find them again in one place.
