Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google & Amazon
These notes were generated by AI and may contain inaccuracies.
Building AI products is very different from building non-AI products. Most people ignore the non-determinism: you don't know how the user might behave with your product and you also don't know how the LLM might respond to that. The second difference is the agency control trade-off. Every time you hand over decision-making capabilities to agentic systems, you're relinquishing some amount of control. This significantly changes the way you should build product.
Recommend building step by step. Starting small forces you to think about the problem you're going to solve. In all these AI advancements, one easy slippery slope is to keep thinking about complexities of the solution and forget the problem that you're trying to solve. It's not about being the first company to have an agent among your competitors. It's about having built the right flywheels in place so that you can improve over time.
Leaders have to get back to being hands-on. A CEO of Rackspace had a daily block from 4 to 6 a.m. for catching up with AI. You must be comfortable with the fact that your intuitions might not be right and you probably are the dumbest person in the room and you want to learn from everyone.
Persistence is extremely valuable. Successful companies building in any new area are going through the pain of learning this, implementing this, and understanding what works and what doesn't work. Pain is the new mode.
Guests: Aishwarya Raani and Kiti Bottom. Kiti works on codecs at OpenAI and has spent the last decade building AI and ML infrastructure at Google and Kumo. Ash was an early AI researcher at Alexa and Microsoft and has published over 35 research papers. Together, they've led and supported over 50 AI product deployments across companies like Amazon, Databricks, OpenAI, Google, and both startups and large enterprises.
They teach the number one rated AI course on Maven, where they teach product leaders all of the key lessons they've learned about building successful AI products. Goal of this episode is to save you and your team a lot of pain and suffering and wasted time trying to build your AI product.
Annual subscriber of Lenny's newsletter gets a year free of products including Lovable, Replit, Bold, Gamma, Notion, Linear, Devon, Posthog, Superhuman, Decrypt, Whisper Flow, Perplexity, Warp, Granola, Magic Pattern, Descript, Chapter AI, Mobius, Stripe Atlas. Head to lennyspodcast.com/productpass.
Merge: Single API for B2B SaaS companies to embed and ship 220+ customer-facing integrations in weeks, not quarters. Like Plaid for B2B SaaS. Companies like Miracle AI, Ramp use it for accounting, HR, ticketing, CRM, file storage. Now supports secure deployment of connectors to AI agents. merge.dev/lenny for $50 Amazon gift card.
Strella: AI customer research platform. Uses AI to run and analyze in-depth interviews automatically. AI moderator asks real follow-up questions, surfaces patterns across hundreds of conversations in hours, not weeks. Used by Amazon, Duolingo for Figma prototype testing, concept validation, customer journey research. strel.io/lenny.
2025 is significantly different than 2024. Skepticism has reduced. Last year, many leaders thought this was another crypto wave. Use cases were Snapchat on your data, calling themselves AI products. This year, companies are rethinking user experiences and workflows, deconstructing and reconstructing processes to build successful AI products.
Execution is still all over the place. This is a three-year-old field with no playbooks or textbooks. AI lifecycle pre-deployment and post-deployment is very different from traditional software lifecycle. Old contracts and handoffs between PMs, engineers, data folks are broken. Need tighter collaboration, owning the same feedback loop, sitting in the same room, looking at agent traces together to decide product behavior.
Tons of similarities between building AI and software systems, but fundamental changes. First: non-determinism. Working with non-deterministic API vs traditional software.
In traditional software like Booking.com, well-mapped decision engine: intention (booking in San Francisco for two nights) converts to action via buttons, forms. In AI, that layer replaced by fluid natural language interface. Users can communicate intentions in tons of ways. Don't know how user will behave (input side). Output: non-deterministic probabilistic LLM API, sensitive to prompt phrasings, black boxes. Don't know output surface.
Working with input, output, and process you don't understand well. Anticipate behavior and build for it. With agentic systems, even harder.
Second: agency control trade-off. Obsessed with autonomous agents, but handing over decision-making relinquishes control. Ensure agent can earn your trust, is reliable enough to make decisions. More agency (ability to make decisions) means losing control. Ensure AI has earned that ability or built trust over time.
Traditional software: predictable experience. AI: can't predict exact same thing every time.
Significantly changes the way you should be building product.
Distinction must exist in your mind when starting to build. Like training to hike Half Dome: don't start at the end, train in minor parts, slowly improve. Don't start with agents with all tools and context on day one. Start deliberately in places with minimal impact and more human control to grip current capabilities, then lean into more agency and less control.
Good: don't feel overwhelmed by fancy AI agents. Everyone starts minimalistic and evolves. Bad: as trying to build one-click agents, don't get overwhelmed; slowly graduate.
Example: customer support agents. OpenAI faced spike in support volume launching products like Images, GPT-4o, GPT-5. Not about dumping help center articles into AI agent.
V1: Suggest to human support agents what AI thinks is right. Get feedback loop: good/bad suggestions, identify drawbacks, blind spots, fix.
V2: Show answer directly to customer (human still reviews?).
V3: Add complexity like issue refunds, raise feature requests with engineering.
Start with all on day one: hard to control complexity. Build step by step, increasing autonomy.
Higher level: behavior calibration. Impossible to predict system behavior upfront. Don't ruin customer experience: keep human control, constrain autonomy. No single right way.
Example: Insurance pre-authorization. Clinicians spend time on blood tests, MRIs. Low-hanging: approve MRIs/blood tests autonomously once patient info known. High-risk like invasive surgery: keep human in loop. Log what humans do to build flywheel for improvement. Not ruining UX, not eroding trust, continuously improve.
Other examples:
- Coding assistant:
- V1: Suggest inline completion, boilerplate snippets.
- V2: Generate larger blocks like tests, refactors for human review.
- V3: Apply changes, open PRs autonomously.
- Marketing assistant:
- V1: Draft emails, social copy.
- V2: Build multi-step campaign, run it.
- V3: Launch, A/B test, auto-optimize across channels.
Problem first: Starting small forces thinking about the problem, break into levels of autonomy. Easy slippery slope in AI advancements: focus on solution complexities, forget problem.
Benefits: Avoid agent doing too much, messing up database, sending unexpected emails.
UC Berkeley paper (Matei Zaharia, Stroker, Databricks folks): 75% of enterprises' biggest problem was reliability, why not comfortable deploying customer-facing products or end-to-end agents replacing workflows. Many AI products today are productivity tools (low autonomy).
Prompt injection/jailbreaking huge risk for AI products, unsolved/unsolvable. Will be huge problem once mainstream, especially with non-deterministic API. Tons of instructions to inject.
It's easy to get AI to do things it shouldn't, and guardrails aren't very good—you can always get around them. As agents become more autonomous and with robots, it's scary that AI could do things it shouldn't. This is a problem. However, for customers adopting AI, companies are still in early stages to gain advantage, improve processes, or streamline existing ones. 2025 has been busy for AI agents and adoption, but penetration isn't deep enough for full advantage. With human-in-the-loop points, many issues can be avoided to focus on streamlining. Companies must adopt AI—every company talked to at OpenAI has areas AI can optimize.
Successful AI products follow a success triangle: great leaders, good culture, and technical progress. Every technology problem is a people problem first.
Great Leaders
Leaders built intuitions over 10-15 years, but AI requires relearning them and being vulnerable. Example: CEO of Rackspace (Gajen) blocks 4-6 a.m. daily for catching up on AI—podcasts, information—no meetings. He does weekend white coding sessions to rebuild intuitions, accept they might be wrong, and learn from everyone. This top-down approach distinguishes successful companies. Bottom-up is impossible if leaders don't trust the technology or have misaligned expectations. Leaders must understand AI's range to guide decisions.
Good Culture
Enterprises adopting AI due to competitors need to empower subject matter experts (SMEs), crucial for defining ideal AI behavior. FOMO culture makes SMEs afraid of job replacement, so they won't engage. Leaders must foster empowerment: augment AI into workflows to 10x productivity, not replacement. This makes the organization work together, opening more opportunities.
Technical Progress
Successful teams obsess over understanding workflows to augment ripe parts with AI vs. needing human-in-the-loop. Automation mixes ML models, deterministic code—not just AI agents. Choose right tool for the problem, not obsess over technology. Understand working with non-deterministic LLM APIs means different dev lifecycle: iterate quickly without ruining customer experience, gather data to estimate behavior, build flywheels. It's not about being first with an agent, but building flywheels to improve over time.
Beware one-click agents promising gains in 2-3 days—skeptical, as enterprise data/infrastructure is messy (taxonomies, tech debt like customer data w1/w2). Obsess over problem and workflows to improve agents over time, not slap on day one. One-click agents are pure marketing. Prefer pipelines that learn over time; significant ROI takes 4-6 months even with best data/infrastructure.
CEO involvement predicts success—e.g., Dan Shipper notes CEOs chatting with ChatGPT/Claude many times daily. Rackspace CEO curates 2-3 quality sources, generates questions, bounces off AI experts.
Evals aren't everything, nor is production monitoring. No reason to trust one extreme fully.
- Evals: Encode trusted product knowledge into datasets of problems agents shouldn't fail (e.g., what matters, bad behaviors).
- Production monitoring: Key metrics on customer use (thumbs up/down, regenerations as implicit signals of unmet expectations). With AI agents, monitor more granularity—implicit/explicit feedback.
Build reliable apps that do right things or alert quickly on wrongs:
- Test pre-deployment (vibes or eval datasets of 10 critical questions that must not fail any changes).
- Post-deploy, production monitoring flags traces to review (can't eval all manually in high-throughput).
- Analyze failure patterns, build evals for critical ones (e.g., agent offering unconfigured refunds).
- Deploy v2, continue monitoring for new issues.
Both essential; one alone won't solve.
Evals hype in 2025: data labelers say experts write evals (actually error analysis notes); PMs should write evals (not full LLM judges); evals as everything feedback loop. Semantic diffusion—term butchered. All agree on actionable feedback loops for AI, but method depends on app. Complex cases hard for LLM judges (emerging patterns like verbosity); better use user signals, fix, check regression. Context matters—avoid prescriptions.
Benchmarks (LM Arena, Artificial Analysis) aren't evals; they're model benchmarks.
Codex (head Boris) skips evals for vibes: balanced—evals + customer listening. Coding agents customizable for engineers, used in varied integrations—hard to eval all. Still, evals ensure core changes don't damage. Example: code review product catches bugs at OpenAI/external; A/B test model/RL changes on mistake-finding, user reactions (annoyance leads to switching off). Also vibes, social media monitoring, team tests on hard problems per engineer/new model.
No concrete evals to bet life on—need everything: evals, customers, vibes.
Brex: intelligent finance platform for founders—high-limit cards, banking, treasury, AI agents for expenses, waste detection, reports per rules. 1 in 3 US startups use it.
Powerful software dev workflow combines discussed ideas into step-by-step for AI products. Developed due to companies pressured by competitors building agents.
Building entirely autonomous agents is difficult because initial uncertainty about user interactions and AI responses makes debugging complex multi-step workflows challenging. Extensive hotfixes become necessary, as seen in a customer support use case where the product was shut down due to unmanageable emerging problems. Recent incidents, like Air Canada's AI hallucinating a refund policy not in their playbook, leading to legal obligations, highlight risks of losing customer trust and causing dangerous decisions for the company.
CCCD creates a flywheel for improving AI products while maintaining trust. It consists of two parts:
Continuous Development:
- Scope capabilities and curate data on expected inputs and outputs.
- Aligns teams (PMs, subject matter experts) on product behavior before building.
- Provides a starting dataset for the AI to perform well on.
- Set up the application and design evaluation metrics (specific dimensions to focus on during evaluation).
- Deploy, run evaluation metrics.
Continuous Calibration:
- Identify unexpected behaviors post-deployment, as initial datasets are often incomplete.
- Users interact in unanticipated ways.
- Analyze behavior, spot error patterns.
- Apply fixes and design new evaluation metrics for emerging patterns.
- Some errors (e.g., tool calling issues due to poor definitions) can be fixed without new metrics.
Start with low agency (fewer decisions, humans in the loop) and high control, gradually increasing agency and reducing control as you build a flywheel of observed behaviors and use cases.
Agency and control as two dimensions, progressing through versions:
-
Version 1: Routing (high control, low agency)
- Classify and route tickets to the right department.
- Enterprise routing is complex due to messy hierarchical taxonomies (e.g., shoes, women's shoes, men's shoes at same layer; duplicate 'for women/for men' sections).
- Undocumented rules exist (e.g., dead nodes from 2019).
- Humans can correct wrong routing.
- Reveals data issues to fix.
-
Version 2: Co-pilot (medium agency/control)
- Generate drafts based on standard operating procedures.
- Log human edits/omissions for free error analysis to feed back into flywheel.
-
Version 3: End-to-End Resolution (high agency, low control)
- Draft resolutions and close tickets.
Lessons per Version (Table Summary):
- Routing: Better routing data, prompts, context engineering structure.
- Co-pilot: Human feedback on drafts.
- Full: End-to-end performance data.
CCCD does not fix all problems at once; new data distributions may require recalibration even after V3. It lowers risk by gathering user behavior data before full autonomy. Builds implicit logging for emerging patterns that evals miss (evals only catch known errors).
Other constraints: Limit by number of actions, topics (high-risk domains need more control), or complexity. Align PMs, engineers, SMEs.
Minimize surprise to advance stages: Calibrate every 1-2 days; if no new data patterns or user behaviors, proceed.
Events disrupting calibration: Model changes (e.g., GPT-4 deprecation, switch to GPT-5 with different properties); evolving user behavior.
Built system for bank underwriters to pick policies from 30-40 page loan applications. Impressed initially, time savings reported. After 3-4 months, users threw entire documents and asked 'for a case like this, what did previous underwriters do?' Required major rebuild: Understand 'case like this' context (income, geo), analyze historical docs vs. simple policy lookup.
Misunderstood (not overhyped): Multi-agents
- Not just break complex problem into sub-agents connected in 'gossip protocol' expecting utopia.
- Successful patterns: Supervisor agent with sub-agents.
- Hard to control (e.g., peer-to-peer in customer support requires guardrails everywhere).
- Better: Single agent that splits its own work, or human/large agent orchestration.
Underrated: Coding agents
- Low penetration outside Bay Area despite chatter.
- 2025-2026: Huge value in process optimization.
Overrated (Ash): Jumping tools/Evals obsession
- Focus on business problem over tool-hopping.
- AI is a tool; design/product thinking (solving pain points) is more valuable/expensive than cheap building.
Proactive/Background Agents:
- Understand workflows by plugging into work contexts.
- Anticipate needs: e.g., ChatGPT Pulse daily updates; coding agent fixes Linear tickets, provides patches to review.
- Alerts: 'This might crash your site,' 'Spike here, refactor database.'
Multimodal Experiences (Ash):
- Progress in generation/understanding beyond LLMs.
- Humans are multimodal; capture nodding, boredom for richer conversation.
- Unlock data: Handwritten docs, messy PDFs unreadable today.
- References: Demis Hassabis (DeepMind/Google) on combining image models, LLMs, world models (Genie).
- Nail design, judgment, taste (implementation cheap via AI).
- Careers shift post-early years: From execution to unique perspective/agency.
- Example: New hire white-coded task-tracking app (vs. paid tool), onboarded team—shows ownership, rethinking experiences.
Start small, iterate fast, build flywheel.
Custom solutions have high maintenance costs and may need replacement as companies scale, but for small companies, the cost of building is much lower now. People who grew up in this era associate a much lower cost with building; they are enthusiastic to try new tools. This contributes to AI products' retention problem because everyone is excited to try new tools. Having agency and ownership will end the busy work era. Employees must focus on end-to-end workflows to bring real impact.
Jason Lumpkin (runs Zaster) replaced his 10-person sales team with 1.2 humans and 20 agents. One agent tracked Salesforce updates automatically from calls. A salesperson quit when the agent exposed he wasn't doing anything.
It will be harder to sit around twiddling your thumbs.
Persistence is valuable because information is at your fingertips; anyone can learn anything overnight and adopt an Iron Man approach. Going through the pain of learning, implementing, and iterating—what works and what doesn't—is the real moat for individuals building AI products.
Pain is the new moat. Successful companies in new areas succeed not by being first to market or having fancy features, but by enduring the pain of understanding non-negotiable requirements, trading them off with model capabilities. This is not straightforward; there's no textbook or threaded path. The iteration—trying approaches, building organizational knowledge from lived experiences—translates into the company's moat, like a product, eval, or tool.
Turning coal into diamond.
Be obsessed with customers and the problem. AI is just a tool. Understand workflows deeply. 80% of top AI engineers and AI PMs spend time understanding workflows, not building fancy models. They're in the weeds with customer behavior and data. Software engineers new to AI are shocked by "look at your data," but it's always been essential. Understanding users and data is a huge differentiator.
Ashwarya's recommendation: When Breath Becomes Air by Paul Kalanithi (Indian-origin neurosurgeon diagnosed with lung cancer at 31-32). Memoir written post-diagnosis, read during COVID. Key quote challenging Socrates' "the unexamined life is not worth living":
If the unexamined life is not worth living, was the unlived life worth examining?
Reminder to live, not just over-examine life, especially in AI era of constant reinvention. Stop evaling life too much.
Kiriti's recommendation: The Three-Body Problem series (3 books). Explores life outside Earth, human decision-making, geopolitics, value of abstract science to progress. Stopping science has devastating effects not noticeable daily. AI is crucial in these areas.
Host loves it (favorite sci-fi series); gets mind-blowing after middle of book 2—keep reading if it starts slow.
Host's recommendation: A Fire Upon the Deep by Vernor Vinge (recommended by Noah Smith). About AGI and superintelligence; epic, few have heard of it.
Ashwarya: Rewatching Silicon Valley—timeless, eerily similar to current AI wave. Their compression algorithm business as precursor to LMs.
Kiriti: Clair Obscur: Expedition 33 game—exceptional gameplay, story, music (unrelated to AI).
Ashwarya: Whisper Flow (conceptual transcription tool). Identifies variables in code; seamless (e.g., "add three exclamation marks" adds them). Free for a year via Lenny's annual newsletter subscription (productpass.com; 18 other products).
Kiriti: Productivity CLI tools—Raycast (shortcuts, commands; also in Lenny's product pass). Caffeinate (prevents Mac from sleeping for long code tasks; run 4-5 hours overnight).
Ashwarya: Dad's advice: "They told it couldn't be done, but the fool didn't know it, so he did it anyway." Be foolish enough to believe you can do anything with heart, despite data showing low success odds (e.g., podcasts >1K subscribers, companies >$1M ARR).
Kiriti: Steve Jobs: "You can only connect the dots looking backwards." Keep moving and experimenting; paths reveal beauty in retrospect.
Ashwarya on Kiriti: Calm, grounded sounding board; anticipates issues; kind, lets work speak; most incredible husband (married 4 years, most beautiful of her life).
Kiriti on Ashwarya: Amazing at teaching/explaining complex ideas simply + persistence. Empowers people in fast-moving AI world by simplifying overwhelming new things.
Ashwarya: LinkedIn for pragmatic AI product insights. GitHub repo (20K stars) with free AI learning resources. Course: Building Enterprise AI Products (problem-first, unlearning mindsets; link in show notes). Free resources, sessions—follow them.
Kiriti: LinkedIn; DMs open for discussing complex products, coding agents, problems.
Subscribe on Apple Podcasts, Spotify; rate/review. Past episodes at lennypodcast.com.
Keep Lenny's Podcast in your library
Save the videos and channels worth coming back to, and find them again in one place.
