Andrej Karpathy: From Vibe Coding to Agentic Engineering
In a Nutshell
Andrej Karpathy describes the shift from **vibe coding**—intuitively prompting LLMs for flawless code chunks—to **agentic engineering**, where professionals coordinate powerful, stochastic agents for 10x+ productivity while maintaining quality through oversight, taste, and verifiable outputs. LLMs represent **Software 3.0**, programmable via prompts and context windows, enabling new paradigms like neural UIs and agent-native infrastructure, but exhibit **jagged intelligence**—excelling in RL-trained domains like code/math while faltering elsewhere due to data biases. Humans retain irreplaceable roles in aesthetics, judgment, specs, and directing agents, with advice for founders to target verifiable domains for fine-tuning and build agent-first environments.
These notes were generated by AI and may contain inaccuracies.
Special guest who helped build modern AI, explain it, and rename it. Co-founded OpenAI in this office. Got Autopilot working at Tesla. Coined vibe coding last year. Recently said he's never felt more behind as a programmer.
A mixture of exhilarating and unsettling. Used agentic tools like Cursor for a year; good for chunks of code but often needed edits. December marked a stark transition: latest models produce flawless chunks, no corrections needed, leading to trusting the system and vibe coding. Many experienced AI last year as ChatGPT-adjacent, but December changed fundamentally with agentic coherent workflows working. Led to infinity side projects; side projects folder extremely full.
Software 1.0: explicit rules in code. Software 2.0: programming via datasets, objectives, neural network architectures. Software 3.0: LLMs as a programmable computer trained on internet multitask data. Programming via prompting and context window over the LLM interpreter performing computation in digital information space.
Traditional install: complex bash script for different platforms. OpenClaw install: copy-paste text to agent. Agent intelligently handles environment, debugs in loop. Programming paradigm: piece of text to copy-paste to agent.
MenuGen: App to photo restaurant menu (no pictures), OCR titles, generate images, rerender menu. Runs on Vercel. Software 3.0 version: Photo to Gemini with prompt "use Nanabanana to overlay the things onto the menu." Nanabanana renders images directly into menu photo pixels. MenuGen app spurious; shouldn't exist. Neural net does work; prompt is image, output is image, no app needed. Reframe: not speedup of existing, but new things possible. Not just faster programming, but automatable general information processing. Example: LLM knowledge bases – LLMs create wikis from documents, recompiling/reframing data. New opportunities beyond what existed.
Progression from websites (90s), mobile apps (2010s), SaaS (cloud era). Extrapolation weird: completely neural computers. Feed raw video/audio into neural net using diffusion to render unique UI. Early computing unsure if calculators or neural nets; went calculator path, now neural nets dominant (intelligence compute > CPU flops). Neural nets as host process, CPUs as co-processor, tool use as appendage for deterministic tasks.
AI automates faster where output verifiable. Traditional computers automate what specifiable in code; LLMs automate verifiable domains via RL rewards in training (giant RL environments). Leads to jagged intelligence: peaks in verifiable (math, code), rough elsewhere.
Training focus/data distribution by labs. Code valuable, so emphasized. Strawberry letter count famously wrong (now patched). New example: "Car wash 50 meters away – drive or walk?" State-of-the-art (Opus 4.7) says walk, despite refactoring 100,000-line codebases or finding zero-days.
Models jagged, so stay in loop, treat as tools. Chess improved GPT-3.5 to GPT-4 due to added chess data in pre-training. At mercy of labs' data choices; no manual. Explore: if in RL circuits/data distribution, excels; else, fine-tune.
Verifiability makes domains tractable for fine-tuning/RL even if labs ignore. Valuable RL environments exist outside labs' focus.
Everything automatable to some extent (e.g., writing via council of LLM judges). Differs in ease.
Vibe coding: raises floor – everyone can build software. Agentic engineering: preserves professional quality bar (no vulnerabilities). Coordinate spiky/stochastic/powerful agents to go faster without sacrificing quality. High ceiling; >10x engineers possible, peaks much higher.
AI-native maximize tools (e.g., Cursor, Claude Code, Codex), invest in setup like vim/VS Code.
Refactor hiring: give big project (e.g., secure Twitter clone). Simulate activity with agents; attack with 10 Codices, 5 Claude 3.5s, 4x GPT-4os – should not break.
Agents like interns; humans provide aesthetics, judgment, taste, oversight. Example: MenuGen agent mismatched Stripe/Google emails for user funds (no persistent user ID). Humans design detailed specs/docs; agents fill blanks. Details offloaded (e.g., PyTorch/NumPy API diffs: keepdims vs keepdim, dim/axis, reshape/permute/transpose). Retain fundamentals (e.g., tensor views/storage efficiency). Humans handle design, engineering sense, unique user IDs.
Ceiling keeps rising.
Hoping that code aesthetics improve. Currently, it doesn't improve because it's not part of the RL (reinforcement learning). There's probably no aesthetics cost or reward, or it's not good enough. When looking at the code, it's often not super amazing—bloaty, lots of copy-paste, awkward brittle abstractions. It works but is really gross. Hope this improves in future models.
Good example: micro GPT project, simplifying LLM training as much as possible. Models hate this and can't do it. Prompting to simplify more feels like pulling teeth, outside of the RL circuits—not light speed. People remain in charge. Nothing fundamental preventing it; labs just haven't done it yet.
Idea from piece on animals versus ghosts: Not building animals, summoning ghosts. These are jagged forms of intelligence shaped by data and reward functions, not intrinsic motivation, fun, curiosity, or empowerment from evolution.
Why framing matters: Helps wrap head around what these things are for better competence in using them. A bit of philosophizing, but recognizes they're not animal intelligences. Yelling at them has no impact. They're statistical simulation circuits: substrate is pre-training (statistics), then RL bolts on top, increasing dependences.
Mindset shift: Influences what's likely to work, how to modify. Be suspicious and figure out over time.
Deep in agents with real permissions, local context, taking actions on behalf. Future: Agent-native environment where everything is rewritten. Currently, everything written for humans.
Pet peeve: Docs tell what to do ("go to this URL"), but want copy-paste for agents.
Decompose workloads into sensors over the world, actuators over the world. Make it agent-native: Describe to agents first, automation around data structures legible to LLMs.
Hoping for agent-first infrastructure. Example: Menuguen blog post—trouble was deploying on Vercel, configuring services, DNS, menus. Ideal: Prompt LLM to build and deploy Menuguen without touching anything. Test for agent-native infrastructure.
Ultimately: Agent representations for people and organizations. My agent talks to your agent for meeting details.
What remains worth learning deeply when intelligence gets cheap?
Mind-blowing tweet: "You can outsource your thinking but you can't outsource your understanding."
Still part of the system; information must enter the brain. Becoming a bottleneck for knowing what to build, why worth doing, directing agents.
Understanding directs thinking and processing, constrained by human understanding. LM knowledge bases exciting: Process information, gain insight from different projections. Prompts for synthetic data generation over fixed data.
Example: Read article, build wiki, ask questions.
Tools enhance understanding, but humans uniquely in charge—LMs don't excel at it. Can't be a good director without it.
Keep Sequoia Capital in your library
Save the videos and channels worth coming back to, and find them again in one place.
