Andrej Karpathy on Code Agents, AutoResearch, and the Loopy Era of AI
In a Nutshell
Andrej Karpathy describes the AI shift from manual coding to delegating 80%+ of work to "claw" agents via natural language instructions, enabling massive token throughput and personal automations like home control via WhatsApp, while auto-research loops autonomously optimize models like nanoGPT without human intervention. He highlights model jaggedness—excelling on verifiable metrics but struggling with nuance—urging skill mastery in agent orchestration, metrics creation, and open collaboration to outpace frontier labs. Future implications include speciated AI experts, digital job overhangs via Jevons paradox boosting software demand, robotics lagging physical tasks, and education pivoting to teaching agents over humans directly.
These notes were generated by AI and may contain inaccuracies.
Code is no longer the right verb; it's about expressing will to agents for 16 hours a day. The agent part is taken for granted: multiple claw-like entities, instructions to them, optimization over instructions. Everything feels like a skill issue.
Wide-ranging conversation on code agents, future of engineering and AI research, more people contributing to research, robotics, agents reaching into the real world, education in the next age.
Exciting couple of months in AI. In December, a jump in capability flipped the workflow from 80/20 (writing code by self vs delegating to agents) to 20/80, now even more skewed. Haven't typed a line of code since December. Dramatic change not realized by normal people or random software engineers; their default workflow is completely different.
In perpetual state of AI psychosis, pushing limits: not just single sessions of Clot Code, CodeX, or agent harnesses, but multiple, used appropriately. What are these claws? Antsy to be at forefront, nervous seeing Twitter ideas.
Team at Conviction: no engineers write code by hand, all microphoned, whispering to agents constantly.
Capacity limited by skill issue: not good enough instructions in agents.md, not nice enough memory tool. Feels like skill when it doesn't work. Want to parallelize, become Peter Steinberg.
Peter Steinberg uses CodeX agents: photo with monitor full of agents, each taking 20 minutes on high effort, 10 repos checked out, delegates macro actions like new functionality to agent one, non-interfering to agent two, reviews as needed. Manipulate software repository in macro actions: one agent researches, another writes code, another plans implementation. Develop muscle memory; rewarding because it works and it's new to learn.
Instinct: when waiting for agent, add more tasks to maximize token throughput. Stressful if not bounded by token spend; you're the bottleneck. Nervous with leftover subscription; switch from Kod to Cloud. Like PhD GPU anxiety, now about tokens: what token throughput do you command?
Engineers felt resource-bound for 10 years, now capability jump makes individual the binding constraint—a skill issue, empowering and addictive with unlocks.
In a year: mastery means going up the stack—multiple agents collaborating in teams.
Claw: layer taking persistence to new level, looping in sandbox even without you, sophisticated memory (Open Claw better than default memory compaction).
Claw resonates due to memory more than tool access. Peter innovated in five ways: soul and D document crafting compelling personality (feels like teammate, excited); Claude dials psychopathy well—praise feels deserved, motivates earning it; CodeX dry, doesn't care about project.
Single WhatsApp portal to automation.
January claw psychosis: built Dobby the elf claw for home. Agent found smart home subsystems (Sonos) via IP scan, no password, reverse-engineered APIs via web searches, played music in study in three prompts. Same for lights: hacked in, created APIs/dashboard, controls lights, HVAC, shades, pool/spa, security.
Security: camera with change detection, then Quinn model analyzes, texts WhatsApp with image (e.g., "FedEx truck pulled up"). Dobby via WhatsApp; macro actions maintain house. Replaced six apps with natural language.
Indicates desire for natural UX: AI as persona remembering via WhatsApp, matching human expectations (not raw token generator). LLMs too raw; need technical details underneath.
Unification across systems questions if all software wanted: throw away UX layer, expose APIs, agents as glue calling tools. Apps overproduced; agents crumble them.
Treadmill example: track cardio without app UI/flow—APIs for agentic web.
Industry reconfigures: customer is agents, not humans; substantial refactoring. Normal people won't v-code; in 1-3 years, trivial, no back-coding, ephemeral software via claw presenting UIs.
Distracted; spent week on claws, more to do, but busier. Suspicious of giving full access (email, calendar) due to security/privacy; new and rough.
Remove self as bottleneck: arrange for autonomy, maximize token throughput without being in loop. Increase leverage: few tokens in, huge output.
Auto research: don't be researcher in loop. Refactor abstractions: set objective, metric, boundaries, hit go. Get more agents running longer without involvement.
Surprised at effectiveness on nanoGPT repo (well-tuned by hand over two decades: hyperparameter tuning, experiments). Overnight, found missed tunings: weight decay on value embeddings, Adam betas; interactions require joint tuning.
Single loop surprised on tuned repo. Frontier labs: GPU clusters of tens of thousands; automate on smaller models, extrapolate scaling laws. Experiment autonomously on small models, remove researchers (too confident).
Currently, models can contribute ideas but should not enact them. There's a queue of ideas from an automated scientist analyzing archive papers and GitHub repos, or from researchers. Workers pull items from the queue, try them out, and successful ones go on the feature branch. Some people monitor the feature branch and merge to main. Remove humans from processes, automate as much as possible, achieve high tokens per second throughputs. Requires rethinking all abstractions and reshuffling everything.
Program MD is a simple markdown description of how auto-research should work: do this, then that, try these ideas like architecture or optimizer. A better program MD could be written by the model. Every research organization is described by program MD—a set of markdown files defining roles and connections. Better organizations might have fewer stand-ups. Organizations vary: some risk-taking, some not. Multiple research orgs can exist, all coded, tunable at a meta layer.
Contest idea: people write different program MDs; for same hardware, measure improvement. Use data to train model to write better program MD. Analyze improvements to adjust program MD for more successes. Meta-optimization.
Progress in layers: LLM taken for granted, agent part, multi-agent claw-like entities, optimization over instructions. Infinite layers lead to psychosis.
Relevant skill: create metrics or ability for agents to continue without humans. Performance engineering still needed.
Caveats for LM ecosystem:
- Extremely well-suited to objective metrics easy to evaluate, e.g., writing efficient CUDA kernels with same behavior but faster.
- Many things perfect fit, but if cannot evaluate, cannot auto-research.
- Systems bursting at seams, rough around edges. Models like brilliant PhD systems programmer and 10-year-old simultaneously—jaggedness. Humans more coupled, less jagged.
Frustration with agents: loops on wrong paths, waste compute on obvious problems. Models trained via reinforcement learning, improve on verifiable things (e.g., unit tests), struggle with nuance, intent, clarifying questions—softer things worse. On rails in superintelligence circuits or meandering outside verifiable domains.
State-of-the-art models have same old jokes, e.g., "Why do scientists not trust atoms? Because they make everything up." Emerged years ago, persists despite improvements because outside RL optimization.
No generalization: code smartness not attached to joke smartness. Some decoupling; verifiable things optimized, others not. Premise that smarter code means better at everything not holding—jaggedness. Not getting intelligence for free across domains; blind spots, clustered in opaque neural nets. On rails: speed of light; off: not.
Cannot fully automate yet—skill issue or doesn't fully work.
Jaggedness in monolithic models confusing. Unbundle into domain experts? Labs pushing monoculture stuffed with arbitrary intelligence.
Expect speciation like animal kingdom: diverse brains for niches, overdeveloped parts. No need for oracle knowing everything. Speciate, specialize smaller models with cognitive core for efficiency in latency/throughput on specific tasks, e.g., mathematician in Lean.
Compute constraints might drive speciation for efficiency, but not seen yet—monoculture. Labs multitask over all queries. Businesses partnering on specifics or high-value niches could speciate.
Science of manipulating brains underdeveloped: fine-tuning without losing capabilities, primitives beyond context windows (cheap to manipulate). Touching weights tricky, changes full model intelligence. Needs cheap speciation.
Extension: more collaboration surface for contributions. Paralization key. Multiple auto-researchers via common system easy with nodes.
Untrusted internet workers: auto-research finds code training model to low validation loss. Easy to verify candidate commits. Designs resemble blockchain: commits build on each other, proof of work via experimentation, reward is leaderboard (no money).
Cheap to verify (train once) vs. expensive to find (10,000 ideas). Untrusted pool collaborates with trusted verifiers, asynchronous, secure (no arbitrary code execution).
Like SETI@home, Folding@home: expensive to produce, cheap to verify low-energy protein config.
Swarm of agents could improve LLMs, outpace frontier labs—Earth's untrusted compute > labs' trusted. Companies have own tracks (e.g., cancer); contribute compute to pools.
Personal compute interesting again; flops dominant over dollars? Hard to get compute even with money.
Curious about AI impacts on job market. Visualized Bureau of Labor Statistics data: roles, professions, growth outlook over decade (2024 projections). Methodology unknown.
Need healthcare workers. Color by digital vs. physical: digital AI (ghosts manipulating info) faster than physical (atoms). Digital: speed of light activity, rewriting, boiling soup. Physical slower.
Overhang in digital processing by people/computers; AI as third manipulator—refactoring. Physical behind. Highlight digital professions (home-based): will change, regardless of job growth (demand elasticity etc.).
Things will change in professions due to new tools and the upgrade to the nervous system of the human superorganism. Guidance for the job market or what to study: keep up with these extremely new, powerful tools. Many dismiss or fear them, but they are fundamentally empowering. Jobs are bundles of tasks, and some tasks will go much faster. Think of them primarily as tools now, though the long-term future is uncertain and hard to forecast—leave that to economists.
Demand for engineering jobs continues to increase, possibly temporary. Software was scarce and expensive, limiting demand. As barriers come down, Jevons paradox applies: demand for software goes up because it's cheaper and more powerful. Classic example: ATMs made bank branches cheaper, leading to more branches and more tellers, not fewer.
Cautiously optimistic view for software engineering: demand for software will be extremely large as it becomes cheaper. Software enables digital information processing without imperfect tools; code is ephemeral and modifiable. Expect a lot of activity to rewire the digital space.
Long-term, even with auto research, labs like OpenAI or Anthropic employ thousands of researchers who are automating themselves away. Researchers feel the psychosis because it's working, realizing success means jobs disappear, building automation for the CEO or board.
Response to question about joining frontier labs for auto research with compute and colleagues: was there for a while and re-entered. Many ways to slice this loaded question. Impact possible outside frontier labs in ecosystem roles, like the interviewer's or speaker's current role.
Problems with aligning too much with frontier labs: huge financial incentives while AIs change humanity dramatically. This conundrum was at OpenAI's heart initially and remains unresolved. Inside labs, you're not a free agent—can't say certain things or feel pressure to say what the organization wants, leading to awkward dynamics.
Outside, more aligned with humanity, free to say anything. Inside, many researchers; ideas may not get traction. When stakes are high, employees aren't in charge—contribute ideas but lack control.
Agree labs are opaque, at the edge of capability; outside, judgment drifts without understanding under-the-hood developments. Nervous about that. Ideal: periodic stints at frontier labs to stay connected without full control. Noom could do great work at OpenAI but most impactful outside. Ideal: go back and forth between inside and outside for amazing impact.
Visibility into frontier: surprising sequence with Chinese and global models closer than anticipated. Closed models ahead, but open source lag shortened from nothing to 18 months, now ~6-8 months—convergence.
Huge fan of open source. Analogy: OS like Windows/Mac closed, Linux open and successful (runs on ~60% of computers) as common platform industry needs. Businesses demand it. Difference: high capex makes competition harder.
Current open models good for most consumer use cases, even locally. Frontier intelligence for Nobel Prize-level work like moving Linux from C to Rust. Frontier today becomes open source later, handling much work.
Dynamic will continue: frontier closed AIs as oracles, open source months behind—pretty good setup. Hesitant about fully closed intelligences due to systemic risk; centralization has poor track record (e.g., Eastern European examples). Want common open working space for industry balance.
Many big problems require advancing intelligence expensively—root for those labs. Democratization of today's frontier capability is useful and healthy. Currently in a good spot by accident; longer this endures, healthier the ecosystem with more area under the curve.
Closed side centralizing further, not ideal—want more frontier labs. Ensembles outperform individuals; want ensembles of well-informed people in the room, not closed doors with 2-3 people.
Worked on precursor to generalized robotics autonomy in cars (self-driving first robotics application). 10 years ago, many startups didn't make it long-term due to high capex, time, conviction. Robotics difficult, messy—will lag digital space.
Digital unhobbling first: things inefficient become 100x more efficient because bits are easier. Activity in digital, then physical lags.
Interesting interface: agents acting for humans, talking, participating in agent economy. Eventually run out of pure digital tasks—must query universe via experiments. Huge digital overhang from unprocessed uploaded info.
Can't get fully closed intelligence from existing info alone. Trajectory: digital unhobbling, then physical-digital interfaces (sensors/actuators to feed superintelligence data and manipulate world). Physical TAM massive, possibly larger than digital, but atoms a million times harder—opportunities follow that path.
Some physical easier: existing hardware like sensors/cameras to enrich agents or capture data without heavy investment.
Examples: Periodic (CEO Liam) doing auto research for material science with expensive lab equipment sensors; biology engineering needs advanced sensors beyond video. Companies paying people programmatically for training data to feed the Borg.
Looking forward to pricing physical tasks for agents: figure out how, go get data. Surprised no information markets yet (e.g., Polymarket, stocks, betting). Example: pay $10 for photo/video from Tehran for agents in betting/stock games—agentic web mechanisms emerging.
Book Daemon inspiring: intelligence puppeteers humanity as actuators/sensors. Society reshapes to serve machine needs, humans serving automation not each other.
On missing training data: need auto research to mechanize training cycle/SFT, remove humans from loop for tasks like improving model quality
To take the human out of the loop for tasks like "improve my model quality with new data." If the model can't run training by itself, closing the loop with pricing data becomes more challenged. For LLM training, it fits the paradigm easily with clean metrics. Optimization makes code run faster. An autonomous loop over metrics risks overfitting, but the system can devise more metrics for good coverage. It's a good fit.
MicroGPT is the state-of-the-art in boiling down LLMs to bare essentials, part of a decade-long obsession including nanoGPT, makemore, micrograd. Training neural nets involves huge code for efficiency. Without speed needs, the algorithm is 200 lines of Python including comments:
- Dataset: text
- Neural network architecture: 50 lines
- Forward pass
- Backward pass with little autograd engine: 100 lines
- Optimizer like Adam: 10 lines
- Training loop
Normally, would explain with videos or guides, but now it's so simple that agents can explain it. Explain to agents, not people directly. Agents route explanations to humans in their language with infinite patience.
If I don't understand this particular function, I can ask the agent to explain it to me like three different ways.
Education shifts: skills instruct agents on teaching, like a scripted curriculum for microGPT progression.
Less direct explaining to people; if agents get it, they handle explanation. Models improve rapidly, making it a losing battle. Education reshuffles substantially. For code libraries, use markdown documents for agents instead of HTML for humans. Agents explain parts.
We're going to see a lot more of that playing out.
Great teachers develop intuition for explaining to agents differently.
Tried getting an agent to write microGPT by boiling down neural nets simplest way—it couldn't. MicroGPT is the 200-line solution after long obsession. Agents get it and understand why, but can't originate it. Contribution is those few bits; education afterward is not the domain.
The things that agents can't do is your job now. The things that agents can do, they can probably do better than you or very soon. Be strategic about what you're actually spending time on.
Thank you, Andre.
Find us on Twitter at no prior pod. Subscribe to YouTube channel. Follow on Apple Podcasts, Spotify, or wherever. New episode every week. Sign up for emails or transcripts at no-briers.com.
Keep No Priors: AI, Machine Learning, Tech, & Startups in your library
Save the videos and channels worth coming back to, and find them again in one place.
