OpenAI founder admits AI isn’t working
In a Nutshell
Karpathy admits that while AI coding tools produce working code, the output is bloated, brittle, and full of poor abstractions that often require him to intervene to avoid serious errors. He describes his workflow of writing highly detailed specs to guide the AI, yet still finds the models unable to simplify or reason beyond their training data, revealing them as sophisticated autocomplete systems rather than true engineers. The core takeaway is that current frontier AI lacks reliable reasoning and code quality, making agentic workflows risky for critical applications like healthcare or finance despite the hype.
These notes were generated by AI and may contain inaccuracies.
Andre Karpathy, one of the founders of OpenAI, helped invent the modern AI stack and coined the term vibe coding. At a conference, he described using AI coding tools and how he has stopped checking their output. He explained that he used to correct the output constantly, but now he does not. He presented this as a productivity progression where the tip is to give up on reviewing the code.
Immediately after this story, Karpathy shared a contrasting observation. He noted that these models can refactor a 100,000-line codebase and perform incredible software engineering tasks, yet they fail at simple tasks like counting the number of Rs in the word "strawberry" or deciding whether to drive or walk to a car wash. He asked how this discrepancy is possible.
Karpathy gave an example from an app he was working on called Menu Genen. An agent made an assumption about reusing emails that did not make sense. He stated that if he had not caught this error, it would have been really catastrophic. He presented this as a charming personal anecdote about how agents do silly things.
The speaker emphasizes that these mistakes are not funny because the same technology is supposed to replace doctors and handle taxes. In a moment of honesty, Karpathy said that when he actually looks at the code, sometimes he gets a little bit of a heart attack because it is not super amazing code. The code is very bloaty, with a lot of copy-paste and awkward abstractions that are brittle. It works, but it is just really gross.
This situation reveals the current state of frontier AI. Engineers with jobs do not admit this because they are riding on the fact that CEOs do not know about the poor code quality. Engineers who hype AI are hoping to hide the fact that the code gives someone like Karpathy a heart attack every time he looks at it. Every software startup will now need a ping-pong table, espresso machine, and a guy with a defibrillator waiting for someone to open a pull request. Pair programming will be reinvented as you and a cardiologist pair programming to ensure you do not pass out from the slop. Standups will change to include questions about chest pain.
Karpathy describes his workflow as writing an extremely detailed document in markdown in English that explains everything he wants the code to do. He includes every single edge case and tries to be as precise as possible to get the best results. The AI then writes the code based on this document. When he looks at the AI-generated code, he asks it to simplify the code, but the model responds that it cannot simplify it any more than this. Karpathy can clearly see ways to simplify the code that the AI cannot figure out.
Karpathy explains this limitation through reinforcement learning. The cherry on top of the base training data is a really curated training set that teaches the models very specific skills. If the task is not well represented either in the base data or in the RL data, there is no force on this planet that can make that LLM solve the problem. He is honest that this is just a very sophisticated autocomplete.
Karpathy presents a contradiction. On one hand, he says he stopped checking the work because the models have made fewer mistakes. On the other hand, he says he gets a heart attack when he looks at the code and acknowledges that the model has made dumb mistakes in his codebase. When he started the interview, he wanted to make it clear that he is not an AI skeptic but an AI accelerationist who supports agentic and vibe coding.
Karpathy offers advice for hiring strong agentic engineers. Most companies have still not refactored their hiring process for agentic engineering capability. They are still interviewing on LeetCode and weird technical problems like giving out puzzles to solve. He proposes that hiring should involve giving candidates a really big project to implement, such as writing a Twitter clone for agents that is really good and really secure.
Before, candidates would practice LeetCode and abstract puzzles. Now, the skill to practice is writing specs. If an interviewer asks you to build a clone of Twitter X, the question becomes what spec you would write. If you have never practiced writing a spec about building a Twitter clone, you might not know what problems there are to solve. You will write something really simple, but as the agent builds it, it will start choking on things you did not properly specify, such as tokens, session length, cookie expiration, change password, and rate limiting.
If you practice building a recommendation engine at home, it would be very impressive in an interview. The speaker has said that there is not much skill in agentic engineering because Claude is doing most of the things. The real skill is not giving something to Claude until it is ready, otherwise you waste time going back and forth. The more you can one-shot things, the better. If you have not practiced building a recommendation engine before, you would never suddenly add it to your prompt in an interview. If you practice it a couple of times, it is easy to do in an interview and surprise people.
The thing you will be tested on in an interview is your ability to one-shot. You will not have time to sit there and work with an agent for four hours unless it is a take-home. For employers who have not updated their hiring process, giving a big project is one way to approach it. Karpathy does not really have the answers to what is happening in the industry. When the interviewer asked what things will be valuable in the future if AI gets good and what is still worth learning, Karpathy gave a response but it was sort of empty. These are hard questions that nobody has figured out, even Karpathy.
If you are feeling a little lost, there is some sense of solace in the fact that even Andre feels a little lost.
Keep Mo Bitar in your library
Save the videos and channels worth coming back to, and find them again in one place.
