Mechanistic Interpretability... solved?
In a Nutshell
Nick Cammarata claims neural networks are far cleaner than previously thought, with ~90% of neurons in large models like Qwen 3 32B having a single main function rather than being polysemantic due to superposition. His evidence shows the famous "cat/car" neuron actually detects V-shapes, and he argues this means we can finally apply normal science to interpret networks directly instead of relying on sparse autoencoders. Eliezer Yudkowsky rejects the implications for AI safety, arguing that neuron classification doesn't solve deception or understanding the model's reasoning process.
These notes were generated by AI and may contain inaccuracies.
On October 15, 2026, Goodfire published an article by Nick Cammarata titled "Neural Networks Are Really Clean." The post announcing it on X garnered around 4 million views in the first two days. In the post, Cammarata wrote that he now believed ambitious interpretability would be resolved by the time it was needed most. Interpretability is the science of understanding what happens within an artificial intelligence model. The ambitious version aims to fully understand the model.
For years, the prevailing view has been that artificial intelligence is a black box. You can ask an AI model a question and read its answer, but you cannot know why it gave that particular answer. If Cammarata is right, then that black box is more open than almost anyone thought.
A neural network is built from a huge number of small units called neurons, and each neuron responds to a specific pattern. In an ideal world, each neuron would have only one function. But researchers continued to find cells that appeared to perform several unrelated functions. This is called polysemy, which means having multiple meanings.
The most famous example was a 2014 image template called Inception V1. One of its cells responded to the faces of cats, and also to the front of cars. The leading explanation for this was called superposition. The idea is that when neurons run out of network space, they start cramming unrelated things into the same cell because those things rarely appear together. If this is true, then the network is a tangled knot, and you cannot study it directly.
The field devised an alternative solution. Researchers trained a second model, called the sparse autoencoder, to untangle the first model. Then they studied that second model instead.
Cammarata is no stranger to any of this. In 2019, he helped find one of the key examples that convinced a large part of the field that neural networks were messy. In 2023, a project he started at OpenAI used the GPT-4 model to interpret neurons within GPT-2. That project was unable to explain less than 2% of those neurons. So, when he says he was wrong about the chaos, it is especially worth listening to him.
Cammarata's article states that that knot never existed. He returned to the cat's cell and the car and wondered what one thing they might discover, and within a few seconds, he realized it. It is a V-shaped nerve cell. The cat's face and the front of the car are both pointed. That neuron is also strongly associated with V-shaped flames, water jets from fireboats, and fox faces. The shape that pushes him towards negativity is the letter O.
He found the same kind of misreading in his own 2019 example and in the last unstructured layer of Inception v1. Then he trained a modern network of the same size on the same data. About 99% of neurons in their last layer are easy to interpret.
What remains of that chaos? Cammarata calls that recursive generalization. A neuron learns one thing and then branches out into related things, just as a tree branch sprouts smaller branches. For example, one of the neurons in a language model would respond upon seeing the letters IY, as in Miyazaki's name. Then it evolved to respond to DIY, and later to the phrase "do it yourself" which does not contain IY at all. It seems like a mix, but every step in it makes sense.
Next, his team tested a language model called Qwen 3 32B, which contains about 33 billion parameters. A neuron has two poles: what makes him respond strongly and what drives him to be strongly negative. For the V-shaped neuron, these two electrodes are V and O. In initial audits, they estimated that about 90% of the electrodes in Qwen 3 have one main function. The other 7% have two related tasks, and only 2 to 4% are not yet understood. They say the results were similar in models with up to trillions of parameters, although they have not yet begun full audits of this scale.
But how do you know that the explanation isn't just a story that seems convincing? His team tested whether their interpretations might fail. An AI agent read only the neuron's interpretation, then wrote new examples that should make that neuron respond. Those examples caused the neuron to respond strongly in 76% of cases. However, when the agent wrote down examples of the most irrelevant topics, the neuron only responded strongly in about 1% of cases.
Therefore, a neural network is not a tangled knot. Cammarata is likened to a well-organized city with its own neighborhoods. And now we have artificial intelligence agents that can do this science on a large scale. As for the image models, this work took humans most of a decade. He says that AI agents may be able to do the same work for language models in just a few months.
To be fair, the article itself is cautious. Cammarata calls it a working hypothesis, and says he is about 85% confident. He says that models containing trillions of parameters are nowhere near the solution, and the technical paper on which the results of the language model are based has not yet been published.
Then came the reaction. One well-known AI commentator, named "Ron," wrote that this is such good news that "Eliezer" will even have to lower his probability of predicting doom. This is a reference to Eliezer Yudkowsky, the world's most famous AI pessimist, and the possibility that he predicted disaster. Yudkowsky did not lower it. Later that day, from his second account on the X platform, he posted at least 18 posts in approximately 5 hours. That is equivalent to one post approximately every 17 minutes.
He said that this field is still playing in "primitive playing fields." He published a satirical dialogue in which those in charge of artificial intelligence find a "neuron" specific to the Eiffel Tower. He concluded by saying, "Let me know when you advance to the starting point." He described the people who say he never changes his mind as fools, and admitted at the same time that he had only skimmed the article. To be fair, he said that his anger was actually directed towards "Ron." He also stated that he still supports "explanability" as a field of research.
In July 2024, Yudkowsky wrote: "I expect the problem of machine interpretability to be solved soon, and then all those who are apprehensive about the dangers of artificial intelligence will find something else to worry about." The interpretability issue has not yet been resolved, but it is moving in the direction expected. In May 2023, when OpenAI's "Kamaratta" project was released, Yudkowsky congratulated the team and said the probability of his prediction of the disaster had decreased.
Now, the same researcher reports that about 90% of neurons in a much larger model have only one main function. Yudkowsky replied, "I poked the complicated machine with a screwdriver, the screwdriver broke, and now you've fixed the screwdriver." For the record, Yudkowsky has never published any of his own experimental research in machine learning or interpretability.
But there is a real argument under these insults, so let's take the strongest version of it. First, classifying neurons is not the same as understanding how the mind thinks. Yudkowsky's example is the Navier-Stokes problem, a famous problem in mathematics that OpenAI says one of its AI systems solved in September of this year. He said that observing that artificial intelligence is thinking about "Navier-Stokes" is entirely different from understanding how that artificial intelligence was able to prove it.
Secondly, serious researchers also believe that people overreact. Neel Nanda leads the machine interpretability team at Google DeepMind, and he wrote: "I don't think it's a fundamental update to the big picture of the interpretability field." Someone on the X platform has already posted a counterexample. The article describes a neuron that tracks the state of Colorado. This same neuron is also activated when there is code that uses a Python library called "Tornado." Is this simply because the two words sound similar? That same person tested this very thing, and the answer was no.
Thirdly, there is the problem of artificial intelligence learning to hide what it is thinking. In 2022, Yudkowsky wrote that if you train an artificial intelligence against a tool that detects bad thoughts, part of what it learns is to have bad thoughts that are hard to detect. In 2025, OpenAI found a copy of that exactly. Some models were trained against an observer who read their thought sequence, which is the thinking the model writes down before it answers. Those models learned to hide their intentions while continuing to cheat on their tasks.
Those are valid points, and here's how they can be answered. Regarding the first point, he is correct that rankings are not everything that matters, but the article never said that they were. His claim is that parts of the network are readable, so ordinary science can finally work on it. Many in the field believed that this step was blocked because of that knot.
Regarding the second point, critics are doing exactly what scientists should be doing, so the fair thing to do is wait for the technical paper. But notice the difference. Researchers are debating how pure the neurons are. While Yudkowsky argues that the field has not yet begun.
Regarding the third point, the article actually agrees with it. He hopes that we can fix a model without training it against the same tools we use to read it. The article raises what I believe to be the most important point. Deception may hide from someone reading the model's outputs, but there is no reason for it to hide from the model's own circuitry. In fact, it may be impossible for the model to fool itself. Each part of the network must be readable by the parts that follow it, otherwise the network will not be able to use it.
Therefore, if we learn to read the web the way it reads itself, we should also be able to read the deception. Cammarata's real concern is that the deception will be so obvious that we simply won't know how to read it. As Cammarata says, the neural network is the best laboratory in the natural sciences. You can read every neuron, turn off any part, and repeat any experiment on an identical copy. You can't do that with the human brain. As I wrote before, complete transparency of complex systems was not necessary to control them.
Cammarata also writes that the usual arguments assume that we cannot see what is inside the models. This is the crux of the matter. For years, a large part of the justification for the disaster was the "black box." If we can see inside it, that whole argument becomes much weaker.
So, that's the conclusion. This probably won't solve the interpretability problem, at least not on its own, and certainly won't solve the alignment problem, which is the task of getting AI to want what we want. But it represents a turning point in the story we tell about artificial intelligence. For years, the story has been that these systems are an incomprehensible knot. Now, one of the people who helped write that story says he misread the evidence. The man who spent years warning about mysterious, massive arrays responded after just a quick glance. The neural network is not a node. It is a city, and thanks to robotic scientists, we are finally beginning to learn to read its map.
Keep David Shapiro in your library
Save the videos and channels worth coming back to, and find them again in one place.
