i was wrong about ai

Name: i was wrong about ai
Uploaded: 2026-05-31T11:00:17.000Z
Duration: 10 min 6 s
Channel: Mo Bitar

Mo BitarMay 31, 202610m

Topics7

The Cat Experiment and Simple Cells0:00 Simple Cells, Complex Cells, and Convolutional Neural Networks1:32 Limitations of CNNs for Language and the Transformer Breakthrough2:31 The Bitter Lesson and Hardware's Role3:31 Animation Analogy and Brain Parallels5:00 Beyond Autocomplete and the World Model Question6:31 Philosophical Implications and Human Centrality8:00

In a Nutshell

The video argues that modern AI, especially transformers, works by taking simple mechanisms—like edge detection in vision or next-token prediction—and scaling them massively with compute, rather than relying on handcrafted human-like understanding. This aligns with the "bitter lesson": over time, brute-force scaling outperforms clever, biologically inspired designs. It questions whether human intelligence and consciousness are fundamentally different or just more instances of simple processes scaled up by evolution.

AI-Generated Notes

These notes were generated by AI and may contain inaccuracies.

In the spring of 1958, in a basement at Johns Hopkins, researchers had a cat strapped into a chair with its eyelids propped open and a wire going directly into the back of its skull into the part of the brain that handles vision. That wire ran to a speaker. When a brain cell fired, the speaker went off. For weeks, they flashed dots of light at this cat using little black spots on glass slides, and nothing happened. Then by chance, one of them went to swap the glass slide and as he slid it into the projector, the edge of the slide swiped across the screen and the cat's brain exploded. Not literally, but the speaker went off like a machine gun.

The neuron did not care about some dot. It cared about the edge. Not just any edge. It had to be at a very specific angle. If you tilted it away, it went quiet. If you tilted it back, it lit up again. They found a single brain cell whose entire reason for existing was basically detecting that there is a line and it leans this specific way.

What they found is that vision is not like a camera. The brain does not take a picture. Instead, the brain has a layer of cells that each give attention to one tiny thing, like a line at this angle or an edge moving that way. They called these things simple cells. Those simple cells feed into complex cells that combine the simple things into slightly less simple things. So edges become corners, corners become shapes, and shapes become a face.

They won the Nobel Prize in 1981 for their discoveries in information processing in visual systems. This was the basis of convolutional neural networks which Yann LeCun pioneered in 1989. It is basically a bunch of small filters, each one responsible for spotting something very simple in a little patch of an image like an edge. Then when you layer and stack these together, they start being able to detect bigger things like cats and dogs in a photo.

Convolutional neural networks turned out to be incredibly good at vision tasks. It is the same tech behind the cameras in self-driving cars. But they were less suited to language. The word convolutional basically refers to sliding across an image and checking one small segment at a time, which means proximity matters. Things that are close together are what get compared.

In language, the things that matter to each other can be very far apart. You can say the word "it" in a sentence and it could point back to something all the way back at the beginning of the paragraph. These networks struggled with these kinds of long-distance connections.

In 2017, a big leap came with the paper "Attention Is All You Need," which introduced transformers, which is what the T in GPT stands for. The idea is that each word, or really each token because words can get chopped into smaller pieces, can pay attention to all the other words around it. Think of every word as sending out a little signal that says this is what I am about while also listening to the signals coming from everyone else. Leaning harder towards the ones that are most relevant to it and barely paying any attention to the ones that are not. This all-at-once way of relating words is essentially what made today's large language models possible.

Sign in to read the full notes

Get access to AI-generated notes, topic timestamps, and more.

i was wrong about ai

The Cat Experiment and Simple Cells00:00

Simple Cells, Complex Cells, and Convolutional Neural Networks01:32

Sign in to read the full notes