Essentials: The Science of Learning & Speaking Languages | Dr. Eddie Chang
In a Nutshell
Dr. Eddie Chang's research decodes speech directly from brain activity in locked-in patients using implanted electrodes over vocal tract motor areas. His team successfully enabled a paralyzed man to produce words and sentences through machine learning, with avatars now incorporating facial expressions for more natural communication. His work also shows that stuttering stems from breakdowns in motor-auditory coordination rather than anxiety or language deficits.
These notes were generated by AI and may contain inaccuracies.
Speech corresponds to the communication signal generated by moving the mouth and vocal tract to produce words. Language is much broader and includes pragmatics, which refers to extracting the gist of what is being said, semantics which is understanding the meaning of words and sentences, and syntax which is how words are assembled in grammatical form. Speech is just one form of language. Other forms include sign language and reading.
The research focuses on speech as the production of an audio signal created by vibrations in the air from the vocal tract. These vibrations are picked up by microphones and by the tiny sensors in the ear which translate them into electrical activity. Speaking is considered one of the most complex motor skills humans perform.
Speech involves shaping the breath starting with expiration. The lungs fill and then push air out. The larynx brings the vocal folds together during exhalation. These vocal folds vibrate at high frequencies between 100 to 200 hertz when air passes through them. Men typically have a larger larynx and a resonance frequency around 100 hertz while women average around 200 hertz.
The sound generated at the larynx is called voicing and serves as the source of voice energy. This sound travels up through the pharynx into the oral cavity where the mouth, tongue, and lips shape the air to create consonants and vowels.
Vocalizations such as crying or laughter are produced differently from speech. These involve exhalation and phonation at the larynx but originate from different brain areas than speech and language regions. People with injuries to speech and language areas can often still moan or vocalize because these functions are controlled by separate neural structures that even non-human primates possess.
Conditions like brain stem stroke or ALS can result in locked-in syndrome where cognition and awareness remain intact but voluntary movement and speech are lost. In brain stem stroke the connection between the cerebrum and the spinal cord is disrupted preventing thoughts from being expressed as speech or written words.
The Bravo clinical trial was established to help such patients. The first participant was a man paralyzed for 15 years following a car accident and subsequent brain stem stroke. He could blink and had limited mouth movement but no intelligible speech. He communicated using a stick attached to his baseball cap controlled by residual neck movements to type letters one by one.
Electrodes were implanted over brain areas controlling the vocal tract including the larynx, lips, tongue, and jaw. These connect through a port in the skull to translate analog brain waves into digital signals processed by machine learning algorithms. The system was trained over weeks to recognize 50 words and generate all possible sentences from those words.
The participant could think of a prompted word such as "outside" and see it appear on screen. The system incorporates autocorrect features similar to texting to improve accuracy. This marked the first time a paralyzed individual produced words and sentences decoded directly from brain activity.
Brain-machine interface work has existed for decades primarily focused on restoring function such as arm movement or cursor control. Recent industry involvement is moving the field toward commercial medical products.
Augmentation refers to using these technologies to enhance abilities beyond normal levels such as super memory or faster communication. While humans have long used augmentation methods like coffee or nicotine, invasive neurotechnologies raise new ethical questions about surgery for non-medical purposes and access to such tools.
Current natural speech systems operate with bandwidth from millions of neurons evolved over millions of years. No existing technology approaches this level of capability. Incremental and subtle enhancements are more likely than dramatic superhuman abilities.
Facial expressions provide critical nonverbal communication cues. A quizzical look signals the need to rephrase or slow down. Visual information from mouth and jaw movements also improves speech intelligibility.
Work is underway to create fully computer-animated avatars that decode and display a person's speech movements and facial expressions. This provides more complete and natural communication for paralyzed individuals. Avatars may also offer feedback that helps people learn to use speech neuroprosthetics more effectively than text output alone.
Stuttering is a speech condition affecting articulation and the coordinated movements of the vocal tract needed for fluent speech. Language components including ideas, meaning, and grammar remain intact. People who stutter have a predisposition to the condition but do not stutter constantly.
Anxiety can trigger or worsen stuttering but is not the underlying cause. The cause remains unclear but involves breakdowns in the precise coordination of brain areas controlling the larynx, lips, and jaw. Normal fluent speech occurs without conscious awareness of these complex movements.
Stuttering can be compared to a symphony where all parts must work in precise coordination. A breakdown in this coordination leads to disfluent speech.
Therapy for stuttering often focuses on helping individuals work through anxiety and develop practical tricks to create conditions where words can actually come out. Some forms of stuttering represent initiation problems where getting started is particularly difficult—the brain may have trouble emitting the initial vowel or consonant. Much of the therapeutic work centers on establishing the right conditions for fluent speech to occur.
Auditory feedback plays a critical role in speech production. Every time a word is spoken, the speaker simultaneously hears what they are saying. This feedback loop proves important because altering auditory feedback can either increase or decrease stuttering frequency. This observation suggests the brain is not simply sending motor commands outward; it also interacts with the auditory processing regions that receive the sounds. When stuttering occurs, something in this coordination between motor output and auditory feedback appears to break down.
Not all stutterers experience symptoms constantly. In fluent moments, the brain achieves a very precise coordination necessary for fluent speech production. The specific conditions that allow this coordination to succeed versus fail remain an area of active investigation.
The conversation concludes with personal reflections on a friendship spanning approximately 38 years. The connection between the speakers was rekindled through shared work in medicine and science. The host expresses appreciation for the guest's contributions, highlighting the impact of work with patients such as Poncho and broader departmental efforts including research on spinal cord function. The guest is thanked for sharing information and for the ongoing work in the field.
Keep Andrew Huberman in your library
Save the videos and channels worth coming back to, and find them again in one place.
