- Scientists have developed a non-invasive AI system that focuses on translating a person’s brain activity into a stream of text.
- The system, called a semantic decoder, could benefit patients who have lost their ability to communicate physically.
- Once the AI system is trained, it can generate a stream of text when a participant listens to or imagines a new story.
Alex Huth (left), Shelly Jin (center), and Jerry Tang (right) prepare to collect brain activity data at the University of Texas Biomedical Imaging Center at Austin. The researchers trained their semantic decoder on dozens of hours of brain activity data from participants, collected in an fMRI scanner.
Photo: Nolan Zink/University of Texas at Austin.
Scientists have developed a non-invasive artificial intelligence system that focuses on translating a person’s brain activity into a string of texts, according to a peer-reviewed study published Monday in the journal. Natural neuroscience.
The system, called a semantic decoder, could benefit patients who have lost their ability to communicate physically after suffering a stroke, paralysis or other degenerative diseases.
Researchers at the University of Texas at Austin developed the system in part using a transformer model, similar to those that power Google Bard’s chatbot and OpenAI’s ChatGPT chatbot.
Study participants trained a decoder by listening to several hours of podcasts inside an fMRI scanner, which is a large piece of machinery that measures brain activity. The system does not require any surgical implants.
Ph.D. Student Jerry Tang prepares to collect brain activity data at the University of Texas Biomedical Imaging Center at Austin.
Photo: Nolan Zink/University of Texas at Austin.
Once the AI system is trained, it can generate a stream of text when the participant listens to or imagines a new story. The resulting text is not exact text, but rather was designed by the researchers with the intent of capturing general thoughts or ideas.
According to a press release, The trained system produces a text that most closely or accurately matches the intended meaning of the participant’s original words in about half the time.
For example, when a participant heard “I don’t have my driver’s license yet” during an experiment, the thoughts were translated to “She hasn’t even started learning to drive yet.”
“For a non-invasive method, this is a real leap forward compared to what has been done before, which was usually single words or short sentences,” Alexander Huth, one of the study leaders, said in the release. “We get the model for decoding language that continues for long periods of time with complex ideas.”
Participants were also asked to watch four video clips without sound while in the scanner, and the AI system was able to accurately describe “certain events” from them, according to the release.
As of Monday, the decoder cannot be used outside of a lab setting because it relies on an fMRI scanner. But the researchers believe it could eventually be used across more portable brain imaging systems.
The lead researchers in the study have filed a patent application for this technology under the PCT.