Video_1_Generating Natural, Intelligible Speech From Brain Activity in Motor, Premotor, and Inferior Frontal Cortices.MP4
Neural interfaces that directly produce intelligible speech from brain activity would allow people with severe impairment from neurological disorders to communicate more naturally. Here, we record neural population activity in motor, premotor and inferior frontal cortices during speech production using electrocorticography (ECoG) and show that ECoG signals alone can be used to generate intelligible speech output that can preserve conversational cues. To produce speech directly from neural data, we adapted a method from the field of speech synthesis called unit selection, in which units of speech are concatenated to form audible output. In our approach, which we call Brain-To-Speech, we chose subsequent units of speech based on the measured ECoG activity to generate audio waveforms directly from the neural recordings. Brain-To-Speech employed the user's own voice to generate speech that sounded very natural and included features such as prosody and accentuation. By investigating the brain areas involved in speech production separately, we found that speech motor cortex provided more information for the reconstruction process than the other cortical areas.
History
Usage metrics
Categories
- Radiology and Organ Imaging
- Decision Making
- Clinical Nursing: Tertiary (Rehabilitative)
- Image Processing
- Autonomic Nervous System
- Cellular Nervous System
- Biological Engineering
- Sensory Systems
- Central Nervous System
- Neuroscience
- Endocrinology
- Artificial Intelligence and Image Processing
- Signal Processing
- Rehabilitation Engineering
- Biomedical Engineering not elsewhere classified
- Stem Cells
- Neurogenetics
- Developmental Biology