Brain-to-speech decoding will require linguistic and pragmatic data
Date
2018Journal
Journal of Neural EngineeringPublisher
Institute of Physics PublishingType
Review
Metadata
Show full item recordAbstract
Objective. Advances in electrophysiological methods such as electrocorticography (ECoG) have enabled researchers to decode phonemes, syllables, and words from brain activity. The ultimate aspiration underlying these efforts is the development of a brain-machine interface (BMI) that will enable speakers to produce real-time, naturalistic speech. In the effort to create such a device, researchers have typically followed a bottom-up approach whereby low-level units of language (e.g. phonemes, syllables, or letters) are decoded from articulation areas (e.g. premotor cortex) with the aim of assembling these low-level units into words and sentences. Approach. In this paper, we recommend that researchers supplement the existing bottom-up approach with a novel top-down approach. According to the top-down proposal, initial decoding of top-down information may facilitate the subsequent decoding of downstream representations by constraining the hypothesis space from which low-level units are selected. Main results. We identify types and sources of top-down information that may crucially inform BMI decoding ecosystems: communicative intentions (e.g. speech acts), situational pragmatics (e.g. recurrent communicative pressures), and formal linguistic data (e.g. syntactic rules and constructions, lexical collocations, speakers' individual speech histories). Significance. Given the inherently interactive nature of communication, we further propose that BMIs be entrained on neural responses associated with interactive dialogue tasks, as opposed to the typical practice of entraining BMIs with non-interactive presentations of language stimuli. © 2018 IOP Publishing Ltd.Identifier to cite or link to this item
https://www.scopus.com/inward/record.uri?eid=2-s2.0-85056661585&doi=10.1088%2f1741-2552%2faae466&partnerID=40&md5=cddf5c0127230edb506b6dd057eb66dc; http://hdl.handle.net/10713/8775ae974a485f413a2113503eed53cd6c53
10.1088/1741-2552/aae466
Scopus Count
Collections
Related articles
- Decoding speech using the timing of neural signal modulation.
- Authors: Jiang W, Pailla T, Dichter B, Chang EF, Gilja V
- Issue date: 2016 Aug
- Decoding spoken phonemes from sensorimotor cortex with high-density ECoG grids.
- Authors: Ramsey NF, Salari E, Aarnoutse EJ, Vansteensel MJ, Bleichner MG, Freudenburg ZV
- Issue date: 2018 Oct 15
- Phonemes: Lexical access and beyond.
- Authors: Kazanina N, Bowers JS, Idsardi W
- Issue date: 2018 Apr
- Brain2Char: a deep architecture for decoding text from brain recordings.
- Authors: Sun P, Anumanchipalli GK, Chang EF
- Issue date: 2020 Dec 16
- Real-time decoding of question-and-answer speech dialogue using human cortical activity.
- Authors: Moses DA, Leonard MK, Makin JG, Chang EF
- Issue date: 2019 Jul 30