Contributed Talk Sessions | Poster Sessions | All Posters | Search Papers

Poster Session A: Tuesday, August 12, 1:30 – 4:30 pm, de Brug & E‑Hall

Optimizing fMRI Data Acquisition for Decoding Natural Speech with Limited Participants

Louis Jalouzot¹, Alexis Thual², Yair Lakretz³, Christophe Pallier⁴, Bertrand Thirion⁵; ¹Ecole Normale Supérieure de Lyon, ²CEA, ³Ecole Normale Supérieure de Paris, ⁴Centre National de la Recherche Scientifique, ⁵INRIA

Presenter: Louis Jalouzot

We investigate optimal strategies for decoding natural speech from fMRI data with limited participants. Using data from Lebel et al. (2023) from 8 participants, we show that deep neural networks can effectively predict LLM-derived text representations with performance directly scaling with the amount of training data. Then, in this data regime, we observe that multi-subject training does not improve decoding accuracy compared to a single-subject approach. Furthermore, we find that our decoders better model syntactic than semantic features. Our results highlight deep phenotyping benefits and suggest multi-subject decoding needs more data per subject or a substantially larger cohort.

Topic Area: Language & Communication

Extended Abstract: Full Text PDF