Contributed Talk Sessions | Poster Sessions | All Posters | Search Papers

Poster Session A: Tuesday, August 12, 1:30 – 4:30 pm, de Brug & E‑Hall

(Dis)similarities in Temporal Neural Dynamics during Reading and Listening

Jiawei Li¹, Adrien Doerig¹, Radoslaw Martin Cichy¹; ¹Freie Universität Berlin

Presenter: Jiawei Li

When reading or listening to a story, similar understanding occurs despite different input modalities—visual for reading and auditory for listening. However, how the brain converts modality-specific stimuli into semantic representations is unclear. To determine the timing of the underlying processes, we recorded EEG data while human participants read or listened to. We then used visual, auditory, and semantic embeddings in encoding models to predict brain activity to words. Models trained on the reading and listening data respectively revealed similar processing time courses, with significant encoding peaks around 250 ms. Encoding models trained on one modality and tested on another again had a similar time course, but unexpectedly with flipped results and negative predictions. These results suggest that reading and listening share a common temporal trajectory for encoding semantic representations, but the underlying mechanisms may differ.

Topic Area: Language & Communication

Extended Abstract: Full Text PDF