Contributed Talk Sessions | Poster Sessions | All Posters | Search Papers

Poster Session B: Wednesday, August 13, 1:00 – 4:00 pm, de Brug & E‑Hall

Learning Multisensory Representations Using Predictive Coding

Parva Alavian¹, Kwangjun Lee¹, Matthias Brucklacher¹, Jorge Mejias¹, Sander Bohte¹, Cyriel Pennartz; ¹University of Amsterdam

Presenter: Parva Alavian

Integration of information arriving in the brain from different sensory modalities is essential for robust perception. In this work, we use the predictive coding framework - a prominent theory of cortical processing- to perform multisensory representation learning. Our model can learn meaningful joint representations from two separate streams of data. These representations function as a form of hetero-associative memory, allowing the network to recall or reconstruct one modality from the other. The reconstructed outputs preserve class-relevant features, even in the absence of one sensory modality. These results suggest that predictive coding networks can serve as a biologically plausible framework for modeling multisensory representation learning.

Topic Area: Predictive Processing & Cognitive Control

Extended Abstract: Full Text PDF