Contributed Talk Sessions | Poster Sessions | All Posters | Search Papers

Poster Session B: Wednesday, August 13, 1:00 – 4:00 pm, de Brug & E‑Hall

Do Dynamics Matter for Neural Alignment? A Comparative study of Video and Static Vision Models

Khaled Jedoui Al-Karkari¹, Yingtian Tang², Martin Schrimpf², Daniel LK Yamins¹; ¹Stanford University, ²EPFL - EPF Lausanne

Presenter: Khaled Jedoui Al-Karkari

Understanding how the brain constructs and updates visual representations from dynamic input is central to our comprehension of perception and cognition. While deep learning has achieved impressive performance in visual tasks, the extent to which these models capture the computational principles of biological vision is unclear. In this paper, we investigate the alignment between representations learned by vision Artificial Intelligence (AI) models and neural activity in biological brains. Using brain fMRI data, we benchmark a diverse range of models, trained on various tasks, in their ability to predict brain responses to image and video stimuli. Our results demonstrate a clear advantage for video-based representations over static image representations across all analyzed brain regions. Our findings suggest that temporal modeling is a key component in the development of models that better align with biological vision, providing new insights into computational modeling of vision.

Topic Area: Visual Processing & Computational Vision

Extended Abstract: Full Text PDF