Contributed Talk Sessions | Poster Sessions | All Posters | Search Papers

Poster Session A: Tuesday, August 12, 1:30 – 4:30 pm, de Brug & E‑Hall

Mapping Modular Processing of Compressed Videos Across Human Visual Cortex

Christina Sartzetaki¹, Iris Groen¹; ¹University of Amsterdam

Presenter: Christina Sartzetaki

How the brain makes sense of the constant stream of visual information it receives largely remains a mystery. One prominent idea is that it evolved specialized pathways for sparse cortical engagement, and those can be accurately captured with handcrafted features; however, deep neural network (DNN) features overall align better with brain representations. In this work we study the brain alignment of a multi-pathway DNN that leverages compressed video formats, and partition the variance captured between its three modular components across visual brain regions recorded with fMRI during video stimuli. We find that its components map well to known brain pathways, and that it captures overall more variance than a 3D convolutional network. Achieved using only existing features in the compressed format, this points to the ineffectiveness of conventional full-frame processing for explaining brain responses to dynamic stimuli and to compression as a potential solution.

Topic Area: Visual Processing & Computational Vision

Extended Abstract: Full Text PDF