Contributed Talk Sessions | Poster Sessions | All Posters | Search Papers

Poster Session B: Wednesday, August 13, 1:00 – 4:00 pm, de Brug & E‑Hall

Video DNNs do not see motion illusions

Marvin Rainer Maechler¹, Ansh Soni¹; ¹University of Pennsylvania, University of Pennsylvania

Presenter: Marvin Rainer Maechler

A key goal of computational neuroscience is to develop models that faithfully mimic human brain processing, including the efficiencies behind it. Visual illusions provide crucial test cases for this, as they are often the consequence of such efficiencies under biological constraints (e.g. efficient coding or optimal inference). Here we utilize illusions to investigate whether computer vision models process video inputs similarly to humans. Focusing on the double-drift illusion, we compare the representational geometry of these video models with behavioral and fMRI data from human subjects viewing the same stimuli. Representational similarity analyses reveal that while these models lack behavioral similarity to human observers, they do mimic the representational structure of some brain areas early in the visual processing hierarchy. Our findings demonstrate that, unlike humans, current vision models represent the physical stimulus at all points and do not combine motion and position information in a human-like manner. We thus find fundamental differences between human vision and DNNs in how temporal visual information is processed at later stages.

Topic Area: Visual Processing & Computational Vision

Extended Abstract: Full Text PDF