Contributed Talk Sessions | Poster Sessions | All Posters | Search Papers

Contributed Talk Session: Friday, August 15, 12:00 – 1:00 pm, Room C1.04
Poster Session C: Friday, August 15, 2:00 – 5:00 pm, de Brug & E‑Hall

A 7T fMRI dataset of synthetic images for out-of-distribution modeling of vision

Alessandro Thomas Gifford¹, Radoslaw Martin Cichy¹, Thomas Naselaris², Kendrick Kay³; ¹Freie Universität Berlin, ²University of Minnesota, Minneapolis, ³University of Minnesota - Twin Cities

Presenter: Alessandro Thomas Gifford

Large-scale visual neural datasets such as the Natural Scenes Dataset (NSD) are boosting NeuroAI research by enabling computational models of the brain with performances beyond what was possible just a decade ago. However, these datasets lack out-of-distribution (OOD) components, which are crucial for the development of more robust models. Here, we address this limitation by releasing NSD-synthetic, a dataset consisting of 7T fMRI responses from the eight NSD subjects for 284 carefully controlled synthetic images. We show that NSD-synthetic’s fMRI responses are OOD with respect to NSD, that brain encoding models exhibit reduced performance when tested OOD on NSD-synthetic compared to when tested in-distribution (ID) on NSD, and that OOD tests on NSD-synthetic reveal differences between encoding models not detected by ID tests—specifically, self-supervised deep neural networks better explain neural responses than their task-supervised counterparts. These results showcase how NSD-synthetic enables OOD generalization tests that facilitate the development of more robust models of visual processing, and the formulation of more accurate theories of human vision.

Topic Area: Visual Processing & Computational Vision

Extended Abstract: Full Text PDF