Contributed Talk Sessions | Poster Sessions | All Posters | Search Papers

Poster Session C: Friday, August 15, 2:00 – 5:00 pm, de Brug & E‑Hall

Brain-Aligned Category-Selective Features from Contrastive Learning

Daniel Janini¹, Radoslaw Martin Cichy¹; ¹Freie Universität Berlin

Presenter: Daniel Janini

Researchers have long debated the origins of category-selective visual cortex. Recently, some have argued that face- and scene-selective cortex can naturally emerge from contrastive self-supervised learning instead of domain-specific learning objectives. Here, we aggregated an image set for testing classic effects of the FFA and PPA. We ran replication fMRI experiments for these effects, characterizing the FFA and PPA’s distinct feature tuning. We then applied this test battery to a self-supervised vision model, finding that its face- and scene-selective features naturally exhibit many of these effects as well. Our findings support the argument that properties of human category-selective cortex can emerge from contrastive learning objectives, though our test battery also revealed specific shortcomings that could be improved in future models.

Topic Area: Object Recognition & Visual Attention

Extended Abstract: Full Text PDF