Contributed Talk Sessions | Poster Sessions | All Posters | Search Papers

Contributed Talk Session: Friday, August 15, 12:00 – 1:00 pm, Room C1.04
Poster Session C: Friday, August 15, 2:00 – 5:00 pm, de Brug & E‑Hall

Shared high-dimensional latent structure in the neural and mental representations of objects

Raj Magesh Gauthaman¹, Michael Bonner¹; ¹Johns Hopkins University

Presenter: Raj Magesh Gauthaman

Recent work has demonstrated that visual cortex representations of natural scenes are high-dimensional, with a power-law spectrum of stimulus-related variance. However, the statistical structure of the mental representations underlying visual behavior remains unknown — is there a limited subset of latent dimensions that fully captures human behavior on a visual task? Here, we investigate the dimensionality of visual object representations in the human mind and brain by analyzing behavioral and fMRI responses from the large-scale THINGS-data collection using spectral decomposition methods. First, we find that neural representations of objects have a high-dimensional power-law structure throughout visual cortex, replicating previous findings for natural scenes. Next, we show that mental representations of objects, inferred directly from human similarity judgments, have an underlying power-law covariance spectrum, consistent with the power-law structure observed in neural representations of these stimuli. Finally, we show that the dimensionality of shared mental and neural representations increases systematically over stages of visual processing from V1 to hV4 to LOC. Our results suggest that a shared high-dimensional latent structure underlies both mental and neural representations of objects.

Topic Area: Object Recognition & Visual Attention

Extended Abstract: Full Text PDF