Contributed Talk Sessions | Poster Sessions | All Posters | Search Papers
Poster Session A: Tuesday, August 12, 1:30 – 4:30 pm, de Brug & E‑Hall
Five animacy dimensions and the CLIP model explain complementary components of visual representational dynamics and similarity judgments
Kamila Maria Jozwik1, Radoslaw Martin Cichy2, Nikolaus Kriegeskorte3; 1University of Cambridge, 2Freie Universität Berlin, 3Columbia University
Presenter: Kamila Maria Jozwik
Distinguishing animate from inanimate things is important for object recognition behaviour and animate and inanimate objects elicit distinct brain and behavioural responses. A recent study evaluated the importance of five object dimensions related to animacy (“being alive”, “looking like an animal”, “having agency”, “having mobility”, and “being unpredictable”) in brain representations and similarity-judgement behaviour. The study introduced a stimulus set that decorrelated these dimensions based on human ratings. Here, we ask: 1) to what extent one of the best computational models of vision (Contrastive Language-Image Pre-Training (CLIP) RN50) can predict dynamic human brain (EEG) and similarity judgement responses to this stimulus set and 2) what unique variance is explained by each animacy dimension ratings and CLIP. We find that CLIP explains a unique portion of the variance of similarity judgements, and a similar total amount of the variance as human ratings for each of the animacy dimensions. EEG responses are also predicted by animacy dimension ratings and CLIP to a similar extent. However, CLIP explains a unique portion of this variance at short latency (140-196 ms after stimulus onset), whereas “looking like animal” dimension rating explains unique variance at longer latency (239-301 ms after stimulus onset). We conclude that both human-generated multi-dimensional animacy ratings and the CLIP model explain unique components of visual representational dynamics and similarity-judgement behaviour and provide insights about specific dimensions of animacy that need to be better captured in future computational models of brain function and behaviour.
Topic Area: Object Recognition & Visual Attention
Extended Abstract: Full Text PDF