Contributed Talk Sessions | Poster Sessions | All Posters | Search Papers

Poster Session B: Wednesday, August 13, 1:00 – 4:00 pm, de Brug & E‑Hall

From Pixels to Human Typicality Judgments: Disentangling Category Structure and Neural Network Representations

Itay Inbar¹, Tal Golan²; ¹Ben Gurion University of the Negev, ²Ben-Gurion University of the Negev

Presenter: Itay Inbar

Humans can consistently rate the typicality of objects with respect to basic-level categories, but what do these ratings reveal about the computational mechanisms underlying categorization? We evaluated human typicality judgments against predictions from image-computable models. Each model paired a vision transformer (ViT), trained on one of five tasks, with one of three category structure models---prototype, exemplar, or a linear decision-bound model. This yielded 15 models systematically varying in representational and category structure assumptions. We found that predictions from a prototype model using the representations of a ViT trained on image classification aligned most closely with human judgments. However, this model’s advantage over the alternatives was not consistently significant, and its performance remained well below the leave-one-subject-out noise ceiling. Simulations showed that although some models were statistically indistinguishable in prediction accuracy, all 15 made distinct predictions. We discuss experimental design considerations that may enable stronger comparisons among these alternative models.

Topic Area: Object Recognition & Visual Attention

Extended Abstract: Full Text PDF