Contributed Talk Sessions | Poster Sessions | All Posters | Search Papers

Poster Session B: Wednesday, August 13, 1:00 – 4:00 pm, de Brug & E‑Hall

Training on Ecologically Relevant Tasks Improves Alignment Between Artificial Neural Network and Human Similarity Judgments

Aidan J. Seidle¹, Jenelle Feather², Malinda McPherson-McNato¹; ¹Purdue University, ²Flatiron Institute

Presenter: Aidan J. Seidle

Artificial neural networks (ANNs) have emerged as leading models for predicting human behavior and neural data. While these models have been extensively studied in the visual domain, their effectiveness in modeling audition is comparatively underexplored. Recent work has found that some ANNs can predict aspects of auditory cortical processing, however it is not clear whether these models capture task-invariant representations of sounds. Here, we used human judgments of similarity as a benchmark for the generalization of auditory model representations. We hypothesized that similarity scores computed from models that best predicted neural activation patterns would strongly correlate with human similarity judgments. We compared human similarity judgments of pairs of sounds to cosine similarity calculated from different layers of seventeen ANNs, as well as a basic spectrotemporal model. The ANNs exhibited a wide range of variability in their correlations with human similarity judgments, and the best models were those trained on multiple tasks. Although there was a significant correlation between the ability of a model to predict fMRI data and the alignment with human similarity measurements, some models showed diverging values. This result suggests that separate criteria for correspondence to human behavior, neural data, and higher-level psychological processes may be necessary.

Topic Area: Visual Processing & Computational Vision

Extended Abstract: Full Text PDF