Contributed Talk Sessions | Poster Sessions | All Posters | Search Papers

Poster Session A: Tuesday, August 12, 1:30 – 4:30 pm, de Brug & E‑Hall

Unveiling Neural Mechanisms of Memorability: Generative AI Reveals Brain-Behavior Links

Hyewon Willow Han¹, Mansoure Jahanian², Johann Cardenas¹, Yalda Mohsenzadeh¹; ¹University of Western Ontario, ²University Health Network

Presenter: Hyewon Willow Han

Some images are more memorable than others, yet the underlying neural mechanisms of memorability are not fully understood. In this study, we introduce a novel framework, “MemBrainGen”, that connects memorability, a behaviorally defined feature of images, with the human brain responses. The framework utilizes generative deep neural network models to investigate how the human brain processes images based on their memorability across visual and memory regions of the human brain. Using MemBrainGen, we successfully manipulated the memorability of natural images in both increasing and decreasing directions, and observed that predicted activations in early-mid visual regions except for V1 showed no difference in response to memorability changes. However, brain regions associated with face and body categories, and the amygdala exhibited increased predicted activation when image memorability was increased. Most notably, V1 and place-associated regions showed lower predicted activation when images with increased memorability were presented to the model. We confirm our findings by demonstrating that brain activation-maximized images have higher memorability scores compared to their original counterparts in high-level visual and memory regions. Reversely, the memorability scores of these images were decreased in the place-selective regions. We further solidify our tested hypotheses by analyzing an independent fMRI dataset. From the univariate analysis with the independent dataset, we found that the direction of changes in brain activation is consistent with the predictions of our framework. This investigation contributes to our understanding of the cognitive processes involved in visual memory. It demonstrates the potential of integrating generative models with neuroimaging to explore the causal links between brain functions and behaviour, paving the way for the formulation of experimentally testable hypotheses.

Topic Area: Memory, Spatial Cognition & Skill Learning

Extended Abstract: Full Text PDF