Contributed Talk Sessions | Poster Sessions | All Posters | Search Papers

Poster Session C: Friday, August 15, 2:00 – 5:00 pm, de Brug & E‑Hall

Leveraging Vision Transformers to Propose a Context-Dependent Computational Mechanism for the Holistic Process of Faces

Srijani Saha¹, Talia Konkle¹, George A. Alvarez¹; ¹Harvard University

Presenter: Srijani Saha

Holistic processing, or the integration of facial features to build an identity representation, has offered a clever solution to a critical problem - how can we seamlessly tell apart faces when there is little inter-class variability? While seminal work in psychology has demonstrated the behavioral consequences of holistic processing where individual features and identities appear different as a result of the facial context (e.g., the Composite Face Effect, Thatcher Illusion), it has been difficult to identify a computational mechanism that operationalizes these context- dependent perceptual effects. Here, we leverage the vision transformer’s architecture to show how local perturbations in a face can update the representations of other face features, thereby affecting the identity representation. The interactions between the perturbed feature and the context updates the representation of the unchanged facial features and identity, the latter towards a different, new identity. The shift in identity primarily occurs when the local changes are naturalistic.

Topic Area: Object Recognition & Visual Attention

Extended Abstract: Full Text PDF