Contributed Talk Sessions | Poster Sessions | All Posters | Search Papers

Poster Session C: Friday, August 15, 2:00 – 5:00 pm, de Brug & E‑Hall

Affective features drive human representations of social touch expressions during naturalistic viewing

Haemy Lee Masson¹, Yaxuan Zhai; ¹Durham University

Presenter: Haemy Lee Masson

Touch is a potent communication tool. It has been suggested that a wide range of factors impact how we perceive social touch. No study has investigated which features we attend to when observing complex, naturalistic social touch expressions. To address this question, we curated 125 video clips showing a wide range of social interactions from the American TV series, Modern Family. Eighty participants watched 45 pseudo-randomly selected videos and performed a multiple arrangement task. This procedure produced the group-averaged pairwise similarity judgements for social touch expressions. Visual, social, and affective features were extracted from each video clip using artificial neural networks (ANN), behavioral experiments, and human annotations. The combination of multiple regression and variance partitioning analyses revealed that selected affective, social, high-level visual, and ANN features collectively explained 52% of the variance in the perception of social touch expressions. Among these, affective features uniquely accounted for 33% of the variance. The current findings suggest that affective features, specifically whether the touch is used to convey positive or negative emotions, drive human perceptions of social touch during naturalistic viewing. Conversely, ANN features explained the least variance, suggesting that the models, trained on action perception and facial expression recognition, may not be sufficient to decode social touch expressions.

Topic Area: Reward, Value & Social Decision Making

Extended Abstract: Full Text PDF