Social-affective features drive human representations of observed actions

Abstract
Data availability
Article and author information
Metrics

Abstract

Humans observe actions performed by others in many different visual and social settings. What features do we extract and attend when we view such complex scenes, and how are they processed in the brain? To answer these questions, we curated two large-scale sets of naturalistic videos of everyday actions and estimated their perceived similarity in two behavioral experiments. We normed and quantified a large range of visual, action-related and social-affective features across the stimulus sets. Using a cross-validated variance partitioning analysis, we found that social-affective features predicted similarity judgments better than, and independently of, visual and action features in both behavioral experiments. Next, we conducted an electroencephalography (EEG) experiment, which revealed a sustained correlation between neural responses to videos and their behavioral similarity. Visual, action, and social-affective features predicted neural patterns at early, intermediate and late stages respectively during this behaviorally relevant time window. Together, these findings show that social-affective features are important for perceiving naturalistic actions, and are extracted at the final stage of a temporal gradient in the brain.

Data availability

Behavioral and EEG data and results have been archived as an Open Science Framework repository (https://osf.io/hrmxn/). Analysis code is available on GitHub (https://github.com/dianadima/mot_action).

The following data sets were generated

1. Dima DC
2. Tomita TM
3. Honey
4. CJ
5. Isik
6. L
(2021) Social-affective features drive human representations of observed actions
Open Science Framework, https://osf.io/hrmxn/.

https://osf.io/hrmxn/

Article and author information

Author details

Diana C Dima

Department of Cognitive Science, Johns Hopkins University, Baltimore, United States

For correspondence
ddima@jhu.edu

Competing interests
The authors declare that no competing interests exist.

"This ORCID iD identifies the author of this article:" 0000-0002-9612-5574
Tyler M Tomita

Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, United States

Competing interests
The authors declare that no competing interests exist.
Christopher J Honey

Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, United States

Competing interests
The authors declare that no competing interests exist.

"This ORCID iD identifies the author of this article:" 0000-0002-0745-5089
Leyla Isik

Department of Cognitive Science, Johns Hopkins University, Baltimore, United States

Competing interests
The authors declare that no competing interests exist.

Funding

National Science Foundation (CCF-1231216)

Leyla Isik

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Ethics

Human subjects: All procedures for data collection were approved by the Johns Hopkins University Institutional Review Board, with protocol numbers HIRB00009730 for the behavioral experiments and HIRB00009835 for the EEG experiment. Informed consent was obtained from all participants.

Copyright

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.