人类通过语义结构理解简单移动图形中的社交互动。
Human Semantic Representations of Social Interactions from Moving Shapes
- 用人类对动态图形的感知标签构建社会互动语义表征
- 动词嵌入模型解释人类相似性判断效果最佳
- 语义信息能补充视觉特征，揭示社交感知机制
人类能从简单移动图形中迅速识别各种社交互动。以往研究多聚焦于视觉特征，本文探讨人类在补足视觉信息时所依赖的语义表征。实验1中，要求参与者根据移动图形的印象进行标签标注，结果显示人类反应具有分布性。实验2通过人类相似性判断测量27种社会互动的表征几何结构，并与基于视觉特征、标签及动画描述语义嵌入的模型预测进行对比。结果表明，语义模型为解释人类判断提供了互补信息；其中，从描述中提取的动词嵌入模型对人类相似性判断的解释力最强。这说明，在简单显示中对社交互动的感知反映了社会互动的语义结构，连接了视觉与抽象表征。
原文摘要 · Abstract (English)
Humans are social creatures who readily recognize various social interactions from simple display of moving shapes. While previous research has often focused on visual features, we examine what semantic representations that humans employ to complement visual features. In Study 1, we directly asked human participants to label the animations based on their impression of moving shapes. We found that human responses were distributed. In Study 2, we measured the representational geometry of 27 social interactions through human similarity judgments and compared it with model predictions based on visual features, labels, and semantic embeddings from animation descriptions. We found that semantic models provided complementary information to visual features in explaining human judgments. Among the semantic models, verb-based embeddings extracted from descriptions account for human similarity judgments the best. These results suggest that social perception in simple displays reflects the semantic structure of social interactions, bridging visual and abstract representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。