首个第一视角社交群体检测数据集，覆盖全球65国真实场景。
EgoGroups: A Benchmark For Detecting Social Groups of People in the Wild
- 构建第一人称视角数据集，捕捉城市中真实社交动态。
- 模型在零样本下表现超越传统监督方法，但受人群密度与文化差异影响。
- 适合研究跨文化社交行为、智能体社会感知的学者使用。
社交群体检测（识别相互互动的人类群体，如家人、朋友、顾客与商家）是智能体在现实世界交互中所需的社会智能关键组成部分。现有基准因场景多样性低且依赖第三人称视角（如监控录像）而受限，难以评估群体在多元文化背景和非约束环境下的形成与演化。为此，我们提出EgoGroups——首个第一人称视角数据集，涵盖全球65个国家，覆盖低、中、高人流场景，以及四种天气/时段条件。数据包含密集的人物与社交群体标注，以及丰富的地理与场景元信息。基于该数据集，我们对前沿视觉语言模型（VLM）与大语言模型（LLM）及监督模型进行了全面评估。发现：在零样本设置下，VLM与LLM可超越监督基线；但人群密度与文化区域显著影响模型性能。
原文摘要 · Abstract (English)
Social group detection, or the identification of humans involved in reciprocal interpersonal interactions (e.g., family members, friends, and customers and merchants), is a crucial component of social intelligence needed for agents transacting in the world. The few existing benchmarks for social group detection are limited by low scene diversity and reliance on third-person camera sources (e.g., surveillance footage). Consequently, these benchmarks generally lack real-world evaluation on how groups form and evolve in diverse cultural contexts and unconstrained settings. To address this gap, we introduce EgoGroups, a first-person view dataset that captures social dynamics in cities around the world. EgoGroups spans 65 countries covering low, medium, and high-crowd settings under four weather/time-of-day conditions. We include dense human annotations for person and social groups, along with rich geographic and scene metadata. Using this dataset, we performed an extensive evaluation of state-of-the-art VLM/LLMs and supervised models on their group detection capabilities. We found several interesting findings, including VLMs and LLMs can outperform supervised baselines in a zero-shot setting, while crowd density and cultural regions clearly influence model performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。