arXiv:2510.18357cs.CV2025-10NeurIPS被引 5

将人-物交互从成对关系升级为群体行为建模,提升复杂场景识别能力。

Learning Human-Object Interaction as Groups

  • 基于空间位置与语义相似性构建动态分组,通过软对应关系传递上下文信息。
  • 在HICO-DET和V-COCO上优于当前最佳方法,尤其在非语言交互任务中表现突出。
  • 适合研究复杂人群体行为理解、多智能体交互建模的开发者与研究人员。

人-物交互检测(HOI-DET)旨在定位人与物体对并识别其交互关系。现有方法通常通过自注意力机制在所有检测实体间传播信息,或使用二分图进行人与物之间的消息传递,但主要关注成对关系,忽略了真实场景中常出现的集体行为(多人与多物共同参与的联合活动)。为此,本文从群体视角重新审视关系建模,提出GroupHOI框架,通过几何邻近性和语义相似性传播上下文信息。为利用几何邻近性,基于边界框的空间特征,使用可学习的邻近度估计器将人与物体划分为不同簇;在每组内,通过自注意力计算软对应关系,聚合并分发上下文线索。为融合语义相似性,增强基于Transformer的交互解码器,引入来自人-物对特征的局部上下文线索。在HICO-DET与V-COCO基准上的大量实验表明,GroupHOI优于当前最优方法;在更具挑战性的非语言交互检测(NVI-DET)任务中也展现出领先性能,该任务涉及群体内部多样化的高阶交互形式。

原文摘要 · Abstract (English)

Human-Object Interaction Detection (HOI-DET) aims to localize human-object pairs and identify their interactive relationships. To aggregate contextual cues, existing methods typically propagate information across all detected entities via self-attention mechanisms, or establish message passing between humans and objects with bipartite graphs. However, they primarily focus on pairwise relationships, overlooking that interactions in real-world scenarios often emerge from collective behaviors (multiple humans and objects engaging in joint activities). In light of this, we revisit relation modeling from a group view and propose GroupHOI, a framework that propagates contextual information in terms of geometric proximity and semantic similarity. To exploit the geometric proximity, humans and objects are grouped into distinct clusters using a learnable proximity estimator based on spatial features derived from bounding boxes. In each group, a soft correspondence is computed via self-attention to aggregate and dispatch contextual cues. To incorporate the semantic similarity, we enhance the vanilla transformer-based interaction decoder with local contextual cues from HO-pair features. Extensive experiments on HICO-DET and V-COCO benchmarks demonstrate the superiority of GroupHOI over the state-of-the-art methods. It also exhibits leading performance on the more challenging Nonverbal Interaction Detection (NVI-DET) task, which involves varied forms of higher-order interactions within groups.

人-物交互群体建模视觉理解Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。