arXiv:2604.02071cs.CVcs.AI2026-04中稿 · CVPR

通过挖掘视觉-语言模型中的实例级上下文,提升人-物交互检测精度。

Mining Instance-Centric Vision-Language Contexts for Human-Object Interaction Detection

  • 从视觉-语言模型中提取实例内、实例间和全局上下文信息。
  • 在HICO-DET和V-COCO上达到当前最优性能,显著超越已有方法。
  • 适合关注场景理解与交互识别的研究者和开发者。

人-物交互(HOI)检测旨在单张图像中定位人与物体的配对并分类其交互行为,该任务需要强大的视觉理解与精细的上下文推理能力。近期方法利用视觉-语言模型(VLM)引入语义先验,显著提升了检测性能。然而,现有方法往往未能充分利用场景中分布广泛的上下文线索。为此,我们提出实例级上下文挖掘网络(InCoM-Net)——一种新框架,将来自VLM的丰富语义知识与物体检测器生成的实例级特征有效融合。该设计通过建模每个检测实例内部、实例之间及其周围场景上下文的关系,实现更深入的交互推理。InCoM-Net包含两个核心组件:实例级上下文精炼(ICR),分别提取来自VLM特征的实例内、实例间和全局上下文;以及渐进式上下文聚合(ProCA),迭代融合多源上下文特征与实例级检测特征,支持高层交互推理。在HICO-DET和V-COCO基准上的大量实验表明,InCoM-Net实现了当前最佳性能,显著超越先前的HOI检测方法。代码已公开于https://github.com/nowuss/InCoM-Net。

原文摘要 · Abstract (English)

Human-Object Interaction (HOI) detection aims to localize human-object pairs and classify their interactions from a single image, a task that demands strong visual understanding and nuanced contextual reasoning. Recent approaches have leveraged Vision-Language Models (VLMs) to introduce semantic priors, significantly improving HOI detection performance. However, existing methods often fail to fully capitalize on the diverse contextual cues distributed across the entire scene. To overcome these limitations, we propose the Instance-centric Context Mining Network (InCoM-Net)-a novel framework that effectively integrates rich semantic knowledge extracted from VLMs with instance-specific features produced by an object detector. This design enables deeper interaction reasoning by modeling relationships not only within each detected instance but also across instances and their surrounding scene context. InCoM-Net comprises two core components: Instancecentric Context Refinement (ICR), which separately extracts intra-instance, inter-instance, and global contextual cues from VLM-derived features, and Progressive Context Aggregation (ProCA), which iteratively fuses these multicontext features with instance-level detector features to support high-level HOI reasoning. Extensive experiments on the HICO-DET and V-COCO benchmarks show that InCoM-Net achieves state-of-the-art performance, surpassing previous HOI detection methods. Code is available at https://github.com/nowuss/InCoM-Net.

HOI检测视觉语言模型上下文推理实例感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。