arXiv:2508.18896cs.CV2025-08被引 3

提升DETR模型对人-物交互的识别精度,通过双路查询增强。

DQEN: Dual Query Enhancement Network for DETR-based HOI Detection

  • 用物体感知特征优化物体查询,聚焦人与物关系。
  • 结合CLIP语义融合交互查询,提升交互理解能力。
  • 适合做视觉交互检测的研究者和开发者参考。

人-物交互(HOI)检测旨在定位人与物的配对并识别其交互行为。近年来,基于DETR的框架已被广泛应用于HOI检测。在基于DETR的模型中,具有明确语义的查询对准确检测HOI至关重要。然而,先前方法通常依赖随机初始化的查询,导致表征模糊,限制模型性能。考虑到HOI类别中的人类固定,而物体和交互行为可变,本文提出双查询增强网络(DQEN),分别增强物体查询与交互查询。具体地,物体查询通过物体感知编码器特征进行增强,使模型更专注地以物体为中心捕捉交互关系;同时设计新型交互语义融合模块,利用CLIP模型生成的候选信息提取语义特征,优化交互查询的初始化,从而提升对交互的理解能力。此外,引入辅助预测单元以改进交互特征表示。所提方法在HICO-Det和V-COCO两个数据集上均取得具有竞争力的性能。代码已开源:https://github.com/lzzhhh1019/DQEN。

原文摘要 · Abstract (English)

Human-Object Interaction (HOI) detection focuses on localizing human-object pairs and recognizing their interactions. Recently, the DETR-based framework has been widely adopted in HOI detection. In DETR-based HOI models, queries with clear meaning are crucial for accurately detecting HOIs. However, prior works have typically relied on randomly initialized queries, leading to vague representations that limit the model's effectiveness. Meanwhile, humans in the HOI categories are fixed, while objects and their interactions are variable. Therefore, we propose a Dual Query Enhancement Network (DQEN) to enhance object and interaction queries. Specifically, object queries are enhanced with object-aware encoder features, enabling the model to focus more effectively on humans interacting with objects in an object-aware way. On the other hand, we design a novel Interaction Semantic Fusion module to exploit the HOI candidates that are promoted by the CLIP model. Semantic features are extracted to enhance the initialization of interaction queries, thereby improving the model's ability to understand interactions. Furthermore, we introduce an Auxiliary Prediction Unit aimed at improving the representation of interaction features. Our proposed method achieves competitive performance on both the HICO-Det and the V-COCO datasets. The source code is available at https://github.com/lzzhhh1019/DQEN.

HOI检测DETR查询增强语义融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。