arXiv:2507.01630cs.CVcs.AI2025-07ICCV被引 2

通过提示与人体邻近感知,提升人-物接触区域检测精度。

Prompt Guidance and Human Proximal Perception for HOT Prediction with Regional Joint Loss

  • 用文本提示引导网络关注相关区域,增强语义关联性。
  • 引入深度感知机制,消除非交互区域干扰,提升定位准确性。
  • 设计区域联合损失和新评估指标,解决类别混乱与负样本问题。

人-物接触(HOT)检测旨在识别人体与物体实际接触的具体区域。现有模型多局限于单一图像类型,常在低交互区域产生过度分割,且难以保持特定区域内类别一致性。为此,本文提出一种名为P3HOT的框架,融合提示引导与人体邻近感知机制。首先,采用语义驱动的提示机制,根据图文相关性引导网络注意力至关键区域;其次,引入可学习参数的人体邻近感知机制,动态感知人体周围的深度范围,有效抑制无交互区域的误判。通过深度计算缓解2D视角下的重叠不确定性,提供准3D观察视角。此外,设计了区域联合损失(RJLoss),用于抑制同一区域内的异常类别;并提出新评估指标AD-Acc.,以更好处理负样本。在两个基准数据集上的实验表明,该方法在四项指标上达到最优性能:在HOT-Annotated数据集上,SC-Acc.、mIoU、wIoU和AD-Acc.分别提升0.7、2.0、1.6和11.0。代码已开源。

原文摘要 · Abstract (English)

The task of Human-Object conTact (HOT) detection involves identifying the specific areas of the human body that are touching objects. Nevertheless, current models are restricted to just one type of image, often leading to too much segmentation in areas with little interaction, and struggling to maintain category consistency within specific regions. To tackle this issue, a HOT framework, termed \textbf{P3HOT}, is proposed, which blends \textbf{P}rompt guidance and human \textbf{P}roximal \textbf{P}erception. To begin with, we utilize a semantic-driven prompt mechanism to direct the network's attention towards the relevant regions based on the correlation between image and text. Then a human proximal perception mechanism is employed to dynamically perceive key depth range around the human, using learnable parameters to effectively eliminate regions where interactions are not expected. Calculating depth resolves the uncertainty of the overlap between humans and objects in a 2D perspective, providing a quasi-3D viewpoint. Moreover, a Regional Joint Loss (RJLoss) has been created as a new loss to inhibit abnormal categories in the same area. A new evaluation metric called ``AD-Acc.'' is introduced to address the shortcomings of existing methods in addressing negative samples. Comprehensive experimental results demonstrate that our approach achieves state-of-the-art performance in four metrics across two benchmark datasets. Specifically, our model achieves an improvement of \textbf{0.7}$\uparrow$, \textbf{2.0}$\uparrow$, \textbf{1.6}$\uparrow$, and \textbf{11.0}$\uparrow$ in SC-Acc., mIoU, wIoU, and AD-Acc. metrics, respectively, on the HOT-Annotated dataset. The sources code are available at https://github.com/YuxiaoWang-AI/P3HOT.

人物接触视觉理解深度感知目标检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。