一次性预测多人体与物体的3D接触关系，速度快精度高。
Pi-HOC: Pairwise 3D Human-Object Contact Estimation
- 为每对人-物创建专用令牌，用交互网络精炼特征
- 在MMHOI和DAMON上准确率显著提升，推理速度提高20倍
- 支持语言指令定位接触点，无需额外训练
在图像中解析真实世界的人-物交互是一项复杂的多对多挑战，尤其在分离细粒度并发物理接触时尤为困难。现有语义接触估计方法要么仅限于单人场景，要么需额外输入物体几何结构（如网格）。当前最先进方法虽利用强大视觉语言模型获取类别级语义，但在多人场景下表现不佳，且推理效率低。本文提出Pi-HOC，一种单阶段、实例感知的框架，用于所有人体-物体对的密集3D语义接触预测。该方法先检测实例，为每对人-物生成专属的HO令牌，并通过InteractionFormer进行优化；再使用基于SAM的解码器，在每个对象的SMPL人体网格上预测密集接触。在MMHOI和DAMON数据集上，Pi-HOC显著优于现有方法，准确率与定位精度更高，同时推理吞吐量提升20倍。进一步实验表明，预测的接触信息可借助测试时优化算法改善SAM-3D图像到网格重建效果，并实现无需额外训练的语言查询参考接触定位。
原文摘要 · Abstract (English)
Resolving real-world human-object interactions in images is a many-to-many challenge, in which disentangling fine-grained concurrent physical contact is particularly difficult. Existing semantic contact estimation methods are either limited to single-human settings or require object geometries (e.g., meshes) in addition to the input image. Current state-of-the-art leverages powerful VLM for category-level semantics but struggles with multi-human scenarios and scales poorly in inference. We introduce Pi-HOC, a single-pass, instance-aware framework for dense 3D semantic contact prediction of all human-object pairs. Pi-HOC detects instances, creates dedicated human-object (HO) tokens for each pair, and refines them using an InteractionFormer. A SAM-based decoder then predicts dense contact on SMPL human meshes for each human-object pair. On the MMHOI and DAMON datasets, Pi-HOC significantly improves accuracy and localization over state-of-the-art methods while achieving 20x higher throughput. We further demonstrate that predicted contacts improve SAM-3D image-to-mesh reconstruction via a test-time optimization algorithm and enable referential contact prediction from language queries without additional training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。