解决单目图像中双手遮挡与穿插问题，实现更真实的双手重建。
From 2D Alignment to 3D Plausibility: Unifying Heterogeneous 2D Priors and Penetration-Free Diffusion for Occlusion-Robust Two-Hand Reconstruction
- 融合多种视觉基础模型的结构先验，隐式指导双手姿态对齐。
- 提出无穿插扩散模型，生成避免碰撞的三维双手配置。
- 在遮挡情况下仍能保持物理合理性和动作连贯性，适合复杂场景应用。
从单目图像中进行双手重建受复杂姿态和严重遮挡的制约，常导致交互错位和双手穿插。本文将问题解耦为2D结构对齐与3D空间交互对齐，分别设计专用模块。针对2D对齐，首次统一视觉基础模型中的异构结构先验（关键点、分割图、深度图），通过融合-对齐编码器隐式吸收其结构知识，实现基础模型级引导而无需基础模型成本。针对3D对齐，提出无穿插扩散模型，学习从穿插姿态到真实无碰撞配置的生成映射。在去噪过程中，利用碰撞梯度引导模型收敛至有效双手交互流形，保持几何与运动一致性。该生成范式使模型在遮挡或模糊输入下仍能生成物理可信结果。在InterHand2.6M与HIC数据集上，交互对齐与穿插抑制性能均达到或超过当前最优水平。
原文摘要 · Abstract (English)
Two-hand reconstruction from monocular images is hampered by complex poses and severe occlusions, which often cause interaction misalignment and two-hand penetration. We address this by decoupling the problem into 2D structural alignment and 3D spatial interaction alignment, each handled by a tailored component. For 2D alignment, we pioneer the attempt to unify heterogeneous structural priors (keypoints, segmentation, and depth) from vision foundation models as complementary structured guidance for two-hand recovery. Instead of extracting priors prediction as explicit inputs, we propose a fusion-alignment encoder that absorbs their structural knowledge implicitly, achieving foundation-level guidance without foundation-level cost. For 3D spatial alignment, we propose a two-hand penetration-free diffusion model that learns a generative mapping from interpenetrated poses to realistic, collision-free configurations. Guided by collision gradients during denoising, the model converges toward the manifold of valid two-hand interactions, preserving geometric and kinematic coherence. This generative formulation approach enables physically credible reconstructions even under occlusion or ambiguous visual input. Extensive experiments on InterHand2.6M and HIC show state-of-the-art or leading performance in interaction alignment and penetration suppression. Project: https://gaogehan.github.io/A2P/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。