提出可微分的双手物交互表示，提升生成与修复的准确性和物理合理性。
A Versatile and Differentiable Hand-Object Interaction Representation
- 用无符号距离和高斯分布建模手物连续交互场，支持可微计算。
- 在交互修复中接触分数提升5%，在合成中相似位移降低46%。
- 适用于多种任务，适合需要真实物理交互的视觉与虚实融合应用。
在计算机视觉、增强现实（AR）和混合现实（MR）中，精确合成手物交互（HOI）至关重要。尽管近期进展显著，现有方法在重建或生成的准确性上仍有提升空间。部分技术通过从显式接触转向使用丰富的HOI场来改进密集对应关系，但缺乏完全可微性或连续性，且仅针对特定任务。为此，我们提出一种粗粒度手物交互表示（CHOIR），一种新型、通用且全可微的HOI建模场。CHOIR利用无符号距离实现形状与姿态的连续编码，并采用多变量高斯分布以少量参数表示密集接触图。为验证其通用性，我们设计了JointDiffusion，一种扩散模型，可基于噪声的手物交互或仅物体几何条件学习抓取分布，适用于修复与合成。实验表明，联合使用CHOIR与JointDiffusion，在两类任务中均优于现有最优方法：修复时接触F1分数提升5%，合成时相似位移降低46%。结果表明，该方法在接触精度与物理真实性方面显著优于专用于特定任务的主流方法。
原文摘要 · Abstract (English)
Synthesizing accurate hands-object interactions (HOI) is critical for applications in Computer Vision, Augmented Reality (AR), and Mixed Reality (MR). Despite recent advances, the accuracy of reconstructed or generated HOI leaves room for refinement. Some techniques have improved the accuracy of dense correspondences by shifting focus from generating explicit contacts to using rich HOI fields. Still, they lack full differentiability or continuity and are tailored to specific tasks. In contrast, we present a Coarse Hand-Object Interaction Representation (CHOIR), a novel, versatile and fully differentiable field for HOI modelling. CHOIR leverages discrete unsigned distances for continuous shape and pose encoding, alongside multivariate Gaussian distributions to represent dense contact maps with few parameters. To demonstrate the versatility of CHOIR we design JointDiffusion, a diffusion model to learn a grasp distribution conditioned on noisy hand-object interactions or only object geometries, for both refinement and synthesis applications. We demonstrate JointDiffusion's improvements over the SOTA in both applications: it increases the contact F1 score by $5\%$ for refinement and decreases the sim. displacement by $46\%$ for synthesis. Our experiments show that JointDiffusion with CHOIR yield superior contact accuracy and physical realism compared to SOTA methods designed for specific tasks. Project page: https://theomorales.com/CHOIR
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。