通过分步建模实现双手操作物体的精准运动生成与接触优化。
DuetHOI: Language-Guided Bimanual Hand--Object Motion Generation with Articulation Planning and Contact Refinement
- 分三阶段生成:先预测接触意图,再规划动作轨迹,最后优化手部姿态。
- 在双臂操作刚性与可动物体上,接触一致性指标优于基线方法。
- 适合需要精细双手协同操作的机器人仿真与动作生成场景。
双臂操作物体的生成需捕捉物体形变演化、双手协调关系及局部手物接触。现有方法将手与物体运动编码为高维统一序列,难以显式建模不同尺度的操作进展、双臂相对运动和局部手姿。本文提出DuetHOI,一种结构化框架:首先通过ContactVAE从语言描述和物体几何预测表面接触意图;在此空间意图基础上,ArtPlanner提供轨迹级形变参考;DualFormer利用物体状态、以物为中心的手部位置及ManiVAE学习的紧凑手姿标记生成全局交互。生成后,ProxiRefine固定物体轨迹,基于当前手物接近度残差修正双臂,提升局部表面贴合度。在可动与刚性物体、单双臂设置下的实验表明,DuetHOI在多数接触与手物一致性指标上超越三个适配基线,尤其在双臂可动交互生成中表现突出。
原文摘要 · Abstract (English)
Bimanual articulated-object interaction generation requires a model to capture the evolution of object articulation, coordination between the two hands, and fine-grained hand--object contact. Existing methods typically encode object and hand motion as a unified high-dimensional sequence, making it difficult to explicitly accommodate the different scales of manipulation progress, relative bimanual motion, and local hand pose. We propose \textbf{DuetHOI}, a structured framework for bimanual articulated-object interaction generation. DuetHOI first uses ContactVAE to predict contact intent on the object surface from language and object geometry. Conditioned on this spatial intent, ArtPlanner provides a trajectory-level articulation reference, and DualFormer generates the global interaction using object states, object-centered hand positions, and compact hand-pose tokens learned by ManiVAE. After global generation, ProxiRefine fixes the object trajectory and residually corrects the two hands using current hand--object proximity, improving local surface alignment. Experiments across articulated and rigid objects and single- and bimanual settings show that DuetHOI outperforms three adapted baselines on most contact and hand--object consistency metrics, with particularly strong results in bimanual articulated interaction generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。