让AI生成从未见过物体的人体交互动画,逼真且连贯。
GenHOI: Generalizing Text-driven 4D Human-Object Interaction Synthesis for Unseen Objects
- 分两阶段:先重建稀疏关键帧,再用接触感知扩散模型补全时间序列
- 在OMOMO和3D-FUTURE数据集上达当前最佳,能处理未见物体
- 引入接触感知编码器与注意力机制,提升动作自然度
尽管扩散模型和大规模动作数据集推动了文本驱动的人体动作生成,但将其扩展到4D人体-物体交互(HOI)仍面临挑战,主要源于大规模4D HOI数据集稀缺。本文提出GenHOI,一种两阶段框架,旨在实现两个目标:1)对未见物体的泛化能力;2)高保真4D HOI序列生成。第一阶段采用Object-AnchorNet,仅从3D HOI数据集学习,重建未见物体的稀疏3D HOI关键帧,减少对大规模4D HOI数据的依赖。第二阶段引入接触感知扩散模型(ContactDM),将稀疏关键帧无缝插值为时间连续的4D HOI序列。为提升生成质量,提出新型接触感知编码器以提取人物接触模式,并设计接触感知HOI注意力机制,有效融合接触信号至扩散模型中。实验表明,该方法在公开可用的OMOMO和3D-FUTURE数据集上达到当前最优性能,展现出强泛化能力与高保真4D HOI生成效果。
原文摘要 · Abstract (English)
While diffusion models and large-scale motion datasets have advanced text-driven human motion synthesis, extending these advances to 4D human-object interaction (HOI) remains challenging, mainly due to the limited availability of large-scale 4D HOI datasets. In our study, we introduce GenHOI, a novel two-stage framework aimed at achieving two key objectives: 1) generalization to unseen objects and 2) the synthesis of high-fidelity 4D HOI sequences. In the initial stage of our framework, we employ an Object-AnchorNet to reconstruct sparse 3D HOI keyframes for unseen objects, learning solely from 3D HOI datasets, thereby mitigating the dependence on large-scale 4D HOI datasets. Subsequently, we introduce a Contact-Aware Diffusion Model (ContactDM) in the second stage to seamlessly interpolate sparse 3D HOI keyframes into densely temporally coherent 4D HOI sequences. To enhance the quality of generated 4D HOI sequences, we propose a novel Contact-Aware Encoder within ContactDM to extract human-object contact patterns and a novel Contact-Aware HOI Attention to effectively integrate the contact signals into diffusion models. Experimental results show that we achieve state-of-the-art results on the publicly available OMOMO and 3D-FUTURE datasets, demonstrating strong generalization abilities to unseen objects, while enabling high-fidelity 4D HOI generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。