用自监督方法修复文本生成动作的物理不合理问题。
A Self-Supervised Approach on Motion Calibration for Enhancing Physical Plausibility in Text-to-Motion
- 输入扭曲动作和文本描述,自动学习修正物理错误。
- 在T2M上降低42.74%的FID分数,提升语义一致性。
- 适合需要真实动作效果的动画、游戏开发者使用。
从文本描述生成语义对齐的人体动作已取得快速进展,但确保动作在语义与物理真实性上均合理仍具挑战。本文提出畸变感知动作校准器(DMC),一个后处理模块,在不破坏原始文本语义的前提下,修复如脚部漂浮等物理不合理的动作。不同于复杂的物理建模,DMC采用自监督、数据驱动的方式,当输入人为扭曲的动作与原始文本描述时,学习生成更符合物理规律的动作。我们在多种文本到动作生成模型上评估DMC,结果表明其在T2M上降低42.74%的FID分数,在T2M-GPT上降低13.20%,同时达到最高R-Precision。应用于高质量模型MoMask时,可使动作穿透减少33.0%,并更接近真实参考动作。这些结果表明,DMC可作为通用后处理框架,有效融合文本语义与物理合理性。
原文摘要 · Abstract (English)
Generating semantically aligned human motion from textual descriptions has made rapid progress, but ensuring both semantic and physical realism in motion remains a challenge. In this paper, we introduce the Distortion-aware Motion Calibrator (DMC), a post-hoc module that refines physically implausible motions (e.g., foot floating) while preserving semantic consistency with the original textual description. Rather than relying on complex physical modeling, we propose a self-supervised and data-driven approach, whereby DMC learns to obtain physically plausible motions when an intentionally distorted motion and the original textual descriptions are given as inputs. We evaluate DMC as a post-hoc module to improve motions obtained from various text-to-motion generation models and demonstrate its effectiveness in improving physical plausibility while enhancing semantic consistency. The experimental results show that DMC reduces FID score by 42.74% on T2M and 13.20% on T2M-GPT, while also achieving the highest R-Precision. When applied to high-quality models like MoMask, DMC improves the physical plausibility of motions by reducing penetration by 33.0% as well as adjusting floating artifacts closer to the ground-truth reference. These results highlight that DMC can serve as a promising post-hoc motion refinement framework for any kind of text-to-motion models by incorporating textual semantics and physical plausibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。