让动作生成先思考再执行,提升语义对齐与物理合理性
Think Before You Move: Latent Motion Reasoning for Text-to-Motion Generation
- 分两阶段生成:先用潜空间规划动作骨架,再生成具体帧
- 引入双粒度编码器,分离语义规划与高频运动细节
- 适用于多种模型,显著提升动作自然度与语言匹配度
当前主流的文本到动作(T2M)生成方法将语言直接映射为连续姿态,存在语义-运动阻抗不匹配问题。本文提出潜空间动作推理(LMR),借鉴认知科学中的分层运动控制,将生成过程重构为‘思考-执行’两阶段。核心是双粒度分词器,将动作分解为语义丰富的规划潜空间和高频率执行潜空间。通过强制模型先自回归推理粗略轨迹再生成帧,有效弥合语言与物理间的不可言说鸿沟。在T2M-GPT(离散)和MotionStreamer(连续)两个基线模型上验证,显著提升语义对齐与物理合理性,证明最优动作规划空间并非自然语言,而是学习得到的运动对齐概念空间。
原文摘要 · Abstract (English)
Current state-of-the-art paradigms predominantly treat Text-to-Motion (T2M) generation as a direct translation problem, mapping symbolic language directly to continuous poses. While effective for simple actions, this System 1 approach faces a fundamental theoretical bottleneck we identify as the Semantic-Kinematic Impedance Mismatch: the inherent difficulty of grounding semantically dense, discrete linguistic intent into kinematically dense, high-frequency motion data in a single shot. In this paper, we argue that the solution lies in an architectural shift towards Latent System 2 Reasoning. Drawing inspiration from Hierarchical Motor Control in cognitive science, we propose Latent Motion Reasoning (LMR) that reformulates generation as a two-stage Think-then-Act decision process. Central to LMR is a novel Dual-Granularity Tokenizer that disentangles motion into two distinct manifolds: a compressed, semantically rich Reasoning Latent for planning global topology, and a high-frequency Execution Latent for preserving physical fidelity. By forcing the model to autoregressively reason (plan the coarse trajectory) before it moves (instantiates the frames), we effectively bridge the ineffability gap between language and physics. We demonstrate LMR's versatility by implementing it for two representative baselines: T2M-GPT (discrete) and MotionStreamer (continuous). Extensive experiments show that LMR yields non-trivial improvements in both semantic alignment and physical plausibility, validating that the optimal substrate for motion planning is not natural language, but a learned, motion-aligned concept space. Codes and demos can be found in \hyperlink{https://chenhaoqcdyq.github.io/LMR/}{https://chenhaoqcdyq.github.io/LMR/}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。