用分层规划和流增强编码,解决文本生成动作的细节与全局一致性矛盾。
PlanMoGPT: Flow-Enhanced Progressive Planning for Text to Motion Synthesis
- 分层生成动作:从粗略计划逐步细化到完整序列。
- FID降低63.8%(0.380→0.141),多样性提升49.9%。
- 适合追求高精度动作生成的研究者与开发者。
大语言模型在多模态生成中取得突破,但在文本到动作生成任务中仍显著落后于非LLM方法。我们发现动作分词粒度是关键瓶颈:细粒度分词导致局部依赖,模型过度关注短期连贯性而牺牲全局语义对齐;粗粒度则损失动作细节。为此,提出PlanMoGPT框架,融合分层规划与流增强的细粒度动作分词。首先,利用LLM自回归能力,从稀疏全局计划逐级细化为完整序列。其次,流增强分词器将下采样分辨率翻倍,码本大小扩大8倍,减少离散化损失;流增强解码器恢复动作细微特征。在文本到动作基准上实验证明,该方法达到领先性能:长序列生成时FID降低63.8%(0.380→0.141),运动多样性提升49.9%,成功破解现有非LLM方法的多样性和质量权衡难题,树立新标准。
原文摘要 · Abstract (English)
Recent advances in large language models (LLMs) have enabled breakthroughs in many multimodal generation tasks, but a significant performance gap still exists in text-to-motion generation, where LLM-based methods lag far behind non-LLM methods. We identify the granularity of motion tokenization as a critical bottleneck: fine-grained tokenization induces local dependency issues, where LLMs overemphasize short-term coherence at the expense of global semantic alignment, while coarse-grained tokenization sacrifices motion details. To resolve this issue, we propose PlanMoGPT, an LLM-based framework integrating progressive planning and flow-enhanced fine-grained motion tokenization. First, our progressive planning mechanism leverages LLMs' autoregressive capabilities to hierarchically generate motion tokens by starting from sparse global plans and iteratively refining them into full sequences. Second, our flow-enhanced tokenizer doubles the downsampling resolution and expands the codebook size by eight times, minimizing detail loss during discretization, while a flow-enhanced decoder recovers motion nuances. Extensive experiments on text-to-motion benchmarks demonstrate that it achieves state-of-the-art performance, improving FID scores by 63.8% (from 0.380 to 0.141) on long-sequence generation while enhancing motion diversity by 49.9% compared to existing methods. The proposed framework successfully resolves the diversity-quality trade-off that plagues current non-LLM approaches, establishing new standards for text-to-motion generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。