用强化学习统一优化文本到动作生成,提升真实感与语义一致性。
MotionRFT: Unified Reinforcement Fine-Tuning for Text-to-Motion Generation
- 构建多维度奖励模型MotionReward,统一不同动作表示的语义空间。
- 提出EasyTune方法,分步优化降低内存占用,峰值仅22.10 GB。
- 在多个数据集上显著提升生成质量,最高减少FID 22.9%。
文本到动作生成已借助扩散模型和流模型取得进展,但监督预训练难以对齐语义一致性、真实感及人类偏好等高层目标。现有后训练方法存在三大局限:(1)仅针对特定动作表示(如关节);(2)优化单一目标,可能损害其他因素;(3)计算开销大、依赖数据、优化粒度粗。本文提出统一强化微调框架MotionRFT,包含异构表示多维奖励模型MotionReward与高效细粒度微调方法EasyTune。MotionReward将异构动作映射至由文本锚定的共享语义空间,实现多维奖励学习;自反思偏好学习进一步增强语义理解而无需额外标注。为提升效率,发现去噪步骤间的递归梯度依赖是关键瓶颈,提出EasyTune方法,逐步优化而非全轨迹优化,实现密集、细粒度且内存高效更新。大量实验验证有效性:在MLD模型上达FID 0.132,峰值内存22.10 GB,较DRaFT节省最多15.22 GB;在关节基ACMDM上FID降低22.9%;在旋转基HY Motion上实现12.6% R-Precision提升与23.3% FID改善。项目页面及代码已公开。
原文摘要 · Abstract (English)
Text-to-motion generation has advanced with diffusion- and flow-based generative models, yet supervised pretraining remains insufficient to align models with high-level objectives such as semantic consistency, realism, and human preference. Existing post-training methods have key limitations: they (1) target a specific motion representation, such as joints, (2) optimize a particular aspect, such as text-motion alignment, and may compromise other factors; and (3) incur substantial computational overhead, data dependence, and coarse-grained optimization. We present a reinforcement fine-tuning framework that comprises a heterogeneous-representation, multi-dimensional reward model, MotionReward, and an efficient, fine-grained fine-tuning method, EasyTune. To obtain a unified semantics representation, MotionReward maps heterogeneous motions into a shared semantic space anchored by text, enabling multidimensional reward learning; Self-refinement Preference Learning further enhances semantics without additional annotations. For efficient and effective fine-tuning, we identify the recursive gradient dependence across denoising steps as the key bottleneck, and propose EasyTune, which optimizes step-wise rather than over the full trajectory, yielding dense, fine-grained, and memory-efficient updates. Extensive experiments validate the effectiveness of our framework, achieving FID 0.132 at 22.10 GB peak memory for MLD model and saving up to 15.22 GB over DRaFT. It reduces FID by 22.9% on joint-based ACMDM, and achieves a 12.6% R-Precision gain and 23.3% FID improvement on rotation-based HY Motion. Our project page with code is publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。