让CLIP理解动作动态，提升文本生成动作的准确性。
MoCLIP: Motion-Aware Fine-Tuning and Distillation of CLIP for Human Motion Generation
- 在CLIP基础上增加动作编码头，用对比学习训练动作序列。
- 在动作生成任务中，Top-1/2/3准确率均提升，FID保持竞争力。
- 适合想改进文本到动作生成效果的研究者和开发者。
人类动作生成在动画、机器人和虚拟现实等领域至关重要，需要模型能从文本描述中有效捕捉动作动态。现有方法多依赖基于文本-图像对训练的CLIP文本编码器，但其训练方式限制了对动作时序与运动学结构的理解。本文提出MoCLIP，一种在动作序列上通过对比学习和约束损失训练的细调CLIP模型，新增动作感知编码头。通过显式引入动作感知表示，MoCLIP在保持与现有CLIP框架兼容性的前提下，显著提升动作保真度，并可无缝集成至各类CLIP基方法。实验表明，MoCLIP在Top-1、Top-2和Top-3准确率上均有提升，同时维持良好FID表现，实现更优的文本到动作对齐。结果验证了其通用性与有效性，确立为增强动作生成的可靠框架。
原文摘要 · Abstract (English)
Human motion generation is essential for fields such as animation, robotics, and virtual reality, requiring models that effectively capture motion dynamics from text descriptions. Existing approaches often rely on Contrastive Language-Image Pretraining (CLIP)-based text encoders, but their training on text-image pairs constrains their ability to understand temporal and kinematic structures inherent in motion and motion generation. This work introduces MoCLIP, a fine-tuned CLIP model with an additional motion encoding head, trained on motion sequences using contrastive learning and tethering loss. By explicitly incorporating motion-aware representations, MoCLIP enhances motion fidelity while remaining compatible with existing CLIP-based pipelines and seamlessly integrating into various CLIP-based methods. Experiments demonstrate that MoCLIP improves Top-1, Top-2, and Top-3 accuracy while maintaining competitive FID, leading to improved text-to-motion alignment results. These results highlight MoCLIP's versatility and effectiveness, establishing it as a robust framework for enhancing motion generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。