用检索增强生成技术，让文字转动作更自然、多样且物理合理。
ReMoMask: Retrieval-Augmented Masked Motion Generation
- 双向动量模型提升跨模态检索精度，解决样本干扰问题。
- 融合生物力学约束，消除动作不同步的伪影。
- 结合无条件生成，显著提升动作多样性与泛化能力。
文本到动作（T2M）生成旨在从自然语言描述中合成真实且语义对齐的人体动作序列。现有方法面临双重挑战：生成模型（如扩散模型）存在多样性不足、误差累积和物理不合理等问题；而检索增强生成（RAG）方法则存在扩散惯性、部分模式崩溃和异步伪影。为此，我们提出ReMoMask，一个统一框架，包含三项创新：1）双向动量文本-动作模型通过动量队列解耦负样本规模与批次大小，显著提升跨模态检索精度；2）语义时空注意力机制在局部融合阶段施加生物力学约束，消除异步伪影；3）RAG-无分类器引导引入少量无条件生成，增强泛化能力。基于MoMask的RVQ-VAE架构，ReMoMask可在极少数步骤内高效生成时序连贯的动作。在标准基准测试上，其表现达到当前最优，相比之前SOTA方法RAG-T2M，在HumanML3D和KIT-ML数据集上分别实现FID分数降低3.88%和10.97%。代码与官网见：https://github.com/AIGeeksGroup/ReMoMask，https://aigeeksgroup.github.io/ReMoMask。
原文摘要 · Abstract (English)
Text-to-Motion (T2M) generation aims to synthesize realistic and semantically aligned human motion sequences from natural language descriptions. However, current approaches face dual challenges: Generative models (e.g., diffusion models) suffer from limited diversity, error accumulation, and physical implausibility, while Retrieval-Augmented Generation (RAG) methods exhibit diffusion inertia, partial-mode collapse, and asynchronous artifacts. To address these limitations, we propose ReMoMask, a unified framework integrating three key innovations: 1) A Bidirectional Momentum Text-Motion Model decouples negative sample scale from batch size via momentum queues, substantially improving cross-modal retrieval precision; 2) A Semantic Spatio-temporal Attention mechanism enforces biomechanical constraints during part-level fusion to eliminate asynchronous artifacts; 3) RAG-Classier-Free Guidance incorporates minor unconditional generation to enhance generalization. Built upon MoMask's RVQ-VAE, ReMoMask efficiently generates temporally coherent motions in minimal steps. Extensive experiments on standard benchmarks demonstrate the state-of-the-art performance of ReMoMask, achieving a 3.88% and 10.97% improvement in FID scores on HumanML3D and KIT-ML, respectively, compared to the previous SOTA method RAG-T2M. Code: https://github.com/AIGeeksGroup/ReMoMask. Website: https://aigeeksgroup.github.io/ReMoMask.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。