LUMA通过双路径锚定提升文本到动作生成的语义对齐,减少运动伪影。
LUMA: Low-Dimension Unified Motion Alignment with Dual-Path Anchoring for Text-to-Motion Diffusion Model
- 引入轻量级MoCLIP与频域低频成分作为双路径语义锚点
- 在HumanML3D和KIT-ML上实现0.035和0.123的FID分数
- 加速收敛1.4倍,适合高保真动作生成任务
当前基于扩散模型的文本到动作生成方法虽有进展,但仍存在语义错位和运动伪影问题。分析表明,网络深层梯度衰减是关键瓶颈,导致高层特征学习不足。为此,我们提出LUMA(Low-dimension Unified Motion Alignment),采用双路径锚定机制增强语义对齐。第一路径使用无外部数据依赖的对比学习训练轻量MoCLIP模型,提供时域语义监督;第二路径从低频DCT分量中提取富含语义的频域互补信号。两条路径通过时序调制机制自适应融合,使模型在去噪过程中逐步实现从粗粒度对齐到细粒度语义优化的过渡。在HumanML3D和KIT-ML数据集上的实验结果表明,LUMA达到最先进性能,FID分别为0.035和0.123。此外,相比基线模型,收敛速度提升1.4倍,具备高效且可扩展的潜力。
原文摘要 · Abstract (English)
While current diffusion-based models, typically built on U-Net architectures, have shown promising results on the text-to-motion generation task, they still suffer from semantic misalignment and kinematic artifacts. Through analysis, we identify severe gradient attenuation in the deep layers of the network as a key bottleneck, leading to insufficient learning of high-level features. To address this issue, we propose \textbf{LUMA} (\textit{\textbf{L}ow-dimension \textbf{U}nified \textbf{M}otion \textbf{A}lignment}), a text-to-motion diffusion model that incorporates dual-path anchoring to enhance semantic alignment. The first path incorporates a lightweight MoCLIP model trained via contrastive learning without relying on external data, offering semantic supervision in the temporal domain. The second path introduces complementary alignment signals in the frequency domain, extracted from low-frequency DCT components known for their rich semantic content. These two anchors are adaptively fused through a temporal modulation mechanism, allowing the model to progressively transition from coarse alignment to fine-grained semantic refinement throughout the denoising process. Experimental results on HumanML3D and KIT-ML demonstrate that LUMA achieves state-of-the-art performance, with FID scores of 0.035 and 0.123, respectively. Furthermore, LUMA accelerates convergence by 1.4$\times$ compared to the baseline, making it an efficient and scalable solution for high-fidelity text-to-motion generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。