arXiv:2412.11193cs.CV2024-12AAAI被引 33

轻量级文本到动作生成模型,参数少10倍,速度更快,效果更优。

Light-T2M: A Lightweight and Fast Model for Text-to-motion Generation

  • 引入局部信息建模与伪双向扫描,提升运动细节表现
  • 参数仅448万(原模型的10%),推理速度提升16%
  • 适合低资源部署,兼顾生成质量与效率

尽管文本到动作(T2M)生成在多个应用中具有重要作用,但现有方法参数量大、推理速度慢,导致使用成本高。为此,本文提出轻量级模型Light-T2M以降低使用成本。首先,重新审视人体运动内在特性,强调局部信息建模的重要性,提出轻量级局部信息建模模块;其次,将Mamba引入T2M任务,减少参数量与显存占用,并设计新型伪双向扫描机制,在不增加参数的前提下实现双向信息捕获;此外,提出自适应文本信息注入器,更高效地融合文本信息。集成上述设计后,Light-T2M模型参数仅为4.48M(MoMask为44.85M),推理速度达0.152秒(对比0.180秒),在HumanML3D数据集上FID降至0.040(原为0.045),KIT-ML数据集上降至0.161(原为0.228)。代码已开源。

原文摘要 · Abstract (English)

Despite the significant role text-to-motion (T2M) generation plays across various applications, current methods involve a large number of parameters and suffer from slow inference speeds, leading to high usage costs. To address this, we aim to design a lightweight model to reduce usage costs. First, unlike existing works that focus solely on global information modeling, we recognize the importance of local information modeling in the T2M task by reconsidering the intrinsic properties of human motion, leading us to propose a lightweight Local Information Modeling Module. Second, we introduce Mamba to the T2M task, reducing the number of parameters and GPU memory demands, and we have designed a novel Pseudo-bidirectional Scan to replicate the effects of a bidirectional scan without increasing parameter count. Moreover, we propose a novel Adaptive Textual Information Injector that more effectively integrates textual information into the motion during generation. By integrating the aforementioned designs, we propose a lightweight and fast model named Light-T2M. Compared to the state-of-the-art method, MoMask, our Light-T2M model features just 10\% of the parameters (4.48M vs 44.85M) and achieves a 16\% faster inference time (0.152s vs 0.180s), while surpassing MoMask with an FID of \textbf{0.040} (vs. 0.045) on HumanML3D dataset and 0.161 (vs. 0.228) on KIT-ML dataset. The code is available at https://github.com/qinghuannn/light-t2m.

文本生成动作轻量化模型高效生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。