用语言指导动作离散化，让生成更高效准确
Language-Guided Transformer Tokenizer for Human Motion Generation
- 用Transformer对齐语言与动作，生成高语义紧凑的离散动作码
- 在HumanML3D和Motion-X上性能超越当前最优，且用一半令牌
- 支持无语言生成，适合需要灵活控制的动作生成场景
本文聚焦动作离散化，将原始动作转化为紧凑的离散符号，该过程对高效动作生成至关重要。传统方法通过增加符号数量提升重建质量，但会加重生成模型的学习负担。为此，我们提出语言引导的离散化（LG-Tok），在编码阶段对齐自然语言与动作，生成语义丰富、紧凑的表示。该方法不仅增强编码与解码效果，还降低生成模型学习难度。现有方法多采用卷积结构，局部感受野难以支持全局语言引导。为此，我们设计基于Transformer的分词器，利用注意力机制实现语言与动作的有效对齐。此外，引入语言丢弃策略，在训练中随机移除语言条件，使解码器具备无语言引导的生成能力。在HumanML3D和Motion-X基准上，LG-Tok分别取得0.542和0.582的Top-1分数，优于当前最优方法（MARDM: 0.500/0.528），FID分别为0.057和0.088，优于0.114和0.147。LG-Tok-mini仅使用一半令牌，仍保持竞争力（Top-1: 0.521/0.588, FID: 0.085/0.071），验证了语义表示的高效性。代码与模型已公开。
原文摘要 · Abstract (English)
In this paper, we focus on motion discrete tokenization, which converts raw motion into compact discrete tokens--a process proven crucial for efficient motion generation. In this paradigm, increasing the number of tokens is a common approach to improving motion reconstruction quality, but more tokens make it more difficult for generative models to learn. To maintain high reconstruction quality while reducing generation complexity, we propose leveraging language to achieve efficient motion tokenization, which we term Language-Guided Tokenization (LG-Tok). LG-Tok aligns natural language with motion at the tokenization stage, yielding compact, high-level semantic representations. This approach not only strengthens both tokenization and detokenization but also simplifies the learning of generative models. Furthermore, existing tokenizers predominantly adopt convolutional architectures, whose local receptive fields struggle to support global language guidance. To this end, we propose a Transformer-based Tokenizer that leverages attention mechanisms to enable effective alignment between language and motion. Additionally, we design a language-drop scheme, in which language conditions are randomly removed during training, enabling the detokenizer to support language-free guidance during generation. On the HumanML3D and Motion-X generation benchmarks, LG-Tok achieves Top-1 scores of 0.542 and 0.582, outperforming state-of-the-art methods (MARDM: 0.500 and 0.528), and with FID scores of 0.057 and 0.088, respectively, versus 0.114 and 0.147. LG-Tok-mini uses only half the tokens while maintaining competitive performance (Top-1: 0.521/0.588, FID: 0.085/0.071), validating the efficiency of our semantic representations. Code and checkpoints are available at https://eanson023.github.io/LG-Tok/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。