MUGEN用连续潜变量统一理解与生成动作,零量化损失,推理更快。
MUGEN: A Unified Framework for Efficient Motion Understanding and Generation

- 用自适应长度的连续潜槽代替离散码本,统一动作与语言表示。
- 单次采样完成文本到动作生成,推理仅需K步语言模型计算。
- 在多个数据集上超越现有方法,生成质量高且理解能力更强。
将人类动作与语言相互关联是实现物理智能系统的关键,此类系统需理解、生成并交流人类行为。现有统一动作-语言框架通过共享离散动作码本连接两者,但量化限制了生成质量。最强生成器通过堆叠残差码本、长自回归推演和数十至百步去噪链提升质量,代价高昂;即使连续潜变量设计也依赖迭代扩散头获取潜空间。而这些解码机制均不支持理解任务。为此,我们提出MUGEN:一个无需代码本、仅一次采样的统一动作-语言框架。单个自适应长度的自动编码器将任意长度动作压缩为少量连续潜槽,作为系统唯一动作表示;语言模型生成这些潜槽以实现文本到动作,亦可读回潜槽实现动作理解。深度路由隐藏状态使每个潜槽从其所需变压器层级读取信息,校准头预测全潜槽联合分布,单次采样即可携带文本条件与跨潜槽变化。解码成本仅为K语言模型步骤、一次采样与一次解码器前传。MUGEN在HumanML3D上的FID指标领先语言模型基线,检索精度超过真实动作参考,在标准评估下达到最佳CIDEr与BLEU@4得分,并在SnapMoGen的所有检索与对齐指标上超越离散令牌最先进的表现。
原文摘要 · Abstract (English)
Grounding human motion in language, and language in motion, is a central step toward physical AI systems that can understand, generate, and communicate human behavior. Unified motion--language systems first coupled the two directions through a shared discrete motion codebook, but quantization limits generation quality. The strongest generators buy quality back at growing cost: stacked residual codebooks enlarge the representation; masked decoding stages, long autoregressive rollouts, and denoising chains of tens to hundreds of steps stretch inference; even the continuous-latent designs among them reach their latent only through an iterative diffusion head; and none of this decoding machinery serves understanding. We therefore propose MUGEN, a unified motion--language framework that pays neither cost: no codebook, one draw. A single adaptive-length autoencoder compresses any-length motion into a few continuous latent slots, the system's only motion representation: the language model generates them for text-to-motion and reads them back for motion understanding. Depth-routed hidden states let each slot read from the transformer depth it needs, and a calibrated head predicts a joint distribution over the full latent set, so a single draw carries the text-conditional, cross-slot variation a description permits. At a decoding cost of K language-model steps, one draw, and one decoder pass, MUGEN leads language-model baselines on FID on HumanML3D while raising retrieval precision above the real-motion reference under the standard evaluator, achieves the best CIDEr and BLEU@4 scores, and surpasses the discrete-token state of the art on every retrieval and alignment metric on SnapMoGen.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。