用连续因果隐空间实现文本驱动的实时动作生成
MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Space
- 在连续因果隐空间中构建自回归模型，避免离散化损失
- 支持长序列生成，误差积累显著减少，响应延迟更低
- 适合实时交互、多轮对话和动态动作组合场景
本文针对文本条件下的流式动作生成难题，提出一种基于扩散的自回归框架——MotionStreamer。该方法通过引入连续因果隐空间，在保持时间因果关系的同时，克服了传统扩散模型对固定长度的依赖，以及GPT类方法因离散化非因果分词导致的延迟与误差累积问题。连续隐变量有效缓解信息损失，提升长期生成稳定性。通过建立当前与历史动作隐变量间的时序因果依赖，模型可充分融合已有信息，实现高精度在线动作解码。实验表明，该方法在生成质量上优于现有方法，并支持多轮生成、长序列生成及动态动作组合等新应用。
原文摘要 · Abstract (English)
This paper addresses the challenge of text-conditioned streaming motion generation, which requires us to predict the next-step human pose based on variable-length historical motions and incoming texts. Existing methods struggle to achieve streaming motion generation, e.g., diffusion models are constrained by pre-defined motion lengths, while GPT-based methods suffer from delayed response and error accumulation problem due to discretized non-causal tokenization. To solve these problems, we propose MotionStreamer, a novel framework that incorporates a continuous causal latent space into a probabilistic autoregressive model. The continuous latents mitigate information loss caused by discretization and effectively reduce error accumulation during long-term autoregressive generation. In addition, by establishing temporal causal dependencies between current and historical motion latents, our model fully utilizes the available information to achieve accurate online motion decoding. Experiments show that our method outperforms existing approaches while offering more applications, including multi-round generation, long-term generation, and dynamic motion composition. Project Page: https://zju3dv.github.io/MotionStreamer/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。