用改进的扩散强迫框架实现文本驱动的实时动作生成
FloodDiffusion: Tailored Diffusion Forcing for Streaming Motion Generation
- 采用双向注意力与低三角时间调度的扩散强迫机制
- 在HumanML3D上实现0.057的FID，达到当前最优
- 适合需要实时生成流畅动作的应用场景
我们提出FloodDiffusion，一种用于文本驱动流式人体动作生成的新框架。给定随时间变化的文本提示，FloodDiffusion可生成与文本对齐、无缝衔接的动作序列，并具备实时延迟。不同于依赖分块或自回归扩散头的方法，我们采用扩散强迫框架来建模这一时变控制事件下的时间序列生成任务。发现直接套用视频模型的原始扩散强迫无法建模真实动作分布。我们证明为保证输出分布建模，需进行三项改进：(i) 使用双向注意力而非单向注意力；(ii) 采用下三角时间调度而非随机调度；(iii) 以连续时变方式引入文本条件。通过这些改进，首次实现基于扩散强迫框架在流式动作生成任务上的最先进性能，在HumanML3D基准上取得0.057的FID。代码、模型与权重已公开。
原文摘要 · Abstract (English)
We present FloodDiffusion, a new framework for text-driven, streaming human motion generation. Given time-varying text prompts, FloodDiffusion generates text-aligned, seamless motion sequences with real-time latency. Unlike existing methods that rely on chunk-by-chunk or auto-regressive model with diffusion head, we adopt a diffusion forcing framework to model this time-series generation task under time-varying control events. We find that a straightforward implementation of vanilla diffusion forcing (as proposed for video models) fails to model real motion distributions. We demonstrate that to guarantee modeling the output distribution, the vanilla diffusion forcing must be tailored to: (i) train with a bi-directional attention instead of casual attention; (ii) implement a lower triangular time scheduler instead of a random one; (iii) utilize a continues time-varying way to introduce text conditioning. With these improvements, we demonstrate in the first time that the diffusion forcing-based framework achieves state-of-the-art performance on the streaming motion generation task, reaching an FID of 0.057 on the HumanML3D benchmark. Models, code, and weights are available. https://shandaai.github.io/FloodDiffusion/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。