一帧生成说话头像，34毫秒延迟，长视频稳定不跑偏。
AvatarForcing: One-Step Streaming Talking Avatars via Local-Future Sliding-Window Denoising
- 每步只处理局部未来窗口，用异构噪声去噪，保持计算恒定。
- 在400个视频的长序列测试中，34毫秒/帧，1.3B模型实时生成。
- 双锚点机制防止长期生成失真，适合直播、虚拟主播场景。
实时说话头像生成需低延迟与分钟级时间稳定性。自回归强制方法虽支持流式推理，但存在暴露偏差，导致误差累积且不可逆；全序列扩散变换器可缓解漂移，却难以实现实时长序列合成。本文提出AvatarForcing，一种一步流式扩散框架，通过固定局部未来窗口的异构噪声去噪，每步输出一个干净块，且每步开销恒定。为稳定无界流，引入双锚点时序强制：风格锚点重索引RoPE以保持相对位置固定，并应用锚点音频零填充；时间锚点复用最近生成的干净块以保证平滑过渡。通过两阶段流式蒸馏实现实时推理，包含离线ODE回填和分布匹配。在标准基准与新构建的400视频长序列基准上，使用1.3B参数学生模型，实现34毫秒/帧的实时流式生成，视觉质量与口型同步表现优异。
原文摘要 · Abstract (English)
Real-time talking avatar generation requires low latency and minute-level temporal stability. Autoregressive (AR) forcing enables streaming inference but suffers from exposure bias, which causes errors to accumulate and become irreversible over long rollouts. In contrast, full-sequence diffusion transformers mitigate drift but remain computationally prohibitive for real-time long-form synthesis. We present AvatarForcing, a one-step streaming diffusion framework that denoises a fixed local-future window with heterogeneous noise levels and emits one clean block per step under constant per-step cost. To stabilize unbounded streams, the method introduces dual-anchor temporal forcing: a style anchor that re-indexes RoPE to maintain a fixed relative position with respect to the active window and applies anchor-audio zero-padding, and a temporal anchor that reuses recently emitted clean blocks to ensure smooth transitions. Real-time one-step inference is enabled by two-stage streaming distillation with offline ODE backfill and distribution matching. Experiments on standard benchmarks and a new 400-video long-form benchmark show strong visual quality and lip synchronization at 34 ms/frame using a 1.3B-parameter student model for realtime streaming. Our page is available at: https://cuiliyuan121.github.io/AvatarForcing/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。