140亿参数模型实现高速高保真音频驱动虚拟人视频生成
AptAvatar: Fast and Vivid Long-Form Audio-Driven Video Generation for Production-Ready Avatars
- 采用端点锚定分布蒸馏提升两步生成效率
- 仅2次神经微分方程求解即生成720p长视频
- 适合影视级虚拟人内容快速生产
生产级音频驱动虚拟人生成需兼顾高效推理与高保真度、强动作表现力。现有加速方法常通过限制架构（如因果注意力、短时序范围）或降低模型容量与分辨率牺牲质量。为避免此类妥协，我们提出AptAvatar——一个140亿参数的长时序音频驱动虚拟人生成框架，实现快速且富有表现力的推理。针对极简两步生成的挑战，引入端点锚定分布蒸馏，通过在冻结的4步桥接生成器上训练专用锚点评分器，提供可实现的终点级锚点，指导两步学生模型演化。为提升长时序一致性，提出自生成历史回放机制，在分块训练中复用早期生成检查点的缓存输出作为历史条件，近似推理时对自生成历史的依赖，无需高昂的在线滚动，缓解累积历史误差导致的质量下降。大量实验表明，AptAvatar仅用2次神经微分方程求解（2 NFEs），即可生成720p长视频，速度提升60倍，同时保持视觉保真度和长时序身份一致性。代码已开源。
原文摘要 · Abstract (English)
Production-ready audio-driven avatar generation requires efficient inference without sacrificing fidelity or motion expressiveness. However, existing acceleration methods often compromise quality through restrictive architectural choices, such as causal attention and short temporal horizons, or by reducing model capacity and resolution. Without such compromises, we propose AptAvatar, a 14B-parameter long-form audio-driven avatar generation framework that delivers fast and expressive inference. For efficiency in production-level applications, AptAvatar addresses the extreme two-step generation challenge. To bridge the gap between the multi-step teacher model and the two-step student model, we introduce Endpoint-Anchored Distribution Distillation. It augments vanilla distribution matching with a dedicated Anchor Score Estimator trained on the trajectory-endpoint distribution defined from a frozen pretrained 4-step bridge generator. This provides an attainable endpoint-level anchor for the evolving two-step student. To improve long-horizon consistency, we further introduce Self-Generated History Replay, which reuses cached outputs from earlier generator checkpoints as history conditions during chunk-wise training. This approximates inference-time conditioning on self-generated histories without costly online rollouts, mitigating quality degradation from accumulated history errors. Extensive experiments demonstrate that AptAvatar generates vivid 720p long-form avatar videos with only 2 NFEs, achieving a 60x speedup while preserving visual fidelity and long-horizon identity. Code is available at https://github.com/TaoLiveAIGC/AptAvatar
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。