用智能代理自动优化视频生成速度,效果比手动调优更好。
Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation

- 用多个智能代理并行探索缓存、稀疏注意力等五种加速技术的组合。
- 在三种不同模型上实现2倍以上加速,质量损失极小(VBench接近原版)。
- 适合需要快速部署视频生成系统的研究者与工程师使用。
现代视频扩散模型通过规模扩展提升了生成质量,但推理成本也随之上升。尽管已有多种加速方法,核心挑战在于最优策略高度依赖具体实例:同一套加速方案在不同模型、硬件或配置下表现差异显著。模型在结构、数值敏感度和注意力分布上存在差异,推理设置在空间/时间分辨率与视频时长上不同,硬件平台在内存层次、数值格式支持和核函数吞吐量方面也各不相同。这些因素构成巨大调优空间,人工性能工程代价高昂。本文提出Sol Video Inference Engine,一个基于智能体的原生、免训练加速框架,将缓存、稀疏注意力、标记剪枝、量化和内核融合五种通用技术整合为可实例化优化的加速栈。针对特定模型、硬件平台与服务配置,平行技能代理优化各技术实现,集成代理将其组合为全局加速方案,人类验证者评估生成质量。我们在三种不同规模与架构的视频模型上验证:64B Cosmos3-Super、22B LTX-2.3 和 2B SANA-Video。仅需少量人工干预,该全栈方案实现了超过2倍的端到端加速,同时保持近似无损的VBench质量,证明了该智能体框架在视频扩散模型加速中的有效性。
原文摘要 · Abstract (English)
Modern video diffusion models achieve higher generation quality through scaling, but this also increases inference cost. Although many acceleration methods have been proposed, a central challenge is that the most effective acceleration strategy is highly instance-specific: a recipe that works well for one combination of model, hardware, and inference configuration often does not transfer to another. Different models vary in architecture, numerical sensitivity, and attention concentration patterns. Inference settings differ in spatial and temporal resolution and video duration, while hardware platforms differ in memory hierarchy, supported numerical formats, and kernel throughput. These factors create a large tuning space, making manual performance engineering costly. We present Sol Video Inference Engine, an agentic, native, training-free acceleration framework for video diffusion models. It organizes five broadly applicable techniques, cache, sparse attention, token pruning, quantization, and kernel fusion, into an agentic acceleration stack for instance-specific optimization. For a concrete deployment target defined by a model, hardware platform, and serving configuration, parallel skill agents optimize the implementation of each technique, an agent integrator composes them into a global acceleration stack, and a human validator provides feedback on generation quality. We instantiate this workflow on three video models with different sizes and architectures: 64B Cosmos3-Super, 22B LTX-2.3, and 2B SANA-Video. With little human effort, the full stack achieves more than 2x end-to-end acceleration while maintaining near-lossless VBench quality, demonstrating the effectiveness of the agent framework for video diffusion acceleration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。