通过缓存语义相似音频，加速文本转音频扩散模型生成。
SoundWeaver: Semantic Warm-Starting for Text-to-Audio Diffusion Serving
- 用语义和时长感知检索缓存音频并对齐，作为生成起点。
- 动态跳过部分扩散步数，实现1.8–3.0倍延迟降低。
- 无需训练、兼容任意模型，适合高并发音频生成场景。
文本到音频的扩散模型虽能生成高保真音频，但需数十次函数求值（NFE），导致延迟达数秒且吞吐量受限。本文提出SoundWeaver，首个无需训练、模型无关的推理加速系统，通过从语义相似的缓存音频中热启动来加速生成。该系统包含三个组件：参考选择器（Reference Selector）利用语义与持续时间感知门控检索并时间对齐缓存候选；跳过门控器（Skip Gater）动态决定跳过的NFE比例；轻量级缓存管理器（Cache Manager）通过质量感知淘汰与优化维持缓存效用。在真实音频数据上，仅使用约1000条缓存项，即实现1.8–3.0倍延迟降低，同时保持或提升听觉质量。
原文摘要 · Abstract (English)
Text-to-audio diffusion models produce high-fidelity audio but require tens of function evaluations (NFEs), incurring multi-second latency and limited throughput. We present SoundWeaver, the first training-free, model-agnostic serving system that accelerates text-to-audio diffusion by warm-starting from semantically similar cached audio. SoundWeaver introduces three components: a Reference Selector that retrieves and temporally aligns cached candidates via semantic and duration-aware gating; a Skip Gater that dynamically determines the percentage of NFEs to skip; and a lightweight Cache Manager that maintains cache utility through quality-aware eviction and refinement. On real-world audio traces, SoundWeaver achieves 1.8--3.0$ \times $ latency reduction with a cache of only ${\sim}$1K entries while preserving or improving perceptual quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。