无需训练即可加速扩散模型语音合成，降低计算量75.4%。
DiTReducio: A Training-Free Acceleration for DiT-Based TTS via Progressive Calibration
- 通过渐进校准消除推理时冗余计算，不依赖训练。
- 在F5-TTS和MegaTTS 3上实现75.4%的浮点运算减少。
- 支持质量与效率灵活调节，适合资源受限场景使用。
尽管扩散变换器（DiT）推动了非自回归语音合成的发展，但其高计算开销仍是瓶颈。现有加速方法多依赖蒸馏技术减少采样步数，仍需高昂训练成本。本文提出DiTReducio，一种无需训练的加速框架，通过渐进校准压缩DiT语音合成模型的计算量。设计了时间跳过与分支跳过两种压缩方法，以消除推理中的冗余计算。基于DiT层中发现的两类注意力模式，提出模式引导策略，选择性应用压缩方法。通过可调压缩阈值，灵活平衡生成质量与计算效率。在F5-TTS和MegaTTS 3上的实验表明，DiTReducio实现75.4%的浮点运算量（FLOPs）降低，实时因子（RTF）提升37.1%，同时保持生成质量。
原文摘要 · Abstract (English)
While Diffusion Transformers (DiT) have advanced non-autoregressive (NAR) speech synthesis, their high computational demands remain an limitation. Existing DiT-based text-to-speech (TTS) model acceleration approaches mainly focus on reducing sampling steps through distillation techniques, yet they remain constrained by training costs. We introduce DiTReducio, a training-free acceleration framework that compresses computations in DiT-based TTS models via progressive calibration. We propose two compression methods, Temporal Skipping and Branch Skipping, to eliminate redundant computations during inference. Moreover, based on two characteristic attention patterns identified within DiT layers, we devise a pattern-guided strategy to selectively apply the compression methods. Our method allows flexible modulation between generation quality and computational efficiency through adjustable compression thresholds. Experimental evaluations conducted on F5-TTS and MegaTTS 3 demonstrate that DiTReducio achieves a 75.4% reduction in FLOPs and improves the Real-Time Factor (RTF) by 37.1%, while preserving generation quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。