TTS模型在Tenstorrent上实现4倍降本,音频质量无损。
Rewriting TTS Inference Economics: Lightning V2 on Tenstorrent Achieves 4x Lower Cost Than NVIDIA L40S
- 通过软硬件协同设计,支持高精度低精度混合计算。
- 相比NVIDIA L40S,同等吞吐下成本降低4倍。
- 适合追求低成本实时语音生成的工程团队。
文语转换(TTS)模型因连续波形生成和对数值微小扰动的感知敏感性,比大语言模型(LLMs)更易受数值不稳定性影响。尽管在语言模型中广泛采用如块浮点8位(BFP8)和低保真度(LoFi)计算等激进精度压缩技术,但应用于TTS系统常导致可听失真、相位不稳定和频谱畸变。本文提出生产级TTS模型Lightning V2,专为Tenstorrent硬件协同优化。通过精度感知架构设计与软硬件协同优化,实现超过95%的LoFi计算保真度和超过80%的BFP8部署,且音频质量无明显退化。利用Tenstorrent的片上网络(NoC)、分布式SRAM及确定性执行模型,减少内存移动和冗余权重加载,提升低精度推理效率。相较于NVIDIA L40S基准,Lightning V2在相同吞吐量下实现约4倍的本地加速器成本降低,同时保持生产级音频质量。结果表明,精度协同设计结合硬件感知优化,可从根本上重塑实时语音推理的经济性。
原文摘要 · Abstract (English)
Text-to-Speech (TTS) models are significantly more numerically fragile than Large Language Models (LLMs) due to their continuous waveform generation and perceptual sensitivity to small numerical perturbations. While aggressive precision reduction techniques such as BlockFloat8 (BFP8) and low-fidelity (LoFi) compute have been widely adopted in language models, applying similar strategies to TTS systems often results in audible artifacts, phase instability, and spectral distortion. In this work, we present Lightning V2, a production-grade TTS model co-optimized for Tenstorrent hardware. Through precision-aware architectural design and hardware-software co-optimization, we achieve over 95% LoFi computational fidelity and more than 80% BlockFloat8 deployment without measurable degradation in audio quality. Leveraging Tenstorrent's Network-on-Chip (NoC), distributed SRAM, and deterministic execution model, we reduce memory movement and redundant weight fetches, enabling efficient low-precision inference. Compared to an NVIDIA L40S baseline, Lightning V2 achieves approximately 4x lower on-prem accelerator cost at equivalent throughput, while maintaining production audio fidelity. Our results demonstrate that precision co-design, combined with hardware-aware optimization, can fundamentally reshape the economics of real-time speech inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。