纯Transformer架构实现高效低延迟音频编码，性能超越传统卷积模型。
TS3-Codec: Transformer-Based Simple Streaming Single Codec
- 全用Transformer层替代卷积，简化结构并减少调参复杂度。
- 仅需12%计算量与77%码率，仍达领先卷积模型的压缩性能。
- 适合对实时性与计算资源敏感的语音/音频应用开发。
神经音频编解码器（NACs）作为音频压缩和语音语言模型音频表征的关键技术备受关注。尽管主流NAC模型以卷积为基础，但纯Transformer、无卷积架构的NAC性能尚未被充分探索。本文提出TS3-Codec，一种基于Transformer的简单流式单编解码器，仅由堆叠的Transformer层与少量线性层构成，完全消除需要精细调参和高计算量的卷积层，从而提升简洁性与表达能力。在流式设置下，该模型性能可媲美甚至优于当前最先进的卷积基编解码器，同时仅需12%的计算量和77%的码率。此外，在相似计算资源下，其性能显著优于卷积基编解码器。
原文摘要 · Abstract (English)
Neural audio codecs (NACs) have garnered significant attention as key technologies for audio compression as well as audio representation for speech language models. While mainstream NAC models are predominantly convolution-based, the performance of NACs with a purely transformer-based, and convolution-free architecture remains unexplored. This paper introduces TS3-Codec, a Transformer-Based Simple Streaming Single Codec. TS3-Codec consists of only a stack of transformer layers with a few linear layers, offering greater simplicity and expressiveness by fully eliminating convolution layers that require careful hyperparameter tuning and large computations. Under the streaming setup, the proposed TS3-Codec achieves comparable or superior performance compared to the codec with state-of-the-art convolution-based architecture while requiring only 12% of the computation and 77% of bitrate. Furthermore, it significantly outperforms the convolution-based codec when using similar computational resources.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。