提出可解耦语音表示的离散神经编解码器，兼顾重建质量与语音转换效果。
Exploring Disentangled Neural Speech Codecs from Self-Supervised Representations
- 基于自监督特征与k-means量化实现语音内容与声纹的结构化解耦
- 重建性能媲美传统编解码器，语音转换效果达到主流方法水平
- 适合需要灵活控制语音特征的研究者，如语音合成与风格迁移
神经音频编解码器（NACs）利用神经网络生成紧凑的音频表示，受到广泛关注，尤其量化编解码器因其与大语言模型兼容性而备受青睐。然而，与文本不同，语音不仅包含语言内容，还包含丰富的副语言特征。将这些元素以纠缠方式编码可能降低灵活性。例如，语音转换（VC）需在保持原语言内容的同时转换说话人特征，这要求表示具有解耦性。受利用k-means量化自监督特征实现语音解耦的语音转换方法启发，我们提出一种具备结构化解耦能力的离散神经编解码器。实验表明，该方法在重建性能上与不显式解耦的传统NACs相当，同时在语音转换效果上达到传统技术水平。
原文摘要 · Abstract (English)
Neural audio codecs (NACs), which use neural networks to generate compact audio representations, have garnered interest for their applicability to many downstream tasks -- especially quantized codecs due to their compatibility with large language models. However, unlike text, speech conveys not only linguistic content but also rich paralinguistic features. Encoding these elements in an entangled fashion may be suboptimal, as it limits flexibility. For instance, voice conversion (VC) aims to convert speaker characteristics while preserving the original linguistic content, which requires a disentangled representation. Inspired by VC methods utilizing $k$-means quantization with self-supervised features to disentangle phonetic information, we develop a discrete NAC capable of structured disentanglement. Experimental evaluations show that our approach achieves reconstruction performance on par with conventional NACs that do not explicitly perform disentanglement, while also matching the effectiveness of conventional VC techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。