一种单阶段训练的语音编解码器，可低比特率编码并实现零样本变声。
SDP-Codec: A Speaker-Decoupled Speech Codec with Pitch Injection for Low-Bitrate Coding and Zero-Shot Voice Conversion
- 通过注入归一化基频，分离说话人特征与语音内容
- 在16kHz和24kHz下均实现良好重建与强变声效果
- 仅用单阶段训练，显著降低说话人信息泄露
说话人解耦语音编解码器可通过分离全局说话人属性与局部内容及语调来降低码率，并支持语音转换。现有方法存在权衡：显式抑制说话人泄漏的方法常依赖多阶段或辅助训练，而简单设计则可能在局部令牌中残留说话人信息。本文提出SDP-Codec，一种基于单阶段优化的说话人解耦、基频注入编解码器。该模型从预训练自监督编码器的连续预量化特征中提取局部令牌，并通过全局条件去归一化与软标签基频重建目标，注入归一化基频（F0）。在16 kHz和24 kHz设置下，SDP-Codec在码率相当的情况下实现了竞争力的语音重建性能与强零样本语音转换能力，且相比其他系统表现出最低的说话人探测准确率，表明说话人泄漏显著减少。
原文摘要 · Abstract (English)
Speaker-decoupled speech codecs can reduce bitrate by separating global speaker attributes from local content and prosody, while supporting voice conversion. Existing speaker-decoupled codecs face a trade-off: methods that explicitly suppress speaker leakage often rely on multi-stage or auxiliary training, whereas simpler designs can leave residual speaker information in local tokens. We propose SDP-Codec, a speaker-decoupled, pitch-injected codec trained with a single-stage optimization pipeline. SDP-Codec derives local tokens from continuous pre-quantization features of a pretrained self-supervised encoder and injects normalized F0 via a pitch encoder-decoder with global-conditioned denormalization and soft-label pitch reconstruction objective. Across 16 kHz and 24 kHz settings, SDP-Codec achieves competitive reconstruction and strong zero-shot voice conversion at comparable bitrates, with the lowest speaker-probing accuracy among compared systems, suggesting reduced speaker leakage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。