用自回归模型提取目标说话人语音，效果优于传统方法。
LauraTSE: Target Speaker Extraction using Auto-Regressive Decoder-Only Language Models
- 基于LauraGPT构建自回归解码器模型，生成目标语音离散编码初稿。
- 引入单步编码器模型融合混合语音与参考语音信息，提升细节还原度。
- 实验验证性能优越，适合语音分离与增强场景的开发者使用。
我们提出LauraTSE，一种基于LauraGPT骨干网络的自回归解码器仅语言模型，用于目标说话人提取。LauraTSE采用小规模自回归解码器仅模型，从混合语音和参考语音的连续嵌入中生成目标语音离散编码的初始层。这些输出作为粗粒度预测。为细化结果，一个单步编码器仅语言模型通过融合混合语音与参考语音信息，重建完整的离散编码表示，添加细粒度细节。实验结果表明，该方法表现优异。此外，我们通过消融实验研究了数据可扩展性及编码器仅模型的贡献。
原文摘要 · Abstract (English)
We propose LauraTSE, an Auto-Regressive Decoder-Only Language Model for Target Speaker Extraction built upon the LauraGPT backbone. LauraTSE employs a small-scale auto-regressive decoder-only language model that generates the initial layers of the target speech's discrete codec representations from the continuous embeddings of both the mixture and reference speech. These outputs serve as coarse-grained predictions. To refine them, a one-step encoder-only language model reconstructs the full codec representation by integrating information from both the mixture and the reference speech, adding fine-grained details. Experimental results show that our approach can achieve promising performance. Additionally, we conduct ablation studies to investigate the data scalability and the contribution of the encoder-only model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。