用6亿参数大模型提升语音验证准确率，压缩后仅损失0.04%性能
Enhancing Speaker Verification with w2v-BERT 2.0 and Knowledge Distillation guided Structured Pruning
- 基于w2v-BERT 2.0提取多层特征，结合层适配器与LoRA微调
- 在Vox1-O/H上达0.12%/0.55%的最低错误率（EER）
- 知识引导结构剪枝压缩80%模型体积，适合部署场景
大规模自监督预训练模型（PTMs）通过提供丰富特征表示，在说话人验证（SV）任务中表现显著提升。本文采用约6亿参数、在143种语言共450万小时无标签数据上训练的w2v-BERT 2.0模型进行SV任务。使用带有层适配器的MFA结构处理模型多层输出，提取说话人嵌入，并引入LoRA实现高效微调。模型在Vox1-O和Vox1-H测试集上分别取得0.12%和0.55%的等错误率（EER）新纪录。此外，应用知识蒸馏引导的结构化剪枝，使模型规模减少80%，仅导致0.04% EER退化。源代码与模型已公开于https://github.com/ZXHY-82/w2v-BERT-2.0_SV。
原文摘要 · Abstract (English)
Large-scale self-supervised Pre-Trained Models (PTMs) have shown significant improvements in the speaker verification (SV) task by providing rich feature representations. In this paper, we utilize w2v-BERT 2.0, a model with approximately 600 million parameters trained on 4.5 million hours of unlabeled data across 143 languages, for the SV task. The MFA structure with Layer Adapter is employed to process the multi-layer feature outputs from the PTM and extract speaker embeddings. Additionally, we incorporate LoRA for efficient fine-tuning. Our model achieves state-of-the-art results with 0.12% and 0.55% EER on the Vox1-O and Vox1-H test sets, respectively. Furthermore, we apply knowledge distillation guided structured pruning, reducing the model size by 80% while achieving only a 0.04% EER degradation. Source code and models are released at https://github.com/ZXHY-82/w2v-BERT-2.0_SV.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。