用说话人分离引导的Qwen-ASR适配多语言双人对话,显著降低识别错误率。
Diarization-Guided Qwen-ASR Adaptation for Multilingual Two-Speaker Conversational Speech
- 先做说话人分离,再按语言分组,用适配后的Qwen-ASR逐段识别。
- 在开发集上达到23.70的tcpMER,比原模型低6.83个百分点。
- 结合合成语音和强化学习,提升多语言对话识别鲁棒性,适合语音助手等场景。
本文介绍我们为MLC-SLM 2026挑战赛多语言双人对话任务设计的系统。该系统将模块化说话人分离前端与适配后的Qwen3-ASR-1.7B识别器结合:前端完成语音活动检测、子段生成、CAMPPlus嵌入提取、双说话人谱聚类及基于RTTM的音频分割。得到的带说话人标签片段按语言或地区分组,由适配的ASR模型解码。针对ASR适配,先在官方训练数据上进行监督全量微调,再通过三阶段流水线的TTS合成语音增强框架进行LoRA微调,最后使用基于WER/CER奖励和对幻觉、重复、长度偏差惩罚的GRPO强化学习进一步优化。在官方开发集上,系统平均tcpMER为23.70,相比释放的Qwen-ASR-1.7B性能降低6.83绝对点;在最终评估集上,平均tcpMER为17.97。消融实验表明,监督微调贡献最大,合成语音LoRA与强化学习进一步提升鲁棒性。
原文摘要 · Abstract (English)
This paper describes our self-designed system for Task 1 of the MLC-SLM 2026 Challenge for multilingual two-speaker conversational speech. The system combines a modular speaker diarization front end with a challenge-adapted Qwen3-ASR-1.7B recognizer. The diarization front end performs voice activity detection, subsegment generation, CAMPPlus speaker embedding extraction, two-speaker spectral clustering, and RTTM-based audio segmentation. The resulting speaker-attributed segments are grouped by language or region and decoded by the adapted ASR model. For ASR adaptation, we first perform supervised full fine-tuning on the official training data, then apply LoRA fine-tuning with synthetic speech generated by a three-pipeline TTS-based synthetic speech augmentation framework, and finally refine the model using GRPO reinforcement learning with rewards based on WER/CER and penalties for hallucination, repetition, and length deviation. On the official development set, the full system achieves an average tcpMER of 23.70, reducing the error rate by 6.83 absolute points relative to the released Qwen-ASR-1.7B performance. On the final evaluation set, the system achieves an average tcpMER of 17.97. Ablation results show that supervised fine-tuning provides the largest gain, while synthetic-speech LoRA adaptation and reinforcement learning further improve robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。