用自蒸馏提升方言识别，不丢普通话能力。
On-Policy Self-Distillation for Multi-Dialect ASR: Mastering Dialects, Retaining Mandarin
- 学生模型用自身解码前缀训练，教师提供软标签
- 方言识别提升，普通话错误率不变
- 适合需要多方言支持的语音系统开发者
当前大规模语音识别模型在普通话识别上已表现优异，并具备一定方言识别能力。然而在真实语音中，其方言识别准确率仍受限。直接方言适配虽可降低方言错误率（CER），但可能提高普通话错误率。为此，本文研究如何在不损害普通话性能的前提下提升多方言识别能力。采用连续预训练（CPT）与方言监督微调（SFT）构建基础，再以在线策略自蒸馏（OPSD）作为最终优化。OPSD通过冻结教师模型，利用参考转录作为特权上下文生成软标签，指导学生模型在自身解码前缀上学习，替代方言数据上的硬交叉熵更新。该方法缓解了自回归模型的训练-测试不匹配问题，有效提升方言识别，同时保持普通话性能。我们在Qwen3-ASR-1.7B基础上评估，在匹配的微调数据与调度下，OPSD实现方言识别提升而未增加普通话错误率，而持续教师强制微调反而提升了普通话错误率。模型权重与评估脚本将公开。
原文摘要 · Abstract (English)
Recent large-scale ASR models already achieve strong Mandarin recognition accuracy and have some ability to recognize Chinese dialects. However, their dialect recognition accuracy is still limited in real-world speech. Direct dialect adaptation can lower dialect CER, but it may also raise Mandarin CER. We therefore study how to adapt a capable ASR model to improve multi-dialect recognition without degrading Mandarin recognition. We adopt an adaptation pipeline where continual pre-training (CPT) and dialect supervised fine-tuning (SFT) provide a strong foundation, and On-Policy Self-Distillation (OPSD) serves as the final refinement. OPSD addresses the train--test mismatch in autoregressive ASR by training the student model on its own decoded prefixes while a frozen teacher, conditioned on the reference transcript as privileged context, provides soft token-level targets. This replaces hard cross-entropy updates on dialect data with distillation, preserving Mandarin ability while refining dialect recognition. We instantiate the framework with Qwen3-ASR-1.7B and evaluate it on public and internal Mandarin and dialect test sets. Under matched refinement data and schedule, OPSD improves dialect recognition without raising Mandarin CER, whereas continued teacher-forced fine-tuning increases Mandarin CER. We will release the model weights and evaluation scripts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。