让语音识别模型更快更准,自动摆脱大模型依赖
ASKD-Whisper: Adaptive Self-knowledge Distillation for Efficient and Low-Latency Automatic Speech Recognition
- 动态降低对大模型的依赖,逐步培养小模型独立推理能力
- 推理速度提升5倍,词错误率比原模型低1.07%
- 适合需要快速部署且泛化能力强的语音识别场景
知识蒸馏是将大规模基础模型压缩为可部署架构的有效方法。在自动语音识别领域,以往研究多要求学生模型严格模仿教师模型的预测分布,但这种静态依赖常导致学生模型继承教师的领域特定盲点和过度自信的幻觉,严重削弱其在分布外数据上的泛化能力。为此,我们提出自适应自知识蒸馏(ASKD),一种动态课程框架:随着训练推进,系统性减弱对学生模型对教师分布的依赖,从而释放其独立推理能力,并随后通过自知识蒸馏阶段作为结构正则化。基于ASKD,我们将庞大的Whisper架构压缩为紧凑版本ASKD-Whisper。在多种声学领域的综合评估中,ASKD-Whisper不仅实现推理延迟5倍加速,还以1.07%更低的词错误率(WER)超越其教师模型。结果表明,ASKD有效防止了教师诱导的过拟合,建立了可泛化的模型压缩新基准。
原文摘要 · Abstract (English)
Knowledge distillation (KD) is one of the most effective paradigms for compressing large-scale foundation models into deployable architectures. In the context of Automatic Speech Recognition (ASR), previous studies have predominantly focused on forcing the student model to strictly mimic the predictive distribution of a massive teacher model. However, this static dependency often presents an inherent trade-off: while the student rapidly acquires basic linguistic representations, it simultaneously inherits the teacher's domain-specific blind spots and over-confident hallucinations, leading to a severe decline in out-of-distribution generalization capacity. To effectively mitigate this issue, we propose Adaptive Self-Knowledge Distillation (ASKD), a dynamic curriculum framework. ASKD systematically decays the dependency on the teacher's distribution as training progresses-thereby unlocking the student's independent reasoning capacity-and subsequently employs a self-knowledge distillation phase to act as a structural regularizer. By applying ASKD, we distill the massive Whisper architecture into a compact variant, ASKD-Whisper. In our comprehensive evaluations across diverse acoustic domains, ASKD-Whisper not only achieves a 5x speedup in inference latency but also outperforms its teacher model by yielding a 1.07% lower word error rate (WER). These results demonstrate that ASKD effectively prevents teacher-induced overfitting and establishes a new state-of-the-art for generalizable model compression.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。