用参数差值修正语音识别伪标签偏差，提升跨方言识别效果
Pseudo2Real: Task Arithmetic for Pseudo-Label Correction in Automatic Speech Recognition
- 通过真实与伪标签模型的权重差生成纠错向量
- 在十种非洲口音上实现最高35%的词错误率降低
- 适合数据稀缺下提升跨域语音识别鲁棒性
在领域偏移下实现鲁棒语音识别至关重要，因实际系统需应对未见口音与领域，而标注数据有限。尽管伪标签提供实用方案，但常引入系统性、口音相关的错误，过滤难以解决。我们提出一种简单参数空间修正方法：在包含真实与伪标签数据的源域中，从相同初始化出发，分别对真实标签和伪标签微调两个语音识别模型，其权重差形成捕捉伪标签偏差的修正向量。将该向量应用于目标域的伪标签模型，可显著提升识别性能，在使用Whisper tiny模型时，于AfriSpeech-200数据集上的十种非洲口音上实现最高达35%的相对词错误率（WER）降低。
原文摘要 · Abstract (English)
Robust ASR under domain shift is crucial because real-world systems encounter unseen accents and domains with limited labeled data. Although pseudo-labeling offers a practical workaround, it often introduces systematic, accent-specific errors that filtering fails to fix. We ask: How can we correct these recurring biases without target ground truth? We propose a simple parameter-space correction: in a source domain containing both real and pseudo-labeled data, two ASR models are fine-tuned from the same initialization, one on ground-truth labels and the other on pseudo-labels, and their weight difference forms a correction vector that captures pseudo-label biases. When applied to a pseudo-labeled target model, this vector enhances recognition, achieving up to a 35% relative Word Error Rate (WER) reduction on AfriSpeech-200 across ten African accents with the Whisper tiny model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。