用三种声学扰动提升长语音识别准确率，无需重新训练。
Whisper-CD: Accurate Long-Form Speech Recognition using Multi-Negative Contrastive Decoding
- 通过对比干净音频与三种声学扰动的输出，实现推理时优化。
- 在五个英文长语音数据集上，错误率降低最多达24.3个百分点。
- 可直接替换现有Whisper系统，速度比束搜索快48%。
使用大型编码器-解码器模型（如Whisper）进行长语音识别时，常出现幻觉、重复循环和内容遗漏等问题，且当先前片段的转录作为解码上下文时，错误会累积并放大。我们提出Whisper-CD，一种无需训练的对比解码框架，将干净音频的输出与三种声学启发式扰动（高斯噪声注入、静音信号、音频时间偏移）生成的负样本输出进行对比。通过log-sum-exp算子聚合这些负样本，构建逐词解码的统一多负样本目标。在五个英文长语音基准测试中，Whisper-CD在CORAAL上将错误率（WER）降低最多24.3个百分点，并实现比束搜索快48%的生成吞吐量。由于Whisper-CD仅在推理阶段运行，可作为即插即用方案应用于已部署的Whisper系统，无需重新训练。
原文摘要 · Abstract (English)
Long-form speech recognition with large encoder-decoder models such as Whisper often exhibit hallucinations, repetition loops, and content omissions. These errors can accumulate and be further amplified when the previous segment's transcription is used as decoding context. We propose Whisper-CD, a training-free contrastive decoding framework that contrasts clean-audio logits against negative logits computed from three acoustically motivated perturbations: Gaussian noise injection, silence signal, and audio temporal shift. We aggregate these negatives via the log-sum-exp operator, building a unified multi-negative objective for token-by-token decoding. Across five English long-form benchmarks, Whisper-CD reduces WER by up to 24.3pp on CORAAL and shows 48% faster token generation throughput than beam search. Because Whisper-CD operates purely at inference time, it can be applied as a drop-in replacement to already-deployed Whisper systems without retraining.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。