用大模型生成领域提示，让语音识别更准
Whisper: Courtside Edition Enhancing ASR Performance Through LLM-Driven Context Generation
- 用多个大模型代理分析比赛语境，生成精准提示
- 在篮球解说上将错误率降低17.0%（从21.7%到18.0%）
- 无需重训练，适合需要快速适配新领域的场景
领域特定语音仍是自动语音识别（ASR）的持续挑战，即使对于最先进的系统如 OpenAI Whisper 也是如此。我们提出 Whisper: Courtside Edition，一种新型多代理大语言模型（LLM）管道，可在不重新训练的情况下增强 Whisper 的转录结果。该管道拦截 Whisper 的初始转录，通过专门的 LLM 代理进行领域上下文识别、命名实体识别和术语检测，并生成紧凑提示以引导 Whisper 的解码器。在 421 段 NBA 篮球解说片段（该领域以密集专有名词和技术术语为特征）上评估，最佳管道实现词错误率（WER）相对降低 17.0%（从 0.217 降至 0.180，p<0.001）。40.1% 的片段表现提升，仅 7.1% 出现退化，显著优于直接转录后编辑。结果表明，基于提示的增强可实现可扩展的领域自适应，为昂贵的模型微调提供实用替代方案。
原文摘要 · Abstract (English)
Domain-specific speech remains a persistent challenge for automatic speech recognition (ASR), even for state-of-the-art systems like OpenAI's Whisper. We introduce Whisper: Courtside Edition, a novel multi-agent large language model (LLM) pipeline that enhances Whisper transcriptions without retraining. The pipeline intercepts Whisper's initial transcript, applies specialized LLM agents for domain context identification, named entity recognition, and jargon detection, and generates compact prompts that guide Whisper's decoder. Evaluated on 421 NBA basketball commentary segments (a domain characterized by dense proper nouns and technical terminology) our best pipeline achieves a statistically significant 17.0% relative reduction in word error rate (WER; from 0.217 to 0.180, p<0.001). Improvements are observed in 40.1% of segments with degradation in only 7.1%, substantially outperforming direct transcript post-editing. These results demonstrate that prompt-based augmentation can deliver scalable domain adaptation for ASR, offering a practical alternative to costly model fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。