不训练也不依赖外部模型，用声学信号修正大模型语音识别的幻觉问题。
Likelihood-Constrained Acoustic Reranking for Training-Free Hallucination Mitigation in LLM-Based ASR
- 基于声学兼容性重排候选词，只干预模型认为可能的选项。
- 在δ=0.60时，减少38.8%至57.1%的幻觉错误，保持标准测试集准确率。
- 适合追求高鲁棒性、无额外训练成本的语音识别系统部署者。
基于大语言模型（LLM）的自动语音识别（ASR）系统凭借强大的语言先验和多语言能力，在常规语音数据上表现优异。但在挑战性条件下，这些先验可能压制声学证据，导致意外翻译、指令执行、重复或灾难性删除。我们提出无需训练的声学重排序方法LCAR，提升声学对齐同时保留基础模型支持。每个解码步骤中，LCAR首先保留基模型概率在贪婪词邻域内的候选词，再利用注意力池化音频嵌入与现有语言模型头计算声学兼容性得分进行重排序。通过仅对模型支持的合理选项进行声学干预，LCAR无需额外训练、外部检测器、参考转录或辅助模型。我们在四个基于LLM的ASR系统上，使用人工听审的文本转语音（TTS）和开源语音挑战套件评估。在δ=0.60时，LCAR消除38.8%–57.1%的检测到的幻觉失败，同时在标准开源测试集上基本维持原有的词错误率（WER）/字符错误率（CER）。
原文摘要 · Abstract (English)
Large language model (LLM)-based automatic speech recognition (ASR) systems achieve strong performance on conventional speech data by leveraging powerful linguistic priors and multilingual capabilities. However, under challenging conditions, these priors can override acoustic evidence, resulting in unintended translation, instruction execution, repetition, or catastrophic deletion. We propose Likelihood-Constrained Acoustic Reranking (LCAR), a training-free decoding method that improves acoustic grounding while preserving support from the base model. At each decoding step, LCAR first retains tokens whose base-model likelihood falls within a margin of the greedy token, then reranks them using an acoustic compatibility score computed from attention-pooled audio embeddings and the existing LM head. By restricting acoustic intervention to plausible, model-supported alternatives, LCAR requires no additional training, external detector, reference transcript, or auxiliary model at inference. We evaluate LCAR on four LLM-based ASR systems using human-audited TTS and open-source speech challenge suites. At $δ=0.60$, LCAR removes 38.8--57.1\% of detector-identified hallucination failures while largely maintaining WER/CER on standard open-source test sets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。