用注意力引导定位关键证据，让语音问答更准更快。
Attention-guided Evidence Grounding for Spoken Question Answering
- 利用语音大模型内部注意力机制，直接定位文本中的关键证据。
- 在三个数据集上降低幻觉率，推理延迟减少约62%。
- 适合需要低延迟、高准确率的语音问答系统开发者。
语音问答（Spoken QA）是一个跨模态挑战：需有效对齐语音查询与文本知识，同时避免级联式语音识别（ASR）系统的延迟和误差传播。本文提出注意力引导证据定位（AEG）框架，通过语音大语言模型（SpeechLLMs）的内部跨模态注意力，显式定位并锚定模型隐空间中的关键证据。为解决预训练模型中注意力分布分散的问题，我们设计了“聚焦证据学习”（LFE），一种监督微调范式，校准模型注意力以区分与查询相关的段落和无关上下文。在SQuAD、HotpotQA和MuSiQue上的实验表明，AEG显著降低幻觉，实现强效率提升，优于大型级联基线（Whisper-Large-v3 + Reranker），推理延迟降低约62%。
原文摘要 · Abstract (English)
Spoken Question Answering (Spoken QA) presents a challenging cross-modal problem: effectively aligning acoustic queries with textual knowledge while avoiding the latency and error propagation inherent in cascaded ASR-based systems. In this paper, we introduce Attention-guided Evidence Grounding (AEG), a novel end-to-end framework that leverages the internal cross-modal attention of Speech Large Language Models (SpeechLLMs) to explicitly locate and ground key evidence in the model's latent space. To address the diffuse attention distribution in pre-trained models, we propose Learning to Focus on Evidence (LFE), a supervised fine-tuning paradigm that calibrates the model's attention mechanism to distinguish query-relevant segments from irrelevant context. Experiments on SQuAD, HotpotQA, and MuSiQue demonstrate that AEG reduces hallucinations and achieves strong efficiency gains, outperforming large-scale cascaded baselines (Whisper-Large-v3 + Reranker) while reducing inference latency by approximately 62%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。