通过可解释注意力对齐,让检索模型更精准找到支持性证据。
Beyond Topical Similarity: Contrastive Evidence Retrieval with Interpretable Attention Alignment in RAG

- 用主观性硬负例筛选提升检索质量
- 注意力对齐使模型选择证据更可信且效果不降
- 适合需要可解释性的医疗等高风险场景
确保RAG中的事实性和可解释性仍是开放且紧迫的问题。本文提出对比证据推理注意力(CERA),首个采用基于主观性的硬负例选择并引入证据归纳偏置的检索框架,通过辅助注意力对齐损失实现对比学习。CERA微调密集检索器时使用两个目标:三元组对比学习和可解释注意力对齐,后者利用人类标注的事实性理由中的词性加权掩码分布来监督CLS到标记的注意力。在大型临床试验报告语料库上的实验表明,基于主观性的硬负例选择显著优于Contriever及传统硬负例基线。此外,理由对齐提升了忠实度,同时保持了竞争力的检索性能,验证了在人类理由引导下注意力可作为更忠实的模型行为解释。超越主题相似性,CERA使检索器能识别构成支持性证据的具体标记,推动RAG系统中更可解释的证据选择。
原文摘要 · Abstract (English)
Ensuring factuality and interpretability in RAG remains an open and urgent problem. We introduce Contrastive Evidence Rationale Attention (CERA), the first retrieval framework to employ subjectivity-based hard negative selection and inject an evidential inductive bias into contrastive learning through an auxiliary attention alignment loss. CERA fine-tunes a dense retriever using two training objectives: triplet-based contrastive learning and interpretable attention alignment, which supervises CLS-to-token attention using a part-of-speech-weighted masking distribution over human-annotated factual rationales as evidence signals. Experiments on a large corpus of clinical trial reports demonstrate that the subjectivity-based hard negative selection substantially improves retrieval effectiveness compared to both Contriever and hard negative selection baselines. Furthermore, rationale alignment improves faithfulness while maintaining competitive retrieval performance, supporting the hypothesis that attention can serve as a more faithful explanation of model behavior when guided by human rationales. Moving beyond topical similarity, CERA enables the retriever to identify the specific tokens that constitute supporting evidence, promoting more interpretable evidence selection in RAG systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。