让注意力模型直接从自身内部找证据，提升问答准确率。
Retrieval from Within: An Intrinsic Capability of Attention-Based Models
- 用解码器注意力直接查询自身编码的证据块
- 在问答任务上比传统检索生成系统召回率和答案质量更高
- 无需外部检索模块，适合追求端到端效率的场景
检索增强生成（RAG）通常将检索与生成视为独立系统。本文探究注意力编码器-解码器模型是否能直接从自身内部表示中检索信息。提出INTRA（INTrinsic Retrieval via Attention）框架：解码器注意力查询预先编码的证据块，并直接将其作为生成上下文复用。该设计天然统一检索与生成，避免了传统RAG中检索器与生成器之间的不匹配问题。同时通过复用预计算的编码状态，分摊了上下文编码开销。在多个问答基准测试中，INTRA在证据召回率和端到端答案质量上均优于强基线检索生成管道。结果表明，注意力模型本身已具备可激发的内生检索能力，无需额外添加外部模块。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) typically treats retrieval and generation as separate systems. We ask whether an attention-based encoder-decoder can instead retrieve directly from its own internal representations. We introduce INTRA (INTrinsic Retrieval via Attention), a framework where decoder attention queries score pre-encoded evidence chunks that are then directly reused as context for generation. By construction, INTRA unifies retrieval and generation, eliminating the retriever-generator mismatch typical of RAG pipelines. This design also amortizes context encoding by reusing precomputed encoder states across queries. On question-answering benchmarks, INTRA outperforms strong engineered retrieval pipelines on both evidence recall and end-to-end answer quality. Our results demonstrate that attention-based models already possess a retrieval mechanism that can be elicited, rather than added as an external module.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。