提出可复现的框架，评估大模型在带引用信息检索中的表现
An Evaluation Framework for Attributed Information Retrieval using Large Language Models
- 构建三种架构对比：直接生成、先检索再生成、先生成再检索
- 在HAGRID数据集上验证，不同策略对答案准确性和引用可靠性影响显著
- 支持任意主干模型，适合评估生成式搜索系统
随着大语言模型（LLMs）在信息检索场景中的成功应用，搜索引擎正采用生成式方法提供带内联引用的答案。现有研究主要关注带引用的问题回答，而本文聚焦更具挑战性的开放性信息检索任务，因其查询形式多样、候选答案的引用来源种类繁多。为此，我们提出一个可复现的评估与基准框架，适用于任意主干大模型及三种不同架构设计：（1）直接生成；（2）先检索再生成；（3）先生成再检索。基于HAGRID数据集的实验表明，不同策略对答案正确性与引用可靠性均有显著影响。
原文摘要 · Abstract (English)
With the growing success of Large Language models (LLMs) in information-seeking scenarios, search engines are now adopting generative approaches to provide answers along with in-line citations as attribution. While existing work focuses mainly on attributed question answering, in this paper, we target information-seeking scenarios which are often more challenging due to the open-ended nature of the queries and the size of the label space in terms of the diversity of candidate-attributed answers per query. We propose a reproducible framework to evaluate and benchmark attributed information seeking, using any backbone LLM, and different architectural designs: (1) Generate (2) Retrieve then Generate, and (3) Generate then Retrieve. Experiments using HAGRID, an attributed information-seeking dataset, show the impact of different scenarios on both the correctness and attributability of answers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。