让检索更懂任务：用LLM生成质量衡量检索效果
Beyond Relevance: Utility-Centric Retrieval in the LLM Era
- 以LLM生成结果的好坏为标准，重新定义检索目标
- 提出区分通用与特定于LLM的实用价值评估框架
- 适合构建智能问答和代理系统的研发者参考
传统信息检索系统以主题相关性为核心优化目标，但相关性仅近似反映信息的真正价值——即是否有助于完成用户任务。大语言模型（LLM）时代的检索增强生成（RAG）改变了这一范式：检索结果不再直接供用户阅读，而是作为LLM生成答案的证据。因此，检索的有效性必须通过其对生成质量的实际贡献来评估，而非仅依赖相关性指标。本教程主张检索目标正从相关性导向转向以LLM为中心的实用性导向。我们提出一个统一框架，涵盖不依赖LLM与针对LLM的实用性、上下文无关与上下文相关的实用性，以及与LLM信息需求和代理式RAG的关联。通过整合最新进展，该教程为设计契合LLM信息获取需求的检索系统提供了概念基础与实践指导。
原文摘要 · Abstract (English)
Information retrieval systems have traditionally optimized for topical relevance-the degree to which retrieved documents match a query. However, relevance only approximates a deeper goal: utility, namely, whether retrieved information helps accomplish a user's underlying task. The emergence of retrieval-augmented generation (RAG) fundamentally changes this paradigm. Retrieved documents are no longer consumed directly by users but instead serve as evidence for large language models (LLMs) that produce answers. As a result, retrieval effectiveness must be evaluated by its contribution to generation quality rather than by relevance-based ranking metrics alone. This tutorial argues that retrieval objectives are evolving from relevance-centric optimization toward LLM-centric utility. We present a unified framework covering LLM-agnostic versus LLM-specific utility, context-independent versus context-dependent utility, and the connection with LLM information needs and agentic RAG. By synthesizing recent advances, the tutorial provides conceptual foundations and practical guidance for designing retrieval systems aligned with the requirements of LLM-based information access.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。