arXiv:2508.05909cs.CL2025-08AAAI被引 2

用新指标提升检索摘要质量,让模型自主选最相关的内容。

Beyond Perplexity: Let the Reader Select Retrieval Summaries via Spectrum Projection Score

  • 通过谱投影分数衡量摘要与模型隐表示的语义匹配度
  • 在5个问答数据集上提升生成性能,效果优于传统困惑度指标
  • 适合关注检索增强生成优化的研究者和工程师

大型语言模型通过检索增强生成(RAG)框架显著提升生成效果,该框架基于检索-阅读范式,将外部知识补充到模型输入中。然而,以往研究常整体评估RAG,难以分离检索模块的真实贡献,尤其因语言模型对提示敏感而加剧了这一问题。本文提出一种轻量级、无需监督的指标——谱投影分数(Spectrum Projection Score, SPS),让阅读器通过比较摘要生成词元构成的区域与模型子空间主方向之间的重叠程度,来评估检索摘要的相关性。基于SPS,我们进一步提出xCompress框架,可在推理阶段动态采样、排序并压缩检索摘要候选。在五个问答基准上使用四款开源LLM进行的大量实验表明,SPS不仅提升了多任务表现,还为检索与生成之间的交互提供了可解释的分析视角。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have shown improved generation performance through retrieval-augmented generation (RAG) following the retriever-reader paradigm, which supplements model inputs with externally retrieved knowledge. However, prior work often evaluates RAG holistically, assessing the retriever and reader jointly, making it difficult to isolate the true contribution of retrieval, particularly given the prompt sensitivity of LLMs used as readers. We move beyond perplexity and introduce Spectrum Projection Score (SPS), a lightweight and supervision-free metric that allows the reader to gauge the semantic alignment of a retrieved summary with its hidden representation by comparing the area formed by generated tokens from the summary, and the principal directions of subspace in the reader and to measure the relevance. Building on SPS we present xCompress, an inference-time controller framework that dynamically samples, ranks, and compresses retrieval summary candidates. Extensive experiments on five QA benchmarks with four open-sourced LLMs show that SPS not only enhances performance across a range of tasks but also provides a principled perspective on the interaction between retrieval and generation.

检索增强生成评估语言模型推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。