提出可生成贡献与证据的重排序模型,让智能体更高效利用检索结果。
Prism-Reranker: Beyond Relevance Scoring -- Jointly Producing Contributions and Evidence for Agentic Retrieval

- 不仅判断相关性,还生成摘要和精炼证据段落
- 在BEIR-QA上各尺寸模型均表现稳健,4B版提升NDCG@10达1.54
- 适合需要精准引用的自动代理与生成系统使用
现代检索流程服务于如RAG和自主智能体等下游应用,仅输出相关性评分已不足够。传统重排序器迫使智能体将整篇文档注入语言模型上下文,浪费大量令牌。本文提出Prism-Reranker,基于Qwen3.5构建四个规模(0.8B、2B、4B、9B)的模型家族,超越单一评分。当判定相关时,模型同时输出:(i) 文档对查询的贡献摘要,(ii) 保留所有查询相关信号但剔除噪声的自包含证据段落。训练采用混合目标:从强商业重排序API进行点对点知识蒸馏,并结合贡献与证据的监督微调。数据集源自KaLM-Embedding开源聚合,补充真实网络文档(通过商业搜索接口获取)及LLM合成变体,并对部分查询重写为关键词形式以适配智能体流量。为解决开放语料中标注不一致问题,使用由五名前沿大模型组成的评委集成对数据重新标注。在BEIR的QA子集及基于LLM评估的贡献与证据质量测试中,所有尺寸模型均表现良好。进一步验证该方法可扩展至现有模型,使Qwen3-Reranker-4B获得贡献与证据能力,平均BEIR-QA NDCG@10提升+1.54。模型权重、训练方案与评测套件均已开源。
原文摘要 · Abstract (English)
Modern retrieval pipelines increasingly serve downstream consumers like retrieval-augmented generation (RAG) and autonomous agents that need more than a scalar relevance score. A reranker that only tells the caller "how relevant" forces the agent to dump entire documents into the language-model context, wasting tokens on tangential passages and boilerplate. We introduce Prism-Reranker, a family of reranker models built on Qwen3.5 at four sizes (0.8B, 2B, 4B, 9B) that goes beyond scalar scoring. In addition to the standard yes/no relevance judgement, whenever the verdict is yes the model emits (i) a contribution statement summarizing how the document helps the query, and (ii) an evidence passage: a self-contained rewrite that preserves every query-relevant signal while discarding noise. Prism-Reranker is trained with a hybrid objective combining point-wise distillation from a strong commercial reranker API with supervised fine-tuning on contribution and evidence targets. We curate training data from KaLM-Embedding's open-source aggregation, augmented with real web documents retrieved via commercial search APIs for open-domain queries and LLM-synthesized variants, and rewrite a portion of queries into keyword-style reformulations to adapt the model to agent-issued traffic. To reconcile inconsistent labels across open corpora and obtain crisp binary supervision, we relabel data with an LLM-as-Judge ensemble aggregating votes from five frontier LLMs. On a QA subset of BEIR and on an LLM-judged evaluation of contribution and evidence quality, Prism-Reranker attains solid results across all four sizes. We further show that the same recipe extends existing LLM-based rerankers, augmenting Qwen3-Reranker-4B with contribution and evidence capabilities while improving its average BEIR-QA NDCG@10 by +1.54 over the base model. Model weights, training recipe, and evaluation suite are released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。