arXiv:2508.16998cs.CLcs.IR2025-08EMNLP被引 21

用双阶段框架提升文档重排序准确率与可解释性。

DeAR: Dual-Stage Document Reranking with Reasoning Agents via LLM Distillation

论文配图:DeAR: Dual-Stage Document Reranking with Reasoning Agents via LLM Distillation
图 1 · 摘自论文原文
  • 分两阶段:先蒸馏细粒度相关性,再用大模型生成推理链
  • 在多个数据集上超越开源基线,最高提升5.1 nDCG@5
  • 适合需要高精度与可解释性的开放域问答系统

大型语言模型(LLMs)通过全局推理提升了文档重排序性能,但单一模型难以兼顾细粒度相关性评分与跨文档整体分析。本文提出开源框架DeAR,采用双阶段策略解耦任务:第一阶段将13B参数的LLaMA教师模型的词级相关性信号,通过交叉熵、RankNet与KL散度混合损失,蒸馏至3-8B的小型学生模型,实现稳健的点对点评分;第二阶段引入LoRA适配器,在20,000条GPT-4o生成的思维链排列上微调,实现基于自然语言的列表级推理与解释。在TREC-DL19/20、八个BEIR数据集及NovelEval-2306上评估,DeAR在DL20上比开源基线高出+5.1 nDCG@5,NovelEval上达90.97 nDCG@10,优于GPT-4达+3.09。无需在Wikipedia上微调,其在开放域问答中表现优异,自然问题数据集上达到54.29 Top-1准确率,超越MonoT5、UPR、RankGPT等基线。消融实验表明,双损失蒸馏确保了稳定校准,使DeAR成为现代重排序系统的高效且可解释解决方案。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have transformed listwise document reranking by enabling global reasoning over candidate sets, yet single models often struggle to balance fine-grained relevance scoring with holistic cross-document analysis. We propose \textbf{De}ep\textbf{A}gent\textbf{R}ank (\textbf{\DeAR}), an open-source framework that decouples these tasks through a dual-stage approach, achieving superior accuracy and interpretability. In \emph{Stage 1}, we distill token-level relevance signals from a frozen 13B LLaMA teacher into a compact \{3, 8\}B student model using a hybrid of cross-entropy, RankNet, and KL divergence losses, ensuring robust pointwise scoring. In \emph{Stage 2}, we attach a second LoRA adapter and fine-tune on 20K GPT-4o-generated chain-of-thought permutations, enabling listwise reasoning with natural-language justifications. Evaluated on TREC-DL19/20, eight BEIR datasets, and NovelEval-2306, \DeAR surpasses open-source baselines by +5.1 nDCG@5 on DL20 and achieves 90.97 nDCG@10 on NovelEval, outperforming GPT-4 by +3.09. Without fine-tuning on Wikipedia, DeAR also excels in open-domain QA, achieving 54.29 Top-1 accuracy on Natural Questions, surpassing baselines like MonoT5, UPR, and RankGPT. Ablations confirm that dual-loss distillation ensures stable calibration, making \DeAR a highly effective and interpretable solution for modern reranking systems.\footnote{Dataset and code available at https://github.com/DataScienceUIBK/DeAR-Reranking.}.

文档重排序大模型蒸馏可解释性链式推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。