arXiv:2602.12528cs.IRcs.CL2026-02被引 3

用扩散语言模型重排文档,效率更高且更灵活。

DiffuRank: Effective Document Reranking with Diffusion Language Models

  • 采用扩散模型实现非自回归的并行文档重排,突破传统生成顺序限制。
  • 三种策略均达到与自回归模型相当甚至更优的重排效果,零样本和微调均有效。
  • 适合需要高效、可调控重排结果的研究者,尤其关注检索系统优化者。

大型语言模型(LLM)推动了文档重排的新范式,但现有方法多依赖自回归生成,存在延迟高、错误传播等问题。本文探索扩散语言模型(dLLMs)在文档重排中的应用,提出DiffuRank框架。不同于自回归模型,dLLMs支持非顺序、并行解码,提升效率与可控性。本文设计三种策略:点对点评估单个查询-文档相关性;基于logit的列表级联合评估;以及基于排列的列表级解码。每种策略均配套训练方法以充分发挥dLLMs优势。在多个基准上测试零样本与微调性能,结果显示,相同规模下dLLMs表现媲美甚至超越自回归模型,证明扩散模型是文档重排中极具潜力的替代架构。

原文摘要 · Abstract (English)

Recent advances in large language models (LLMs) have inspired new paradigms for document reranking. While this paradigm better exploits the reasoning and contextual understanding capabilities of LLMs, most existing LLM-based rerankers rely on autoregressive generation, which limits their efficiency and flexibility. In particular, token-by-token decoding incurs high latency, while the fixed left-to-right generation order causes early prediction errors to propagate and is difficult to revise. To address these limitations, we explore the use of diffusion language models (dLLMs) for document reranking and propose DiffuRank, a reranking framework built upon dLLMs. Unlike autoregressive models, dLLMs support more flexible decoding and generation processes that are not constrained to a left-to-right order, and enable parallel decoding, which may lead to improved efficiency and controllability. Specifically, we investigate three reranking strategies based on dLLMs: (1) a pointwise approach that uses dLLMs to estimate the relevance of each query-document pair; (2) a logit-based listwise approach that prompts dLLMs to jointly assess the relevance of multiple documents and derives ranking lists directly from model logits; and (3) a permutation-based listwise approach that adapts the canonical decoding process of dLLMs to the reranking tasks. For each approach, we design corresponding training methods to fully exploit the advantages of dLLMs. We evaluate both zero-shot and fine-tuned reranking performance on multiple benchmarks. Experimental results show that dLLMs achieve performance comparable to, and in some cases exceeding, that of autoregressive LLMs with similar model sizes. These findings demonstrate the promise of diffusion-based language models as a compelling alternative to autoregressive architectures for document reranking.

文档重排扩散模型大模型检索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。