arXiv:2509.07485cs.IR2025-09EMNLP

用多视角嵌入提升检索效率,小模型达到大模型效果

Multi-view-guided Passage Reranking with Large Language Models

  • 不生成文本,通过多视角嵌入编码查询与段落信息
  • 220M参数模型性能媲美7B大模型,推理速度提升100倍
  • 对位置偏差不敏感,适合需要高效精准排序的场景

大型语言模型在段落重排序任务中表现优异,但仍面临效率低和外部偏差敏感的问题。现有方法多依赖自回归生成和滑动窗口策略,随着段落数量增加计算开销剧增;同时位置或选择偏差会干扰段落表征并增加输入顺序敏感性。为此,我们提出一种新型非生成式重排序模型MVP,通过无偏的多视角嵌入编码查询-段落信息。每个视角结合查询感知的段落嵌入生成独立锚向量,并在单次解码中直接计算相关性得分。此外,采用正交损失使各视角更具区分度。大量实验表明,仅有220M参数的MVP在性能上媲美7B规模微调模型,推理延迟降低100倍;3B参数版本在域内与域外基准上均达到领先水平。

原文摘要 · Abstract (English)

Recent advances in large language models (LLMs) have shown impressive performance in passage reranking tasks. Despite their success, LLM-based methods still face challenges in efficiency and sensitivity to external biases. (1) Existing models rely mostly on autoregressive generation and sliding window strategies to rank passages, which incur heavy computational overhead as the number of passages increases. (2) External biases, such as position or selection bias, hinder the model's ability to accurately represent passages and increase input-order sensitivity. To address these limitations, we introduce a novel passage reranking model, called Multi-View-guided Passage Reranking (MVP). MVP is a non-generative LLM-based reranking method that encodes query-passage information into diverse view embeddings without being influenced by external biases. For each view, it combines query-aware passage embeddings to produce a distinct anchor vector, which is then used to directly compute relevance scores in a single decoding step. In addition, it employs an orthogonal loss to make the views more distinctive. Extensive experiments demonstrate that MVP, with just 220M parameters, matches the performance of much larger 7B-scale fine-tuned models while achieving a 100x reduction in inference latency. Notably, the 3B-parameter variant of MVP achieves state-of-the-art performance on both in-domain and out-of-domain benchmarks. The source code is available at: https://github.com/bulbna/MVP

检索排序大模型效率优化多视角

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。