arXiv:2506.12364cs.AIcs.CL2025-06被引 26

用强化学习提升多模态文档重排序,让模型像人一样推理并选更准的文档。

MM-R5: MultiModal Reasoning-Enhanced ReRanker via Reinforcement Learning for Document Retrieval

  • 分两阶段训练:先教模型完整推理,再用奖励机制优化排序决策。
  • 在多领域基准上,召回率@1比最优纯检索方法提升超4%。
  • 适合需要高精度文档检索与可解释性分析的场景,如报告问答。

多模态文档检索系统支持文本、图像和版式信息的跨模态信息获取,广泛应用于基于文档的问题回答、报告分析和交互式内容摘要等场景。重排序器通过重新排列候选结果提升检索精度,但现有方法在训练策略和整体效果上仍有较大提升空间,且缺乏显式推理过程,难以分析与优化。本文提出MM-R5,一种基于强化学习的多模态推理增强重排序器,旨在提供更高效可靠的多模态重排序解决方案。该方法分两阶段训练:监督微调(SFT)阶段聚焦指令遵循能力,引导模型生成完整高质量的推理链;为此引入新颖的数据构建策略,生成丰富高质量的推理数据。强化学习(RL)阶段设计任务特异性奖励框架,包含针对多模态候选的重排序奖励,以及基于复合模板的奖励以进一步优化推理质量。在涵盖多个领域的公开挑战性基准MMDocIR上进行大量实验,MM-R5在多数指标上达到当前最优表现,并在其余指标上媲美更大规模模型。相较于最佳仅检索方法,其召回率@1提升超过4%。结果验证了推理增强训练流程的有效性。代码已开源:https://github.com/i2vec/MM-R5。

原文摘要 · Abstract (English)

Multimodal document retrieval systems enable information access across text, images, and layouts, benefiting various domains like document-based question answering, report analysis, and interactive content summarization. Rerankers improve retrieval precision by reordering retrieved candidates. However, current multimodal reranking methods remain underexplored, with significant room for improvement in both training strategies and overall effectiveness. Moreover, the lack of explicit reasoning makes it difficult to analyze and optimize these methods further. In this paper, We propose MM-R5, a MultiModal Reasoning-Enhanced ReRanker via Reinforcement Learning for Document Retrieval, aiming to provide a more effective and reliable solution for multimodal reranking tasks. MM-R5 is trained in two stages: supervised fine-tuning (SFT) and reinforcement learning (RL). In the SFT stage, we focus on improving instruction-following and guiding the model to generate complete and high-quality reasoning chains. To support this, we introduce a novel data construction strategy that produces rich, high-quality reasoning data. In the RL stage, we design a task-specific reward framework, including a reranking reward tailored for multimodal candidates and a composite template-based reward to further refine reasoning quality. We conduct extensive experiments on MMDocIR, a challenging public benchmark spanning multiple domains. MM-R5 achieves state-of-the-art performance on most metrics and delivers comparable results to much larger models on the remaining ones. Moreover, compared to the best retrieval-only method, MM-R5 improves recall@1 by over 4%. These results validate the effectiveness of our reasoning-enhanced training pipeline. Our code is available at https://github.com/i2vec/MM-R5 .

多模态检索强化学习推理增强重排序

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。