arXiv:2602.05408cs.IR2026-02中稿 · the Full Research …

用多维度信号提升多媒体搜索满意度,让结果更懂用户需求。

Rich-Media Re-Ranker: A User Satisfaction-Driven LLM Re-ranking Framework for Rich-Media Search

  • 通过查询拆解与多模态信号融合,精细建模用户复杂意图。
  • 上线后用户参与度与满意度显著提升,效果可量化验证。
  • 适合关注搜索体验优化与多模态排序的工业级应用者。

重排序在现代信息检索系统中至关重要,能有效提升初始结果的相关性以满足用户信息需求。然而,现有方法在提升用户满意度方面存在两大局限:难以充分建模多维度用户意图,且忽略视觉感知等丰富侧信息。为此,我们提出 Rich-Media Re-Ranker 框架,通过多维、细粒度建模来增强用户搜索满意度。该框架首先引入查询规划器(Query Planner),分析会话中的查询演化序列,将原始查询分解为清晰互补的子查询,以覆盖用户潜在意图。随后,超越文本内容,整合候选结果的丰富侧信息,包括基于视觉语言模型(VLM)评估器生成的视觉内容信号。这些综合信号结合精心设计的重排序原则(涵盖内容相关性、质量、信息增益、新颖性及封面图像视觉呈现)进行处理。最终由基于大语言模型(LLM)的重排序器执行整体评估。为进一步提升 VLM 评估器与 LLM 重排序器在不同场景下的适应性,我们采用多任务强化学习增强其能力。该框架已部署于大规模工业搜索系统,显著提升了在线用户参与率与满意度指标。

原文摘要 · Abstract (English)

Re-ranking plays a crucial role in modern information search systems by refining the ranking of initial search results to better satisfy user information needs. However, existing methods show two notable limitations in improving user search satisfaction: inadequate modeling of multifaceted user intents and neglect of rich side information such as visual perception signals. To address these challenges, we propose the Rich-Media Re-Ranker framework, which aims to enhance user search satisfaction through multi-dimensional and fine-grained modeling. Our approach begins with a Query Planner that analyzes the sequence of query refinements within a session, decomposing the query into clear and complementary sub-queries to enable broader coverage of users' potential intents. Subsequently, moving beyond primary text content, we integrate richer side information of candidate results, including signals modeling visual content generated by the VLM-based evaluator. These comprehensive signals are then processed alongside carefully designed re-ranking principle that considers multiple facets, including content relevance and quality, information gain, information novelty, and the visual presentation of cover images. Then, the LLM-based re-ranker performs the holistic evaluation based on these principles and integrated signals. To enhance the scenario adaptability of the VLM-based evaluator and the LLM-based re-ranker, we further enhance their capabilities through multi-task reinforcement learning. The proposed framework has been deployed in a large-scale industrial search system, yielding substantial improvements in online user engagement rates and satisfaction metrics.

搜索排序多模态LLM应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。