多信号协同建模商品多模态表示,提升电商搜索排序效果
MMRM: A Multiplex Multimodal Representation Model for Product Ranking in E-commerce Search

- 用多个协同信号联合微调大模型,生成商品多模态统一表示
- 单次推理产出多维度商品特征,排名性能提升显著
- 适合关注电商搜索优化与多模态融合的工程与研究者
多模态信息对电商搜索排序至关重要。现有方法通常通过协同信号微调通用多模态大模型(MLLM),将生成的表示作为商品特征输入排序模型。但存在两大局限:(1)仅依赖单一协同信号微调MLLM,未能利用多样化的任务信号;(2)将多模态表示当作普通特征,未充分挖掘其在用户行为建模中的潜力。为此,我们提出多路多模态表示模型(MMRM),通过共享主干网络与任务特定标记及投影层,统一对多种协同信号进行学习,在一次推理中生成全面的多路商品表示。此外,我们在排序模型中引入多路用户表示策略,基于搜索行为序列建模,利用多路商品表示生成任务相关的用户表征。大量实验表明MMRM兼具高效性与优越性。值得注意的是,该模型已成功部署于京东电商搜索系统,为数百万日活用户提供显著性能提升。
原文摘要 · Abstract (English)
Multimodal information is pivotal for e-commerce search ranking. Existing works leverage multimodal data typically by fine-tuning general Multimodal Large Language Models (MLLMs) via collaborative signals, subsequently integrating the derived representations into ranking models as item features. Despite their efficacy, these methods face two primary limitations: (1) they rely on a single collaborative signal for MLLM fine-tuning, failing to exploit the heterogeneous signals essential for multitask ranking; and (2) they treat multimodal representations as regular item features in ranking models, underutilizing their latent potential for user behavior modeling. To address these challenges, we propose the Multiplex Multimodal Representation Model (MMRM), a unified framework that aligns MLLMs with diverse collaborative signals. By employing a shared backbone with task-specific tokens and projection layers, MMRM simultaneously learns from multiple signals and generates comprehensive multiplex item representations in a single inference pass. Furthermore, we introduce a multiplex user representation strategy in ranking models, which derives task-specific user representations via search-based behavior sequence modeling leveraging multiplex item representations. Extensive experiments demonstrate MMRM's superior efficiency and effectiveness. Notably, MMRM has been successfully deployed in the JD e-commerce search engine, yielding significant performance gains for millions of daily users.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。