arXiv:2602.06654cs.IRcs.AI2026-02

针对美食配送中多模态检索的训练不均衡问题,提出分阶段预训练方法提升检索效果。

Multimodal Generative Retrieval Model with Staged Pretraining for Food Delivery on Meituan

  • 采用分阶段预训练策略,逐步聚焦不同模态任务,避免单一模态主导
  • 在美团真实数据上,召回率提升2.17%~5.10%,点击率增1.02%
  • 适合需要高精度多模态推荐的电商与本地生活平台应用

多模态检索模型在美食配送等场景中日益重要,能通过丰富特征满足用户多样化需求并实现精准匹配。主流方法通常采用查询与物品间的双塔结构,并联合优化塔内与塔间任务。然而我们发现,联合优化常导致某些模态主导训练过程,其他模态被忽视;同时各模态训练速度不一致易引发单轮次问题。为此,我们提出分阶段预训练策略,引导模型在各阶段专注特定任务,有效关注并利用多模态特征,并灵活控制每阶段训练过程以避免单轮次问题。此外,为更好利用压缩高维多模态嵌入的语义ID(SID),设计生成与判别双重任务,帮助模型理解SID、查询与物品特征之间的关联,从而提升整体性能。大规模真实美团数据实验表明,该方法在R@5、R@10、R@20上分别提升3.80%、2.64%、2.17%,在N@5、N@10、N@20上分别提升5.10%、4.22%、2.09%。在线A/B测试显示,该方法使平台收入提升1.12%,点击率提升1.02%,验证了其在实际应用中的有效性与优越性。

原文摘要 · Abstract (English)

Multimodal retrieval models are becoming increasingly important in scenarios such as food delivery, where rich multimodal features can meet diverse user needs and enable precise retrieval. Mainstream approaches typically employ a dual-tower architecture between queries and items, and perform joint optimization of intra-tower and inter-tower tasks. However, we observe that joint optimization often leads to certain modalities dominating the training process, while other modalities are neglected. In addition, inconsistent training speeds across modalities can easily result in the one-epoch problem. To address these challenges, we propose a staged pretraining strategy, which guides the model to focus on specialized tasks at each stage, enabling it to effectively attend to and utilize multimodal features, and allowing flexible control over the training process at each stage to avoid the one-epoch problem. Furthermore, to better utilize the semantic IDs that compress high-dimensional multimodal embeddings, we design both generative and discriminative tasks to help the model understand the associations between SIDs, queries, and item features, thereby improving overall performance. Extensive experiments on large-scale real-world Meituan data demonstrate that our method achieves improvements of 3.80%, 2.64%, and 2.17% on R@5, R@10, and R@20, and 5.10%, 4.22%, and 2.09% on N@5, N@10, and N@20 compared to mainstream baselines. Online A/B testing on the Meituan platform shows that our approach achieves a 1.12% increase in revenue and a 1.02% increase in click-through rate, validating the effectiveness and superiority of our method in practical applications.

多模态检索分阶段训练美团推荐系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。