用视觉语言模型提升电商商品推荐效果
Improving Visual Recommendation on E-commerce Platforms Using Vision-Language Models
- 基于百万商品图文对微调SigLIP模型，生成用于推荐的图像嵌入
- 离线评估nDCG@5提升9.1%，线上点击率增50%、转化率增14%
- 适合做视觉相似推荐的工程师和产品经理参考
在月活超2000万用户的大型电商平台中，推荐视觉相似商品对提升用户发现符合偏好的商品效率至关重要。本研究将视觉语言模型（VLM）应用于日本主流C2C平台Mercari的商品推荐系统。具体地，我们使用三个月内收集的一百万个商品图像-标题对，对采用sigmoid对比损失的SigLIP模型进行微调，构建用于推荐系统的图像编码器。评估包括历史交互日志的离线分析与生产环境中的在线A/B测试。离线分析显示，该模型在nDCG@5上相较基线提升9.1%；在线测试中，点击率提高50%，转化率提升14%。结果表明，基于VLM的编码器在电商推荐中有效，为视觉相似推荐系统的设计提供了实用洞见。
原文摘要 · Abstract (English)
On large-scale e-commerce platforms with tens of millions of active monthly users, recommending visually similar products is essential for enabling users to efficiently discover items that align with their preferences. This study presents the application of a vision-language model (VLM) -- which has demonstrated strong performance in image recognition and image-text retrieval tasks -- to product recommendations on Mercari, a major consumer-to-consumer marketplace used by more than 20 million monthly users in Japan. Specifically, we fine-tuned SigLIP, a VLM employing a sigmoid-based contrastive loss, using one million product image-title pairs from Mercari collected over a three-month period, and developed an image encoder for generating item embeddings used in the recommendation system. Our evaluation comprised an offline analysis of historical interaction logs and an online A/B test in a production environment. In offline analysis, the model achieved a 9.1% improvement in nDCG@5 compared with the baseline. In the online A/B test, the click-through rate improved by 50% whereas the conversion rate improved by 14% compared with the existing model. These results demonstrate the effectiveness of VLM-based encoders for e-commerce product recommendations and provide practical insights into the development of visual similarity-based recommendation systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。