arXiv:2601.06458cs.IRcs.CV2026-01被引 3

用图像和文字联合建模,提升电商推荐精度。

PixRec: Leveraging Visual Context for Next-Item Prediction in Sequential Recommendation

  • 融合商品图文信息,构建双塔视觉语言模型。
  • 在亚马逊数据集上,顶级推荐准确率提升3倍,前十名提升40%。
  • 适合关注多模态推荐与电商场景落地的开发者。

大语言模型在仅文本的序列推荐任务中表现出色,通过提示设计、对比对齐和领域微调实现。然而,这些方法忽略了电商等真实场景中的丰富视觉信息。本文提出PixRec——一个结合文本属性与商品图像的视觉语言框架。其架构采用能联合处理图文序列的视觉语言模型骨干,保持双塔结构与混合训练目标,并对物品-物品及用户-物品交互进行多模态特征对齐。在增强产品图像的Amazon Reviews数据集上,实验显示其在顶级推荐准确率上比纯文本模型提升3倍,在前十名准确率上提升40%,表明视觉特征有助于区分文本描述相似的商品。本工作为多模态推荐系统的规模化训练、视觉-文本特征融合优化及推理性能评估指明了方向,推动了视觉信息在真实电商推荐系统中的应用。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have recently shown strong potential for usage in sequential recommendation tasks through text-only models, which combine advanced prompt design, contrastive alignment, and fine-tuning on downstream domain-specific data. While effective, these approaches overlook the rich visual information present in many real-world recommendation scenarios, particularly in e-commerce. This paper proposes PixRec - a vision-language framework that incorporates both textual attributes and product images into the recommendation pipeline. Our architecture leverages a vision-language model backbone capable of jointly processing image-text sequences, maintaining a dual-tower structure and mixed training objective while aligning multi-modal feature projections for both item-item and user-item interactions. Using the Amazon Reviews dataset augmented with product images, our experiments demonstrate $3\times$ and 40% improvements in top-rank and top-10 rank accuracy over text-only recommenders respectively, indicating that visual features can help distinguish items with similar textual descriptions. Our work outlines future directions for scaling multi-modal recommenders training, enhancing visual-text feature fusion, and evaluating inference-time performance. This work takes a step toward building software systems utilizing visual information in sequential recommendation for real-world applications like e-commerce.

序列推荐多模态视觉语言电商

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。