arXiv:2503.23312cs.AIcs.CV2025-03KDD被引 8

让对话推荐系统读懂图片,提升时尚家居类商品推荐精度。

LaViC: Adapting Large Vision-Language Models to Visually-Aware Conversational Recommendation

  • 用自蒸馏压缩图像为少量视觉令牌,降低计算开销。
  • 融合对话与压缩视觉信息,在多个数据集上超越纯文本方法。
  • 适合需要视觉细节的推荐场景,如穿搭、家装设计。

对话式推荐系统通过对话理解用户需求并提供个性化建议。尽管文本信息在多数领域足够,但时尚、家居等视觉驱动领域仍需颜色、风格、设计等详细视觉信息。为此,我们提出LaViC(大视觉语言对话推荐框架),将紧凑的图像表示融入对话推荐系统。LaViC采用两阶段流程:(1) 视觉知识自蒸馏,将产品图像从数百个标记压缩为少量视觉标记,显著降低计算开销;(2) 推荐提示调优,使模型能统一捕获对话上下文与蒸馏后的视觉标记。为支持严谨评估,我们构建了一个新数据集,将Reddit对话与亚马逊多类别视觉商品列表对齐(如时尚、美妆、家居),覆盖真实用户查询与商品外观。大量实验表明,LaViC显著优于仅依赖文本的对话推荐方法及开源视觉语言基线,在准确率上媲美甚至超过主流专有模型(如GPT-3.5-turbo、GPT-4o-mini、GPT-4o),验证了显式使用视觉数据捕捉商品属性的重要性及本框架的有效性。代码与数据集已开源。

原文摘要 · Abstract (English)

Conversational recommender systems engage users in dialogues to refine their needs and provide more personalized suggestions. Although textual information suffices for many domains, visually driven categories such as fashion or home decor potentially require detailed visual information related to color, style, or design. To address this challenge, we propose LaViC (Large Vision-Language Conversational Recommendation Framework), a novel approach that integrates compact image representations into dialogue-based recommendation systems. LaViC leverages a large vision-language model in a two-stage process: (1) visual knowledge self-distillation, which condenses product images from hundreds of tokens into a small set of visual tokens in a self-distillation manner, significantly reducing computational overhead, and (2) recommendation prompt tuning, which enables the model to incorporate both dialogue context and distilled visual tokens, providing a unified mechanism for capturing textual and visual features. To support rigorous evaluation of visually-aware conversational recommendation, we construct a new dataset by aligning Reddit conversations with Amazon product listings across multiple visually oriented categories (e.g., fashion, beauty, and home). This dataset covers realistic user queries and product appearances in domains where visual details are crucial. Extensive experiments demonstrate that LaViC significantly outperforms text-only conversational recommendation methods and open-source vision-language baselines. Moreover, LaViC achieves competitive or superior accuracy compared to prominent proprietary baselines (e.g., GPT-3.5-turbo, GPT-4o-mini, and GPT-4o), demonstrating the necessity of explicitly using visual data for capturing product attributes and showing the effectiveness of our vision-language integration. Our code and dataset are available at https://github.com/jeon185/LaViC.

对话推荐视觉语言图像压缩多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。