arXiv:2606.09082cs.IR2026-06

提升视觉信息在推荐系统中的作用,让模型更会‘看’。

Teach Multimodal Recommendation Model to See via Personalized Visual Extraction and Adaptive Learning

论文配图:Teach Multimodal Recommendation Model to See via Personalized Visual Extraction and Adaptive Learning
图 1 · 摘自论文原文
  • 通过反馈引导的视觉提取增强视觉特征学习
  • 动态调整视觉学习权重,缓解图文模态不平衡
  • 无需修改主模型,适配多种推荐系统框架

多模态序列推荐(MSR)融合文本与视觉信息以提升推荐质量。然而,近期研究及我们的实证分析表明,视觉特征常被低估,贡献远低于文本信号。这主要源于两个问题:视觉表征学习不足(预训练编码器未能捕捉偏好相关线索)和图文优化失衡(文本特征主导学习过程)。为此,我们提出一个即插即用的框架REVEAL,通过反馈引导的视觉提取(FVE)和自适应视觉学习(AVL),分别改进视觉特征提取与跨模态优化。FVE利用任务级反馈优化提示引导的视觉区域选择,AVL动态重置视觉学习权重以缓解模态不平衡。在多个真实数据集和多种MSR骨干模型上的实验表明,REVEAL能持续提升推荐性能。进一步分析显示,性能提升源于对偏好相关视觉区域更有效的注意力聚焦以及训练中更充分的视觉信息利用。代码已开源。

原文摘要 · Abstract (English)

Multimodal sequential recommendation (MSR) incorporates textual and visual information to improve recommendation quality. However, recent studies and our empirical analysis show that visual features are often underutilized, thereby contributing far less than textual signals. We attribute this issue to two factors: insufficient visual representation learning (pretrained encoders fail to capture preference-relevant cues) and unbalanced visual-text optimization (textual features dominate the learning process). To address these issues, we propose Teach Multimodal Recommendation Model to See via Personalized Visual Extraction and Adaptive Learning (REVEAL), a plug-and-play framework that enhances visual representation learning and cross-modal optimization without modifying the original recommendation backbone. REVEAL consists of Feedback-Guided Visual Extraction (FVE), which refines prompt-guided visual extraction through task-level feedback, and Adaptive Visual Learning (AVL), which dynamically reweights visual learning to alleviate modality imbalance. Experiments on multiple real-world datasets and MSR backbones demonstrate that REVEAL consistently improves recommendation performance. Further analysis shows that these gains arise from more effective attention to preference-relevant visual regions and better visual utilization during training. The code is available at https://github.com/YutongLi2024/REVEAL.

多模态推荐视觉提取自适应学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。