arXiv:2506.20817cs.IRcs.MM2025-06被引 6

用电影剧情和预告片增强推荐系统,提升冷门影片的检索效果。

RAG-VisualRec: An Open Resource for Vision- and Text-Enhanced Retrieval-Augmented Generation in Recommendation

  • 用大模型生成剧情描述,融合视频视觉特征,丰富稀疏元数据。
  • 基于CCA的融合策略使召回率优于单一模态基线,重排显著提升nDCG。
  • 开源资源含质量评估模块,适合研究多模态推荐与生成内容审核。

本文针对电影推荐中因项目元数据(如片名、类型)稀疏导致检索质量受限的问题,提出RAG-VisualRec——一个开放资源与可复现的多模态推荐流程。该流程结合(i)大模型生成的影片剧情描述,以及(ii)预告片提取的视觉(及可选音频)嵌入,支持检索增强生成(RAG)与协同过滤式工作流。通过可配置的融合策略(如PCA、CCA)在检索前整合多模态信号,并支持可选的大模型重排。我们引入LLMGenQC,一种基于大模型的批判性质量控制模块,对合成剧情进行语义一致性、连贯性、安全性与合理性检查,公开评分与通过/失败标签。消融实验表明:基于CCA的融合策略持续提升召回率;大模型重排在文本证据有限时显著改善top-K选择,提升nDCG。通过发布RAG-VisualRec,推动多模态RAG推荐、生成内容质量审计与长尾评估研究。所有代码、数据与文档均公开于:https://github.com/RecSys-lab/RAG-VisualRec。

原文摘要 · Abstract (English)

This paper addresses the challenge of building multimodal recommender systems for the movie domain, where sparse item metadata (e.g., title and genres) can limit retrieval quality and downstream recommendations. We introduce RAG-VisualRec, an open resource and reproducible pipeline that combines (i) LLM-generated item-side plot descriptions and (ii) trailer-derived visual (and optional audio) embeddings, supporting both retrieval-augmented generation (RAG) and collaborative-filtering style workflows. Our pipeline augments sparse metadata into richer textual signals and integrates modalities via configurable fusion strategies (e.g., PCA and CCA) before retrieval and optional LLM-based re-ranking. Beyond providing the resource, we provide a complementary analysis that increases transparency and reproducibility. In particular, we introduce LLMGenQC, a critic-based quality-control module (LLM-as-judge) that audits synthetic synopses for semantic alignment with metadata, consistency, safety, and basic sanity checks, releasing critic scores and pass/fail labels alongside the generated artifacts. We report ablation studies that quantify the impact of key design choices, including retrieval depth, fusion strategy, and user-embedding construction. Across experiments, CCA-based fusion consistently improves recall over unimodal baselines, while LLM-based re-ranking typically improves nDCG by refining top-K selection from the retrieved candidate pool, especially when textual evidence is limited. By releasing RAG-VisualRec, we enable further research on multimodal RAG recommenders, quality auditing of LLM-generated side information, and long-tail oriented evaluation protocols. All code, data, and detailed documentation are publicly available at: https://github.com/RecSys-lab/RAG-VisualRec.

多模态推荐RAG大模型生成质量评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。