用大模型增强电影推荐，融合音视频文本多模态信息
ViLLA-MMBench: A Unified Benchmark Suite for LLM-Augmented Multimodal Movie Recommendation
- 用LLM自动补全缺失电影信息，生成高质量剧情摘要
- 支持多种融合方式与推荐模型，验证文本特征对冷启动提升显著
- 适合做多模态推荐与生成式AI融合研究的团队使用
长视频内容推荐需要联合建模视觉、音频和文本模态，但现有基准大多仅关注原始特征或局限的融合方式。我们提出ViLLA-MMBench，一个可复现、可扩展的统一基准，用于大语言模型增强的多模态电影推荐。基于MovieLens和MMTF-14K构建，对三类模态（音频：块级i-vector；视觉：CNN, AVF；文本）进行密集项嵌入对齐。利用先进LLM（如OpenAI Ada）自动丰富缺失或稀疏元数据，为数千部电影生成高质量剧情摘要。所有文本（原始或增强）通过可配置编码器（Ada, LLaMA-2, Sentence-T5）嵌入，生成多个即用型数据集。管道支持早期、中期、晚期融合（拼接、PCA、CCA、秩聚合），并兼容多种推荐模型（MF, VAECF, VBPR, AMR, VMF）用于消融实验。评估涵盖准确率（Recall, nDCG）及冷启动率、覆盖率、新颖性、多样性、公平性等超越准确率指标。结果表明，基于LLM的增强与强文本嵌入显著提升冷启动与覆盖率，尤其在与音视频特征融合时表现更优。系统性基准测试揭示了通用与模型/指标特异性组合。代码、嵌入与配置均已开源，支持可复现、公平的多模态推荐研究，并推动大规模推荐中生成式AI的规范集成。
原文摘要 · Abstract (English)
Recommending long-form video content demands joint modeling of visual, audio, and textual modalities, yet most benchmarks address only raw features or narrow fusion. We present ViLLA-MMBench, a reproducible, extensible benchmark for LLM-augmented multimodal movie recommendation. Built on MovieLens and MMTF-14K, it aligns dense item embeddings from three modalities: audio (block-level, i-vector), visual (CNN, AVF), and text. Missing or sparse metadata is automatically enriched using state-of-the-art LLMs (e.g., OpenAI Ada), generating high-quality synopses for thousands of movies. All text (raw or augmented) is embedded with configurable encoders (Ada, LLaMA-2, Sentence-T5), producing multiple ready-to-use sets. The pipeline supports interchangeable early-, mid-, and late-fusion (concatenation, PCA, CCA, rank-aggregation) and multiple backbones (MF, VAECF, VBPR, AMR, VMF) for ablation. Experiments are fully declarative via a single YAML file. Evaluation spans accuracy (Recall, nDCG) and beyond-accuracy metrics: cold-start rate, coverage, novelty, diversity, fairness. Results show LLM-based augmentation and strong text embeddings boost cold-start and coverage, especially when fused with audio-visual features. Systematic benchmarking reveals universal versus backbone- or metric-specific combinations. Open-source code, embeddings, and configs enable reproducible, fair multimodal RS research and advance principled generative AI integration in large-scale recommendation. Code: https://recsys-lab.github.io/ViLLA-MMBench
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。