用大模型融合视觉文本特征，提升广告记忆度预测效果
LLM-based Fusion of Multi-modal Features for Commercial Memorability Prediction
- 以Gemma-3大模型为骨干，通过多模态投影融合预计算的视觉与文本特征
- 在MediaEval 2025测试集上表现优于梯度提升树基线，泛化能力更强
- 引入专家定义的记忆点提示词，增强模型可解释性，适合广告评估场景
本文针对MediaEval 2025工作坊竞赛中‘广告记忆度预测’任务的子任务2，提出一种基于Gemma-3大模型的多模态特征融合系统。该系统通过多模态投影整合预先计算的视觉（ViT）与文本（E5）特征，并采用低秩适应（LoRA）进行微调。为对比，构建了一个经过大量调优的梯度提升树集成基线。关键贡献在于利用专家定义的记忆度维度生成大模型推理提示，指导融合过程。实验表明，该系统在最终测试集上展现出更强的鲁棒性与泛化性能。代码已开源：https://github.com/dsgt-arc/mediaeval-2025-memorability。
原文摘要 · Abstract (English)
This paper addresses the prediction of commercial (brand) memorability as part of "Subtask 2: Commercial/Ad Memorability" within the "Memorability: Predicting movie and commercial memorability" task at the MediaEval 2025 workshop competition. We propose a multimodal fusion system with a Gemma-3 LLM backbone that integrates pre-computed visual (ViT) and textual (E5) features by multi-modal projections. The model is adapted using Low-Rank Adaptation (LoRA). A heavily-tuned ensemble of gradient boosted trees serves as a baseline. A key contribution is the use of LLM-generated rationale prompts, grounded in expert-derived aspects of memorability, to guide the fusion model. The results demonstrate that the LLM-based system exhibits greater robustness and generalization performance on the final test set, compared to the baseline. The paper's codebase can be found at https://github.com/dsgt-arc/mediaeval-2025-memorability
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。