用三阶段方法把原始多模态特征变成推荐好用的嵌入表示。
From Raw Features to Effective Embeddings: A Three-Stage Approach for Multimodal Recipe Recommendation
- 先用大模型理解图文内容，再通过用户-菜谱互动传播关系，最后用对比学习优化嵌入。
- 在两个真实数据集上，召回率比现有方法高7%到15%。
- 适合做食物推荐系统、多模态嵌入学习的研究者和工程师。
食谱推荐已成为网络食品平台的核心任务。核心挑战在于有效利用超越用户-食谱交互的丰富多模态特征。我们的分析表明，即使简单使用多模态信号也能达到有竞争力的性能，提示系统性增强这些信号具有巨大潜力。我们提出TESMR，一个三阶段框架，通过：(1) 使用具备多模态理解能力的基础模型进行基于内容的增强；(2) 通过用户-食谱交互图上的消息传播实现基于关系的增强；(3) 通过可学习嵌入的对比学习进行基于学习的增强，逐步将原始多模态特征转化为有效嵌入。在两个真实数据集上的实验显示，TESMR优于现有方法，在Recall@10指标上提升7%-15%。
原文摘要 · Abstract (English)
Recipe recommendation has become an essential task in web-based food platforms. A central challenge is effectively leveraging rich multimodal features beyond user-recipe interactions. Our analysis shows that even simple uses of multimodal signals yield competitive performance, suggesting that systematic enhancement of these signals is highly promising. We propose TESMR, a 3-stage framework for recipe recommendation that progressively refines raw multimodal features into effective embeddings through: (1) content-based enhancement using foundation models with multimodal comprehension, (2) relation-based enhancement via message propagation over user-recipe interactions, and (3) learning-based enhancement through contrastive learning with learnable embeddings. Experiments on two real-world datasets show that TESMR outperforms existing methods, achieving 7-15% higher Recall@10.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。