arXiv:2602.00682cs.IRcs.AI2026-02被引 6

用最优传输对齐语言模型与推荐系统,提升多模态推荐效果

RecGOAT: Graph Optimal Adaptive Transport for LLM-Enhanced Multimodal Recommendation with Dual Semantic Alignment

  • 构建双粒度对齐框架,融合图神经网络与最优传输理论
  • 在三个公开数据集上超越现有方法,线上平台验证可扩展性
  • 适合关注多模态推荐、大模型融合的工业与研究者

将大语言模型(LLM)表示融入多模态推荐展现出潜力,但生成式语言模型表示与推荐系统依赖的基于ID的协同信号之间存在语义异质性。未经对齐地直接注入LLM特征反而会降低推荐性能。为此,我们提出RecGOAT,一种基于图神经网络和最优传输理论的双粒度语义对齐框架。RecGOAT首先通过多模态注意力图丰富协同语义,捕捉物品-物品、用户-物品、用户-用户关系,并利用LLM推断的行为偏好初始化用户表示。随后在两个互补粒度上对齐LLM生成的模态表示与推荐ID:(1) 实例级对齐通过跨模态对比学习(CMCL),生成具有判别力的样本级表示;(2) 分布级对齐通过最优自适应传输(OAT),最小化ID分布与LLM语义间的1-Wasserstein距离,构建统一且一致对齐的特征空间。理论上证明,统一表示的目标误差严格低于任一单模态表示,误差间隙由Wasserstein距离和InfoNCE损失界定,为对齐一致性与融合全面性提供严格保障。在三个公开基准上的大量实验表明其达到顶尖性能。在大规模在线广告平台上的部署进一步验证了RecGOAT的工业可扩展性。代码已开源:https://github.com/6lyc/RecGOAT-LLM4Rec。

原文摘要 · Abstract (English)

Integrating large language model (LLM) representations into multimodal recommendation has shown promise, yet a fundamental challenge remains largely overlooked: the semantic heterogeneity between generative LM representations and the ID-based collaborative signals that recommendation systems rely on. Naively injecting LM features without alignment degrades recommendation performance rather than improving it. To resolve this, we propose RecGOAT, a dual-granularity semantic alignment framework built on graph neural networks and optimal transport theory. RecGOAT first enriches collaborative semantics through multimodal attentive graphs that capture item-item, user-item, and user-user relationships, initializing user representations via LLM-inferred behavioral preferences. It then aligns LM-derived modality representations with recommendation IDs at two complementary granularities: (1) instance-level alignment via cross-modal contrastive learning (CMCL), which produces discriminative per-sample representations; and (2) distribution-level alignment via optimal adaptive transport (OAT), which minimizes the 1-Wasserstein distance between ID distributions and LLM semantics to produce a unified, consistently aligned feature space. Theoretically, we prove that the unified representation achieves strictly lower target error than any single-modality representation, with the gap bounded by the Wasserstein distance and the InfoNCE loss, providing rigorous guarantees for both alignment consistency and fusion comprehensiveness. Extensive experiments on three public benchmarks demonstrate state-of-the-art performance. Deployment on a large-scale online advertising platform further validates RecGOAT's industrial scalability. Our code is available at https://github.com/6lyc/RecGOAT-LLM4Rec.

多模态推荐大模型融合最优传输图神经网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。