arXiv:2506.11999cs.IRcs.CL2025-06被引 2

构建推荐领域通用模型,统一生成与嵌入任务。

Generative Representational Learning of Foundation Models for Recommendation

  • 设计多任务训练框架,解决知识冲突与收敛不一致问题。
  • 在多个推荐任务上超越现有基线,表现领先。
  • 提供首个涵盖生成与嵌入的综合性推荐数据集。

构建能够跨多种任务表现卓越的通用模型是人工智能领域的长期目标。随着通用基础模型浪潮席卷各领域,其影响力已延伸至推荐系统。尽管近期研究探索了推荐基础模型在各类生成任务中的应用,但常忽视关键的嵌入任务,且在多任务学习中面临知识共享与冲突、收敛速度不一致等挑战。为此,我们提出 RecFound,一种面向推荐基础模型的生成表征学习框架。我们构建了首个覆盖多种场景下生成与嵌入任务的综合性推荐基础模型数据集。基于该数据集,我们提出一种新型多任务训练方案:采用任务级低秩专家混合(TMoLE)处理知识共享与冲突,引入分步收敛采样调度器(S2Sched)缓解收敛不一致问题,并设计模型合并模块平衡各任务性能。实验表明,RecFound 在多种推荐任务中均达到当前最优性能,显著优于现有基线。

原文摘要 · Abstract (English)

Developing a single foundation model with the capability to excel across diverse tasks has been a long-standing objective in the field of artificial intelligence. As the wave of general-purpose foundation models sweeps across various domains, their influence has significantly extended to the field of recommendation systems. While recent efforts have explored recommendation foundation models for various generative tasks, they often overlook crucial embedding tasks and struggle with the complexities of multi-task learning, including knowledge sharing & conflict resolution, and convergence speed inconsistencies. To address these limitations, we introduce RecFound, a generative representational learning framework for recommendation foundation models. We construct the first comprehensive dataset for recommendation foundation models covering both generative and embedding tasks across diverse scenarios. Based on this dataset, we propose a novel multi-task training scheme featuring a Task-wise Mixture of Low-rank Experts (TMoLE) to handle knowledge sharing & conflict, a Step-wise Convergence-oriented Sample Scheduler (S2Sched) to address inconsistent convergence, and a Model Merge module to balance the performance across tasks. Experiments demonstrate that RecFound achieves state-of-the-art performance across various recommendation tasks, outperforming existing baselines.

推荐系统基础模型多任务学习生成模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。