arXiv:2501.14268cs.IRcs.AI2025-01中稿 · WWW2025被引 2

将推荐系统视为大模型,通过自适应微调提升兴趣变化捕捉能力

Pre-train and Fine-tune: Recommenders as Large Models

  • 把推荐系统当作大模型,分知识压缩与匹配两阶段微调
  • 在百亿级美食平台上线后显著提升业务收益
  • 方法可解释性强,适合工业级推荐系统迭代

用户兴趣随时间、区域、场景等变化剧烈,传统推荐系统难以捕捉。现有跨域学习虽能缓解此问题,但工业级推荐系统结构复杂、数据量大、训练成本高,难以重构和重训。为此,本文将推荐系统视为大规模预训练模型,提出基于信息瓶颈理论的微调框架,并设计信息感知自适应核(IAK)技术。该方法将微调分为知识压缩与知识匹配两个阶段,使训练过程显式逼近这两个目标。实验表明,该方法在离线与在线测试中均表现优异。所提方法已在百亿级在线食品平台首页部署数月,带来显著商业收益。同时,论文总结了实际部署中的关键经验,揭示了微调在推荐系统中的潜在问题及应对方案。

原文摘要 · Abstract (English)

In reality, users have different interests in different periods, regions, scenes, etc. Such changes in interest are so drastic that they are difficult to be captured by recommenders. Existing multi-domain learning can alleviate this problem. However, the structure of the industrial recommendation system is complex, the amount of data is huge, and the training cost is extremely high, so it is difficult to modify the structure of the industrial recommender and re-train it. To fill this gap, we consider recommenders as large pre-trained models and fine-tune them. We first propose the theory of the information bottleneck for fine-tuning and present an explanation for the fine-tuning technique in recommenders. To tailor for recommendation, we design an information-aware adaptive kernel (IAK) technique to fine-tune the pre-trained recommender. Specifically, we define fine-tuning as two phases: knowledge compression and knowledge matching and let the training stage of IAK explicitly approximate these two phases. Our proposed approach designed from the essence of fine-tuning is well interpretable. Extensive online and offline experiments show the superiority of our proposed method. Besides, we also share unique and important lessons we learned when deploying the method in a large-scale online platform. We also present the potential issues of fine-tuning techniques in recommendation systems and the corresponding solutions. The recommender with IAK technique has been deployed on the homepage of a billion-scale online food platform for several months and has yielded considerable profits in our business.

推荐系统大模型微调工业部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。