用大模型统一解决推荐系统中的多任务问题,省去特征工程和模型维护。
360Brew: A Decoder-only Foundation Model for Personalized Ranking and Recommendation
- 用文本接口的解码器模型统一处理多个推荐任务
- 1500亿参数模型在30+任务上表现媲美或超过现有系统
- 无需特征工程,可快速适配新场景和跨域问题
排名与推荐系统是众多在线体验的基础,从搜索结果到个性化内容分发。这些系统已发展为复杂的多层架构,依赖海量数据并常集成数千个预测模型。模型的维护与优化过程繁琐,需大量特征工程,不仅加剧技术债务,也阻碍向新兴领域拓展。本文提出使用具有文本接口的大规模基础模型来应对这一挑战。我们的方法具备三大优势:(1)单一模型可管理排名与推荐中涉及的多种预测任务;(2)基于文本接口的解码器模型凭借其推理理解能力,能泛化至新推荐界面及域外问题;(3)通过自然语言定义任务并用自然语言表达用户行为与社交关系,彻底消除特征工程与复杂模型依赖图的维护需求。我们推出了预生产模型360Brew V1.0,一个1500亿参数、仅解码器结构的模型,基于LinkedIn数据与任务训练微调。该模型可在LinkedIn平台多个业务线中完成超过30项预测任务,在离线指标上表现媲美或超越当前生产系统,且无需针对具体任务进行微调。而这些任务传统上均由独立模型长期开发维护,团队规模与我们相当或更大。
原文摘要 · Abstract (English)
Ranking and recommendation systems are the foundation for numerous online experiences, ranging from search results to personalized content delivery. These systems have evolved into complex, multilayered architectures that leverage vast datasets and often incorporate thousands of predictive models. The maintenance and enhancement of these models is a labor intensive process that requires extensive feature engineering. This approach not only exacerbates technical debt but also hampers innovation in extending these systems to emerging problem domains. In this report, we present our research to address these challenges by utilizing a large foundation model with a textual interface for ranking and recommendation tasks. We illustrate several key advantages of our approach: (1) a single model can manage multiple predictive tasks involved in ranking and recommendation, (2) decoder models with textual interface due to their comprehension of reasoning capabilities, can generalize to new recommendation surfaces and out-of-domain problems, and (3) by employing natural language interfaces for task definitions and verbalizing member behaviors and their social connections, we eliminate the need for feature engineering and the maintenance of complex directed acyclic graphs of model dependencies. We introduce our research pre-production model, 360Brew V1.0, a 150B parameter, decoder-only model that has been trained and fine-tuned on LinkedIn's data and tasks. This model is capable of solving over 30 predictive tasks across various segments of the LinkedIn platform, achieving performance levels comparable to or exceeding those of current production systems based on offline metrics, without task-specific fine-tuning. Notably, each of these tasks is conventionally addressed by dedicated models that have been developed and maintained over multiple years by teams of a similar or larger size than our own.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。