用用户请求代替曝光记录训练推荐模型,省存储、提效率、强效果。
Request-Only Optimization for Recommendation Systems
- 以用户请求为单位训练,天然去重特征数据
- 减少重复计算通信,支持更大规模神经网络
- 适合生成式推荐等新架构,工业级部署友好
深度学习推荐模型(DLRMs)是全球规模最大的机器学习应用之一。行业级DLRMs基于海量推荐数据训练,每日服务数十亿用户。为利用用户长期历史中的丰富信号,现有模型复杂度已达到每样本万亿次浮点运算(TFLOPs)。这一规模与海量训练数据要求新的存储与训练算法,以高效提升系统质量。本文提出请求仅优化(ROO)训练与建模范式,同时提升存储效率、训练效率与模型性能。通过协同设计数据(请求仅数据)、基础设施(基于请求的数据处理流水线)与模型架构(请求仅神经网络),将用户请求作为训练数据单元,取代传统以用户曝光为单位的方式。该设计在数据日志中实现原生特征去重,显著节省存储空间;通过消除同一请求内多个曝光间的重复计算与通信,使大型神经网络(如生成式推荐器)能更有效地捕捉用户兴趣信号,从而支持更高复杂度模型的训练与部署。
原文摘要 · Abstract (English)
Deep Learning Recommendation Models (DLRMs) represent one of the largest machine learning applications on the planet. Industry-scale DLRMs are trained with petabytes of recommendation data to serve billions of users every day. To utilize the rich user signals in the long user history, DLRMs have been scaled up to unprecedented complexity, up to trillions of floating-point operations (TFLOPs) per example. This scale, coupled with the huge amount of training data, necessitates new storage and training algorithms to efficiently improve the quality of these complex recommendation systems. In this paper, we present a Request-Only Optimizations (ROO) training and modeling paradigm. ROO simultaneously improves the storage and training efficiency as well as the model quality of recommendation systems. We holistically approach this challenge through co-designing data (i.e., request-only data), infrastructure (i.e., request-only based data processing pipeline), and model architecture (i.e., request-only neural architectures). Our ROO training and modeling paradigm treats a user request as a unit of the training data. Compared with the established practice of treating a user impression as a unit, our new design achieves native feature deduplication in data logging, consequently saving data storage. Second, by de-duplicating computations and communications across multiple impressions in a request, this new paradigm enables highly scaled-up neural network architectures to better capture user interest signals, such as Generative Recommenders (GRs) and other request-only friendly architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。