发现推荐系统序列建模有类似大模型的缩放规律，提升转化率4.3%。
LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation
- 采用两级架构，将长序列计算异步前置到用户模型中
- 引入语义特征使模型能有效利用更深更长的结构，实现4.3%转化率提升
- 适用于高并发、低延迟场景，适合工业级推荐系统部署
我们提出LLaTTE（基于大模型风格的时序事件潜在变换器），一种可扩展的Transformer架构，用于生产环境中的广告推荐。通过系统性实验，我们发现推荐系统的序列建模遵循类似大语言模型的幂律缩放规律。关键发现是：语义特征能改变缩放曲线——它们是实现有效扩展的前提，使模型能充分利用更深、更长架构的容量。为在严格延迟约束下持续受益于缩放，我们引入两阶段架构，将大型长上下文模型的重计算任务异步迁移至上游用户模型。我们证明上游优化可预测地传递至下游排序任务。该多阶段框架已在Meta部署为最大用户模型，在Facebook Feed和Reels上带来4.3%的转化率提升，且服务开销极小，为工业级推荐系统应用缩放规律提供了实用范式。
原文摘要 · Abstract (English)
We present LLaTTE (LLM-Style Latent Transformers for Temporal Events), a scalable transformer architecture for production ads recommendation. Through systematic experiments, we demonstrate that sequence modeling in recommendation systems follows predictable power-law scaling similar to LLMs. Crucially, we find that semantic features bend the scaling curve: they are a prerequisite for scaling, enabling the model to effectively utilize the capacity of deeper and longer architectures. To realize the benefits of continued scaling under strict latency constraints, we introduce a two-stage architecture that offloads the heavy computation of large, long-context models to an asynchronous upstream user model. We demonstrate that upstream improvements transfer predictably to downstream ranking tasks. Deployed as the largest user model at Meta, this multi-stage framework drives a 4.3\% conversion uplift on Facebook Feed and Reels with minimal serving overhead, establishing a practical blueprint for harnessing scaling laws in industrial recommender systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。