用Transformer重构推荐系统，大幅减少人工特征并提升排序效果。
From Features to Transformers: Redefining Ranking for Scalable Impact
- 改进Transformer结构，融合学习型归一化与用户历史与物品的联合注意力。
- 仅用少量特征超越旧系统，模型越大、数据越多、上下文越长效果越好。
- 支持批量联合打分，自动提升推荐多样性，适合大规模工业场景。
我们提出LiGR，LinkedIn开发的大规模排序框架，将先进的基于Transformer的建模架构引入生产环境。通过引入一种改进的Transformer架构，结合学习型归一化和对用户历史与候选物品的联合集合注意力，实现多项突破：（1）几乎摒弃了传统手工特征工程，仅用少数特征即超越此前最先进的系统（原系统使用数百个特征）；（2）验证了排序系统的缩放定律，表明模型越大、训练数据越多、上下文序列越长，性能越优；（3）支持集合式联合打分，自动提升推荐多样性。为实现大模型高效推理，我们提出单次遍历用户历史与集合注意力的高效服务技术。同时总结了多种消融实验与A/B测试的关键发现，明确了最具影响力的几项技术路径。
原文摘要 · Abstract (English)
We present LiGR, a large-scale ranking framework developed at LinkedIn that brings state-of-the-art transformer-based modeling architectures into production. We introduce a modified transformer architecture that incorporates learned normalization and simultaneous set-wise attention to user history and ranked items. This architecture enables several breakthrough achievements, including: (1) the deprecation of most manually designed feature engineering, outperforming the prior state-of-the-art system using only few features (compared to hundreds in the baseline), (2) validation of the scaling law for ranking systems, showing improved performance with larger models, more training data, and longer context sequences, and (3) simultaneous joint scoring of items in a set-wise manner, leading to automated improvements in diversity. To enable efficient serving of large ranking models, we describe techniques to scale inference effectively using single-pass processing of user history and set-wise attention. We also summarize key insights from various ablation studies and A/B tests, highlighting the most impactful technical approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。