arXiv:2603.03988cs.IR2026-03被引 1

SORT让Transformer在电商推荐中高效落地,性能提升显著且延迟大幅降低。

SORT: A Systematically Optimized Ranking Transformer for Industrial-scale Recommenders

  • 采用请求中心采样与局部注意力,缓解特征稀疏与标签稀疏问题。
  • 线上测试显示订单、买家、GMV分别提升7.47%、6.67%、8.65%,延迟下降62%。
  • 适合大规模工业级推荐系统,尤其对低密度标签场景有强适配性。

尽管Transformer在大语言模型中表现出色,但在工业级排序模型中的应用仍受限于高特征稀疏性和低标签密度。本文提出SORT(Systematically Optimized Ranking Transformer),通过请求中心采样、局部注意力、查询剪枝和生成式预训练等优化,有效应对上述挑战。同时,改进分词、多头注意力和前馈网络模块,稳定训练并扩大模型容量。训练系统优化使模型每秒浮点运算利用率(MFU)达45%。大量实验表明,SORT在数据量、模型规模和序列长度上均具优异可扩展性,且能灵活融合多种特征。阿里速卖通线上A/B测试验证:订单增长7.47%,买家数上升6.67%,商品交易总额(GMV)提升8.65%,延迟降低62%,吞吐量提高589%。SORT已全面上线生产环境,服务所有用户。

原文摘要 · Abstract (English)

While Transformers have achieved remarkable success in LLMs through superior scalability, their application in industrial-scale ranking models remains nascent, hindered by the challenges of high feature sparsity and low label density. In this paper, we propose SORT (Systematically Optimized Ranking Transformer), a scalable model designed to bridge the gap between Transformers and industrial-scale ranking models. We address the high feature sparsity and low label density challenges through a series of optimizations, including request-centric sample organization, local attention, query pruning, and generative pre-training. Furthermore, we introduce a suite of refinements to the tokenization, multi-head attention (MHA), and feed-forward network (FFN) modules, which collectively stabilize the training process and enlarge the model capacity. To maximize hardware efficiency, we optimize our training system to elevate the model FLOPs utilization (MFU) to 45%. Extensive experiments demonstrate that SORT outperforms strong baselines and exhibits excellent scalability across data size, model size, and sequence length, while remaining flexible at integrating diverse features. Finally, online A/B testing in large-scale e-commerce scenarios confirms that SORT achieves significant gains in key business metrics, including orders (+7.47%), buyers (+6.67%), and GMV (+8.65%), while simultaneously cutting latency by 62% and boosting throughput nearly sevenfold (+589%). SORT has been fully deployed in production, serving all users on AliExpress.

推荐系统Transformer工业级排序优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。