通过延迟计算用户与候选的交互,提升推荐系统推理效率。
ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation

- 按请求共享特征,仅对每个请求计算一次模型部分。
- 在公开数据集上实现最高3倍QPS提升,且精度不降。
- 适合大规模推荐系统,尤其关注成本与性能平衡的场景。
现代推荐模型通过扩大特征交互和序列模块来提升预测质量,但生产成本限制了其扩展。本文提出请求导向计算共享(ROCS),利用推荐推理中请求侧特征在多个候选间共享的特点,将请求-候选交互尽可能延后,隔离候选相关表示,并对每个请求仅计算模型中大量部分一次而非每个候选一次,显著提升推理效率并保持或提升预测质量。为此,我们设计广义层掩码(GLM)以在特征交互架构中实现候选隔离,以及深度交叉注意力(DCA)将请求导向共享扩展至序列架构。为支持高效GPU部署,我们协同设计了核内广播优化(IKBO),大幅加速ROCS模型执行。在公开基准测试中,ROCS在各类推荐骨干网络上持续改善质量-效率权衡。在生产规模负载下,检索模型最高实现3倍QPS提升且无质量下降,短视频排序模型在0.5%相对LogLoss改进的同时获得50% QPS增长。ROCS已部署于覆盖广告与自然内容、检索与排序阶段的大规模推荐系统,处理超过两个数量级的推理复杂度,带来显著线上收益并降低基础设施成本。
原文摘要 · Abstract (English)
Modern recommendation models gain prediction quality by scaling feature-interaction and sequence modules, but production cost constraints cap how far systems can scale. In this work, we propose Request-Oriented Compute Sharing (ROCS), a modeling and inference paradigm that exploits a unique property of recommendation inference: each user request is evaluated against many candidates, while request-side features are shared across candidates. ROCS defers request-candidate interactions as late as possible, isolates candidate-dependent representations, and evaluates substantial portions of the model once per request rather than once per candidate, significantly improving inference efficiency while maintaining or improving prediction quality. To realize this paradigm, we develop Generalized Layer Masking (GLM) to enforce candidate isolation in feature-interaction architectures, and Deep Cross Attention (DCA) to extend request-oriented sharing to sequence architectures. To support efficient GPU deployment, we co-design In-Kernel Broadcast Optimization (IKBO) that significantly accelerates ROCS model execution. Experiments on public benchmarks show that ROCS consistently improves the quality-efficiency tradeoff across recommendation backbones. On production-scale workloads, ROCS achieves up to a 3x QPS improvement on retrieval models without quality degradation and a 0.5% relative LogLoss improvement with a 50% QPS gain on a short-form video ranking model. ROCS has been deployed across large-scale recommendation systems spanning ads and organic surfaces, retrieval and ranking stages, and more than two orders of magnitude in inference complexity, delivering significant online gains at reduced infrastructure cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。