arXiv:2511.14881cs.IR2025-11被引 3

用统一模型在GPU上实现高效推荐,提升速度与精度。

SilverTorch: A Unified Model-based System to Democratize Large-Scale Recommendation on GPUs

  • 将索引过滤和搜索整合为模型层,全程基于GPU
  • 吞吐量达现有方案23.7倍,成本降低13.35倍
  • 支持复杂模型如多任务检索,适合工业级部署

在大规模场景下服务深度学习推荐模型(DLRM)面临挑战。现有方法依赖CPU上的专用近似最近邻(ANN)索引与过滤服务,导致开销大且缺乏软硬件协同设计机会。为此,本文提出SilverTorch,一种基于模型的统一推理系统,将所有组件集成于单一模型中。通过引入基于GPU的模型化布隆过滤器进行特征筛选,并融合Int8 ANN内核实现最近邻搜索,实现了计算与显存的双重优化。通过联合设计,消除冗余计算并减少显存占用。进一步引入过架构评分层与价值模型,支持多任务检索与分数聚合,提升召回精度并支撑更复杂模型。在真实工业级数据集上的评估表明,SilverTorch相较当前最优方案提升23.7倍吞吐量;相比基于CPU的方案,成本降低13.35倍,同时精度更高。该系统已在实际生产环境大规模部署,服务数百个模型,支持多样化推荐场景。

原文摘要 · Abstract (English)

Serving deep learning based recommendation models (DLRM) at scale is challenging. Existing approaches rely on dedicated ANN indexing and filtering services on CPUs, suffering from non-negligible costs and missing co-design opportunities. Such inefficiency makes them difficult to support complex model architectures, such as learned similarities and multi-task retrieval. In this paper, we present SilverTorch, a model-based serving system that brings all components into one unified model. It unifies model serving by replacing standalone indexing and filtering services with model layers. We propose a model-based GPU Bloom index for feature filtering and a fused Int8 ANN kernel for nearest neighbor search. Through co-design of the ANN search and feature filtering, we reduce GPU memory usage and eliminate computation. Benefiting from this design, we scale up retrieval by introducing an OverArch scoring layer and a multi-task retrieval with a Value Model to aggregate scores. These advancements improve the retrieval accuracy and enable future studies for serving more complex models. Our evaluation on industry-scale datasets show that SilverTorch achieves up to 23.7\times higher throughput compared to the state-of-the-art approaches. We also demonstrate that SilverTorch solution is 13.35\times more cost-efficient than CPU-based solution while improving accuracy via serving more complex models. SilverTorch is deployed at scale, serving hundreds of models online and supporting recommendation for diverse applications.

推荐系统GPU加速模型融合工业部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。