SCRec通过压缩与硬件协同,让推荐模型在单服务器上实现超高速推理。
SCRec: A Scalable Computational Storage System with Statistical Sharding and Tensor-train Decomposition for Recommendation Models
- 用统计分片和张量分解压缩模型,减少数据量
- 单机实现55.77倍加速,精度零损失,能效比多卡系统高13.35倍
- 适合需要低延迟、高能效的工业级推荐系统部署
深度学习推荐模型(DLRMs)在社交网络与视频流等场景中至关重要。然而,随着性能提升,其参数规模已达太字节(TB),内存带宽需求超过太字节/秒(TB/s)。同时,模型内工作负载分布不均,难以优化。本文提出SCRec,一种可扩展的计算存储推荐系统,支持TB级工业级DLRMs,保障高带宽需求。SCRec采用基于混合整数规划(MIP)的成本模型,依据数据访问模式高效调度,并自适应配置内存中心与计算中心核心。系统集成硬件加速单元,高效重构极压缩的张量-列车(TT)格式嵌入向量。通过软硬件协同,在单服务器上消除通信开销,显著提升推理性能:相比CPU-DRAM系统,最高提速55.77倍且精度无损;相比多GPU系统,能效提升最高达13.35倍。
原文摘要 · Abstract (English)
Deep Learning Recommendation Models (DLRMs) play a crucial role in delivering personalized content across web applications such as social networking and video streaming. However, with improvements in performance, the parameter size of DLRMs has grown to terabyte (TB) scales, accompanied by memory bandwidth demands exceeding TB/s levels. Furthermore, the workload intensity within the model varies based on the target mechanism, making it difficult to build an optimized recommendation system. In this paper, we propose SCRec, a scalable computational storage recommendation system that can handle TB-scale industrial DLRMs while guaranteeing high bandwidth requirements. SCRec utilizes a software framework that features a mixed-integer programming (MIP)-based cost model, efficiently fetching data based on data access patterns and adaptively configuring memory-centric and compute-centric cores. Additionally, SCRec integrates hardware acceleration cores to enhance DLRM computations, particularly allowing for the high-performance reconstruction of approximated embedding vectors from extremely compressed tensor-train (TT) format. By combining its software framework and hardware accelerators, while eliminating data communication overhead by being implemented on a single server, SCRec achieves substantial improvements in DLRM inference performance. It delivers up to 55.77$\times$ speedup compared to a CPU-DRAM system with no loss in accuracy and up to 13.35$\times$ energy efficiency gains over a multi-GPU system.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。