拆分推荐系统嵌入计算，降低网络开销提升效率
Disaggregating Embedding Recommendation Systems with FlexEMR
- 利用嵌入查询的时间空间局部性减少网络传输量
- 设计多线程RDMA引擎并行处理查询子请求
- 适合大规模推荐系统部署与资源优化场景
基于嵌入的推荐系统（EMR）因内存需求持续增长，高效服务成为挑战。当前做法将模型拆分部署在多个独立服务器上，固定配置GPU、CPU和DRAM，导致资源利用率低且成本高。将嵌入操作与神经网络推理解耦是一种有前景的解决方案，但带来新的网络挑战。本文提出FlexEMR以优化EMR解耦设计：一是利用嵌入查询的时间与空间局部性，减少网络数据移动；二是设计优化的多线程RDMA引擎，支持并发查询子请求。文章阐述了每项技术的设计空间，并展示了早期原型的初步结果。
原文摘要 · Abstract (English)
Efficiently serving embedding-based recommendation (EMR) models remains a significant challenge due to their increasingly large memory requirements. Today's practice splits the model across many monolithic servers, where a mix of GPUs, CPUs, and DRAM is provisioned in fixed proportions. This approach leads to suboptimal resource utilization and increased costs. Disaggregating embedding operations from neural network inference is a promising solution but raises novel networking challenges. In this paper, we discuss the design of FlexEMR for optimized EMR disaggregation. FlexEMR proposes two sets of techniques to tackle the networking challenges: Leveraging the temporal and spatial locality of embedding lookups to reduce data movement over the network, and designing an optimized multi-threaded RDMA engine for concurrent lookup subrequests. We outline the design space for each technique and present initial results from our early prototype.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。