优化推荐系统嵌入层查询，提速最高达6.5倍
Deep Recommender Models Inference: Automatic Asymmetric Data Flow Optimization
- 设计专用数据流，单核高效查询嵌入表
- 自动异构映射多核，实测提速1.5至6.5倍
- 对查询分布不敏感，适合真实复杂场景
深度推荐模型（DLRMs）推理是Meta数据中心中占比超过79%的核心AI任务。其性能瓶颈在于嵌入层，该层需频繁随机访问内存以检索不同大小的嵌入向量。本文提出定制化数据流设计，包括四种在单核上高效查询嵌入表的策略，以及一个框架，可自动将嵌入表异构映射到SoC多核。使用华为昇腾AI加速器评估，相比默认编译器提升1.5x至6.5x，极端不平衡负载下超过20x。此外，该方法对查询分布的依赖性远低于基线。
原文摘要 · Abstract (English)
Deep Recommender Models (DLRMs) inference is a fundamental AI workload accounting for more than 79% of the total AI workload in Meta's data centers. DLRMs' performance bottleneck is found in the embedding layers, which perform many random memory accesses to retrieve small embedding vectors from tables of various sizes. We propose the design of tailored data flows to speedup embedding look-ups. Namely, we propose four strategies to look up an embedding table effectively on one core, and a framework to automatically map the tables asymmetrically to the multiple cores of a SoC. We assess the effectiveness of our method using the Huawei Ascend AI accelerators, comparing it with the default Ascend compiler, and we perform high-level comparisons with Nvidia A100. Results show a speed-up varying from 1.5x up to 6.5x for real workload distributions, and more than 20x for extremely unbalanced distributions. Furthermore, the method proves to be much more independent of the query distribution than the baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。