Ember编译器让去耦访问执行架构高效运行嵌入查找，性能比GPU高2.6倍。
Ember: A Compiler for Efficient Embedding Operations on Decoupled Access-Execute Architectures
- 为去耦架构设计多级中间表示，支持深度优化
- 在端到端模型上实现6.4倍于GPU的能效比
- 自动将PyTorch/TensorFlow代码转为高效DAE指令，媲美手写代码
不规则嵌入查找是推荐系统、稀疏大语言模型和图学习模型的关键瓶颈。本文首次证明，通过将这些查找操作卸载至专用访问单元，去耦访问-执行（Decoupled Access-Execute, DAE）处理器在端到端模型上相比GPU实现2.6倍的性能提升和6.4倍的性能/瓦特能效比。为此，我们提出Ember编译器，可自动将PyTorch和TensorFlow代码转换为优化的DAE代码。与现有DAE编译器不同，Ember采用针对不同优化层级设计的多级中间表示，从而实现所有优化技术，性能达到与手写代码相当的水平，充分释放了DAE架构在大规模应用中的潜力。
原文摘要 · Abstract (English)
Irregular embedding lookups are a critical bottleneck in recommender models, sparse large language models, and graph learning models. In this paper, we first demonstrate that, by offloading these lookups to specialized access units, Decoupled Access-Execute (DAE) processors achieve 2.6$\times$ higher performance and 6.4$\times$ higher performance/watt than GPUs on end-to-end models. Then, we propose the Ember compiler for automatically generating optimized DAE code from PyTorch and TensorFlow. Conversely from other DAE compilers, Ember features multiple intermediate representations specifically designed for different optimization levels. In this way, Ember can implement all optimizations to match the performance of hand-written code, unlocking the full potential of DAE architectures at scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。