用浮点数特性快速近似指数函数，大幅加速模型推理。
QuAKE: Speeding up Model Inference Using Quick and Approximate Kernels for Exponential Non-Linearities
- 利用浮点数表示特性设计近似指数运算，无需额外硬件
- 在多种模型上实现10%-45%的推理提速，性能几乎无损
- 适合部署在服务器及移动设备的CPU上，尤其看重效率的场景
随着机器学习应用日益广泛，模型规模持续增大，提升推理阶段的计算效率成为关键挑战。在众多常用模型架构（如Transformer）中，大量计算消耗来自Softmax等指数型非线性函数。本文提出QuAKE，一组基于IEEE-754浮点数表示特性的新型算子，可在不依赖专用硬件、额外内存或预计算的前提下，快速近似指数函数。针对Softmax、GELU和Logistic等常见非线性函数，我们设计了优化策略以提升效率。基准测试显示，在各类模型架构与规模下，服务器级CPU推理速度提升10%-35%，嵌入式及移动端CPU提升5%-45%。在多个领域标准数据集上的评估表明，使用QuAKE算子可带来显著提速，同时在下游任务中几乎无性能损失。
原文摘要 · Abstract (English)
As machine learning gets deployed more and more widely, and model sizes continue to grow, improving computational efficiency during model inference has become a key challenge. In many commonly used model architectures, including Transformers, a significant portion of the inference computation is comprised of exponential non-linearities such as Softmax. In this work, we develop QuAKE, a collection of novel operators that leverage certain properties of IEEE-754 floating point representations to quickly approximate the exponential function without requiring specialized hardware, extra memory, or precomputation. We propose optimizations that enhance the efficiency of QuAKE in commonly used exponential non-linearities such as Softmax, GELU, and the Logistic function. Our benchmarks demonstrate substantial inference speed improvements between 10% and 35% on server CPUs, and 5% and 45% on embedded and mobile-scale CPUs for a variety of model architectures and sizes. Evaluations of model performance on standard datasets and tasks from various domains show that QuAKE operators are able to provide sizable speed benefits with little to no loss of performance on downstream tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。