用标量矩阵乘法替代传统卷积，减少内存开销并加速推理。
SMM-Conv: Scalar Matrix Multiplication with Zero Packing for Accelerated Convolution
- 提出新方法：用标量矩阵乘代替im2col+GEMM的卷积计算
- 在常见网络上实现显著提速，相比现有间接方法
- 适合需要低内存占用的CPU推理场景
我们提出一种针对基于CPU架构的推理加速卷积的新方法。主流方法是将图像通过im2col转换为矩阵列，再与权重矩阵进行通用矩阵乘法（GEMM）。但该方法存在两大缺陷：(a) im2col需大内存缓冲且访问效率低；(b) GEMM虽优化于科学计算，却不适配卷积特性。本文提出利用标量矩阵乘法的新策略，有效降低内存开销。实验表明，所提方法在常用网络架构上相比现有间接方法实现了显著加速。
原文摘要 · Abstract (English)
We present a novel approach for accelerating convolutions during inference for CPU-based architectures. The most common method of computation involves packing the image into the columns of a matrix (im2col) and performing general matrix multiplication (GEMM) with a matrix of weights. This results in two main drawbacks: (a) im2col requires a large memory buffer and can experience inefficient memory access, and (b) while GEMM is highly optimized for scientific matrices multiplications, it is not well suited for convolutions. We propose an approach that takes advantage of scalar-matrix multiplication and reduces memory overhead. Our experiments with commonly used network architectures demonstrate a significant speedup compared to existing indirect methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。