arXiv:2604.04599cs.DCcs.CV2026-04

通过布局传播优化连续矩阵乘法,提升科学计算与机器学习性能。

LP-GEMM: Integrating Layout Propagation into GEMM Operations

  • 将GEMM分解为可传递数据布局的结构,避免重复打包。
  • 在x86和RISC-V上实现平均2.25倍加速,超越OpenBLAS。
  • 适用于需要高性能矩阵运算的推理系统开发。

在科学计算和现代机器学习工作负载中,依赖的连续矩阵乘法(GEMM)常占据主要执行时间。尽管先进BLAS库对单个GEMM调用进行了深度优化,但受限于BLAS API,每次调用需独立打包输入矩阵并恢复输出到标准内存布局。在连续GEMM中,这一限制导致冗余的打包与解包,浪费计算资源。本文提出LP-GEMM,一种将布局传播融入GEMM操作的分解方法,在保持边界处完整BLAS语义正确性的同时,消除不必要的数据重打包。我们在x86(AVX-512)和RISC-V(RVV 1.0)架构上评估了LP-GEMM在MLP类和Attention类工作负载上的表现。结果表明,相较于OpenBLAS,LP-GEMM在Intel x86平台上对连续GEMM实现了平均2.25倍的加速,并在与厂商优化库如Intel MKL的对比中表现具有竞争力。我们进一步通过仅使用BLAS级GEMM调用实现了一个独立的C++版Llama-3.2推理路径,验证了该方法在真实场景中的可行性。结果表明,利用操作间的数据布局传播可显著提升性能。

原文摘要 · Abstract (English)

In Scientific Computing and modern Machine Learning (ML) workloads, sequences of dependent General Matrix Multiplications (GEMMs) often dominate execution time. While state-of-the-art BLAS libraries aggressively optimize individual GEMM calls, they remain constrained by the BLAS API, which requires each call to independently pack input matrices and restore outputs to a canonical memory layout. In sequential GEMMs, these constraints cause redundant packing and unpacking, wasting valuable computational resources. This paper introduces LP-GEMM, a decomposition of the GEMM kernel that enables packing-layout propagation across sequential GEMM operations. This approach eliminates unnecessary data repacking while preserving full BLAS semantic correctness at the boundaries. We evaluate LP-GEMM on x86 (AVX-512) and RISC-V (RVV 1.0) architectures across MLP-like and Attention-like workloads. Our results show average speedups of 2.25x over OpenBLAS on Intel x86 for sequential GEMMs and competitive gains relative to vendor-optimized libraries such as Intel MKL. We demonstrate the practicality of the approach beyond microbenchmarks by implementing a standalone C++ version of the Llama-3.2 inference path using exclusively BLAS-level GEMM calls. These results confirm that leveraging data layout propagation between operations can significantly boost performance.

矩阵乘法高性能计算布局优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。