针对苹果M系列芯片优化稀疏三值矩阵乘法，显著提升量化模型推理速度。
Accelerating Sparse Ternary GEMM for Quantized ML on Apple Silicon
- 设计新型分块交织稀疏数据格式，提升内存局部性。
- 向量版实现25%稀疏下提速5.59倍，达理论峰值50.2%。
- 适配苹果芯片架构，对不同稀疏度保持稳定性能。
现有库对苹果M系列CPU上的稀疏三值通用矩阵乘法（GEMM）优化不足。本文提出专为M系列处理器设计的稀疏三值GEMM内核，包含一系列架构感知优化：新颖的分块交错稀疏数据格式以提升内存局部性，增强指令级并行（ILP）的策略，以及基于NEON的单指令多数据（SIMD）向量化以利用数据级并行。其标量实现相比传统三值压缩稀疏列（TCSC）基线，在50%非零值的大矩阵上最高提速5.98倍，达到处理器理论峰值的50.2%，且在不同稀疏度下表现稳定；向量实现则在25%稀疏度的大矩阵上最高提速5.59倍，同样保持跨稀疏度稳定性。
原文摘要 · Abstract (English)
Sparse Ternary General Matrix-Matrix Multiplication (GEMM) remains under-optimized in existing libraries for Apple Silicon CPUs. We present a Sparse Ternary GEMM kernel optimized specifically for Apple's M-series processors. We propose a set of architecture-aware optimizations, including a novel blocked and interleaved sparse data format to improve memory locality, strategies to increase Instruction-Level Parallelism (ILP), and NEON-based Single Instruction Multiple Data (SIMD) vectorization to exploit data-level parallelism. Our scalar implementation achieves up to a 5.98x performance increase over a traditional Ternary Compressed Sparse Column (TCSC) baseline for large matrices with 50% ternary nonzero values (sparsity), reaching up to a 50.2% of the processor's theoretical peak performance, and remains stable across varying sparsity levels. Our vectorized implementation delivers up to a 5.59x performance increase for large matrices with 25% sparsity, and remains stable across varying sparsity levels.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。