用频谱分析揭示稀疏矩阵结构,提升计算性能优化
Spectral Analysis for Sparse Matrix Computation: Insights and Potential

- 将稀疏矩阵视为二维信号,通过快速傅里叶变换分析频域特征
- 频谱特征能捕捉传统空间统计忽略的全局结构,提升性能预测精度
- 在大模型推理中引入频谱特征,实现1.035–1.245倍加速,适合性能优化研究者
稀疏计算是科学计算、图分析和机器学习的基础,但其性能高度依赖于稀疏模式的多样性。这是因为缓存复用、内存合并和负载均衡对稀疏模式极为敏感。本文首次探索稀疏矩阵计算与谱分析之间的联系,将稀疏矩阵视为二维信号,通过快速傅里叶变换分析其频域表示。结果表明,频谱特征能揭示传统空间统计无法充分捕捉的全局结构特性,为理解稀疏计算性能提供互补信息。实验表明,在基于机器学习的SpMV格式选择中引入频谱特征,优于仅使用空间特征的先进模型。该工作建立了频谱特性与稀疏计算间的原理性联系,为稀疏结构表征与优化提供了新视角。在剪枝后的大语言模型解码中,加入频谱特征可改进核选择,实现1.035–1.245倍的内核加速。
原文摘要 · Abstract (English)
Sparse computations are fundamental to scientific computing, graph analytics, and machine learning, yet their performance is highly sensitive to the diverse sparsity and patterns. This is because cache reuse, memory coalescing, and load balancing depend critically on the sparsity patterns. This work gives the first known exploration of the connections between sparse matrix computation and spectral analysis by treating sparse matrices as two-dimensional signals and analyzing their frequency-domain representations through Fast Fourier Transform. We show that spectral signatures uncover global structural characteristics that are not sufficiently captured by conventional spatial statistics and provide complementary information for understanding sparse computation performance. Experiments on incorporating spectral features into machine-learning-based SpMV format selection demonstrate the usefulness of such spectral analysis over a state-of-the-art spatial-only model. By uncovering the principled connections between spectral characteristics and sparse matrix computations, this work introduces a novel analytical perspective into sparse computation, and provides a new approach to enhancing the current sparse structure characterization and optimization. On pruned LLM decoding, adding spectral features improves kernel selection and yields 1.035--1.245$\times$ kernel speedups.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。