用神经网络自动发现高效谱算法,提升大规模矩阵计算精度与速度
Learning to Discover Iterative Spectral Algorithms
- 基于粗略谱信息自监督训练,预测矩阵多项式递推系数
- 真实数据上迭代次数减少近10倍,精度显著优于传统方法
- 可发现类切比雪夫的最优多项式,适合高性能计算领域
我们提出AutoSpec,一种用于发现大规模数值线性代数与优化中迭代谱算法的神经网络框架。其自监督模型利用输入算子的粗粒度谱信息(如特征值估计和残差范数),预测用于计算或应用矩阵多项式的递推系数,以适配下游任务。AutoSpec的有效性依赖于三个要素:可执行短循环的网络架构;在小型合成问题上高效训练并迁移至大规模真实算子;以及任务定义的目标函数,确保在训练集中涵盖的各类谱分布下实现期望的逼近或预条件行为。我们将AutoSpec应用于对对称正定矩阵的代表性任务:加速矩阵函数逼近、加速稀疏线性求解器、以及特征值计算中的谱滤波/预条件。在真实世界矩阵上,所学算法相比无谱感知基线,精度提升达数量级,或迭代次数减少一个数量级。我们发现生成的多项式具有类似切比雪夫逼近的等波纹特性。代码已开源:https://github.com/zihanghliu/AutoSpec。
原文摘要 · Abstract (English)
We introduce AutoSpec, a neural network framework for discovering iterative spectral algorithms for large-scale numerical linear algebra and numerical optimization. Our self-supervised models adapt to input operators using coarse spectral information (e.g., eigenvalue estimates and residual norms), and predict recurrence coefficients for computing or applying a matrix polynomial tailored to a downstream task. The effectiveness of AutoSpec relies on three ingredients: an architecture whose inference pass implements short, executable numerical linear algebra recurrences; efficient training on small synthetic problems with transfer to large-scale real-world operators; and task-defined objectives that enforce the desired approximation or preconditioning behavior across the range of spectral profiles represented in the training set. We apply AutoSpec to discovering algorithms for representative tasks on spd matrices: accelerating matrix function approximation; accelerating sparse linear solvers; and spectral filtering/preconditioning for eigenvalue computations. On real-world matrices, the learned procedures deliver up to order-of-magnitude improvements in accuracy and/or reductions in iteration count, relative to spectrum-agnostic baselines. We find clear connections to classical theory: the induced polynomials may exhibit equioscillation behavior characteristic of Chebyshev polynomial approximation. The code is available at: https://github.com/zihanghliu/AutoSpec .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。