arXiv:2606.30328stat.MLcs.LG2026-06

用外推法提升病态线性系统求解精度,兼容自动微分。

Extrapolating from Regularised Solutions for Solving Ill-Conditioned Linear Systems in Machine Learning

论文配图:Extrapolating from Regularised Solutions for Solving Ill-Conditioned Linear Systems in Machine Learning
图 1 · 摘自论文原文
  • 通过理查森外推融合多次正则化求解结果
  • 相比单次正则化解,精度显著提升
  • 专为快速原型设计与自动微分优化

现代机器学习算法快速原型开发依赖轻量级数值计算。病态线性系统求解常用Tikhonov正则化,但正则项(nugget)大小选择困难,且数据自适应方法破坏自动微分,导致端到端训练不稳定。现有方法虽进行多次求解以选nugget,却仅返回一次结果,造成资源浪费。本文提出autonugget,一个基于JAX的Python工具包,可自动、稳定求解病态线性系统,完全兼容自动微分。该方法利用理查森外推融合多个不同nugget下的求解结果,得到更准确的最终解,显著优于单一nugget近似。

原文摘要 · Abstract (English)

Rapid prototyping of algorithms is a critical step in modern machine learning. Most algorithms exploit linear algebra, creating a need for lightweight numerical routines which -- while potentially sub-optimal for the task at hand -- can be rapidly implemented. For the numerical solution of ill-conditioned linear systems of equations, the standard solution for prototyping is Tikhonov-regularised inversion using a nugget. However, selection of the size of nugget is often difficult, and the use of data-adaptive procedures precludes automatic differentiation, introducing instabilities into end-to-end training. Further, while data-adaptive procedures perform multiple linear solves to select the size of nugget, only the result of one such solve is returned, which we argue is wasteful. This paper aims to circumvent the above difficulties, presenting autonugget; a Python package for automatic and stable numerical solution of linear systems suitable for rapid prototyping, and fully compatible with automatic differentiation using JAX. autonugget combines multiple linear solves using Richardson extrapolation to determine the solution of the ill-conditioned system, improving in accuracy over approximations based on a single nugget.

线性系统自动微分数值方法JAX

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。