arXiv:2605.07286math.NAcs.LG2026-05

用稀疏结构和快速奇异值分解,提升随机特征网络求解微分方程的效率与稳定性。

Sparse Random-Feature Neural Networks with Krylov-Based SVD for Singularly Perturbed ODE

论文配图:Sparse Random-Feature Neural Networks with Krylov-Based SVD for Singularly Perturbed ODE
图 1 · 摘自论文原文
  • 在隐藏层激活中引入结构化稀疏性,改善低秩病态问题。
  • 采用稀疏奇异值分解求解线性最小二乘,训练速度提升显著。
  • 适用于高维或刚性微分方程,对强对流问题仍保持高精度。

随机特征神经网络(RFNN)通过固定隐藏层和解析确定输出权重实现快速训练,但其密集的隐藏层激活表示常导致低秩与严重病态问题,限制了高维或刚性系统的可扩展性和数值稳定性。本文提出一种稀疏框架,在隐藏层激活中引入结构化稀疏性以提高矩阵秩,并采用稀疏奇异值分解(sSVD)高效求解线性最小二乘问题,有效应对不良条件数。研究基于兰茨-戈卢布-卡汉双对角化技术的稀疏SVD理论,实验揭示其局限性并验证正交化步骤的必要性。结果表明,该方法在求解一维稳态对流-扩散方程(尤其强对流情形)时,相比标准稠密实现,不仅保持或提升解的准确性,还显著提升训练效率与鲁棒性。

原文摘要 · Abstract (English)

Random-feature neural networks (RFNNs), including architectures with fixed hidden layers and analytically determined output weights, offer fast training but often suffer from issues due to dense representations of the hidden layer activation. Their reliance on dense feature mappings and least squares solvers can limit scalability and numerical stability, particularly for high-dimensional or stiff systems. Specifically, the activation matrix is observed to be low-rank and extremely ill-conditioned. In this work, we propose a sparse framework for RFNNs that integrates structured sparsity into the hidden layer activations that increases the rank and employs Sparse Singular Value Decomposition (sSVD) for solving the resulting linear least squares problem scalably and efficiently while catering to the bad condition number. We explore the theory behind Lanczos-Golub-Kahan Bidiagonalization technique for sparse SVD and conduct some experiments to identify some limitations and justify the requirement for orthogonalization step in our application. Then, we demonstrate that the proposed method maintains or improves solution accuracy for solving the benchmark one-dimensional steady convection-diffusion equations case having stronger advection, while achieving substantial gains in training efficiency and robustness compared to standard dense implementations.

神经网络微分方程稀疏计算奇异值分解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。