arXiv:2506.15064cs.LGcs.NA2025-06

通过分阶段残差修正,提升神经网络在复杂问题上的精度与鲁棒性。

HiPreNets: High-Precision Neural Networks through Progressive Training

  • 分阶段训练残差网络,聚焦高误差区域优化
  • 在费曼数据集上逼近机器精度,20维电力系统预测提速238倍
  • 适合高精度安全敏感场景,如科学计算与工程仿真

深度神经网络在科学与工程的非线性问题求解中表现强大,但随着问题复杂度增加,构建高精度模型面临挑战。非凸优化与超参数敏感性导致性能难以持续提升,传统方法侧重最小化均方误差,忽视安全敏感应用中至关重要的 $L^{ ty}$ 范数误差。为此,我们提出 HiPreNets,一种通过逐级残差精修的高精度神经网络训练框架。从初始网络出发,每阶段在当前集成的归一化残差上训练修正网络,系统性降低平均误差与最坏情况误差。框架核心在于集中训练资源于输入域的高误差区域,通过损失函数设计、自适应数据采样、局部区块划分和边界感知训练等技术实现。我们在 Feynman 数据集的基准回归任务上验证该框架,结果持续优于标准全连接网络及已报道的 Kolmogorov-Arnold 网络,在部分问题上逼近机器精度。进一步应用于 20 维电力系统常微分方程流映射学习,为该类多阶段方法所研究的最高维度问题,实现显著降低的 RMSE 与 $L^{ ty}$ 误差,且构建的代理模型比直接数值模拟快 238 倍。

原文摘要 · Abstract (English)

Deep neural networks are powerful tools for solving nonlinear problems in science and engineering, but training highly accurate models becomes challenging as problem complexity increases. Non-convex optimization and sensitivity to hyperparameters make consistent performance improvement difficult, and traditional approaches prioritize minimizing mean squared error while overlooking the $L^{\infty}$ norm error that is critical in safety-sensitive applications. To address these challenges, we present HiPreNets, a progressive framework for training high-precision neural networks through sequential residual refinements. Starting from an initial network, each stage trains a refinement network on the normalized residuals of the ensemble so far, systematically reducing both average and worst-case error. A key theme throughout the framework is concentrating training effort on high-error regions of the input domain, which we pursue through complementary techniques including loss function design, adaptive data sampling, localized patching, and boundary-aware training. We validate the framework on benchmark regression problems from the Feynman dataset, where it consistently outperforms standard fully connected networks and reported Kolmogorov-Arnold Networks results, with accuracy approaching machine precision depending on select problems. We further apply the framework to learning the flow map of a 20-dimensional power system ODE, which appears to be the highest dimensional problem studied using this class of multistage methods, achieving substantial reductions in both RMSE and $L^{\infty}$ norm error while enabling a surrogate that predicts system state $238\times$ faster than direct numerical simulation.

高精度神经网络科学计算多阶段训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。