用预训练基函数初始化神经网络,提升函数逼近的效率与泛化能力。
Weights initialization of neural networks for function approximation
- 先在参考域上训练基网络逼近多项式,再用其参数初始化目标网络。
- 一维和二维实验显示训练速度提升,跨域泛化性能显著增强。
- 适合需要快速部署、跨域迁移的科学计算与机器学习任务。
基于神经网络的函数逼近在科学计算和机器学习发展中起关键作用。然而,训练此类模型面临诸多挑战:(i) 每个目标函数通常需从头训练新模型;(ii) 性能高度依赖结构与超参数选择;(iii) 模型在训练域外泛化能力差。为克服这些难题,我们提出一种基于基函数预训练的可复用初始化框架。该方法首先在参考域上训练基神经网络以逼近多项式族,其学习到的参数用于初始化更复杂目标函数的网络。为进一步提升在任意域上的适应性,引入域映射机制,将输入变换至参考域,从而保持与预训练模型的结构对应。在一维和二维设置下的大量数值实验表明,该方法在训练效率、泛化能力和模型可迁移性方面均有显著提升,凸显了基于初始化策略在可扩展、模块化神经函数逼近中的潜力。完整代码已公开于 Gitee。
原文摘要 · Abstract (English)
Neural network-based function approximation plays a pivotal role in the advancement of scientific computing and machine learning. Yet, training such models faces several challenges: (i) each target function often requires training a new model from scratch; (ii) performance is highly sensitive to architectural and hyperparameter choices; and (iii) models frequently generalize poorly beyond the training domain. To overcome these challenges, we propose a reusable initialization framework based on basis function pretraining. In this approach, basis neural networks are first trained to approximate families of polynomials on a reference domain. Their learned parameters are then used to initialize networks for more complex target functions. To enhance adaptability across arbitrary domains, we further introduce a domain mapping mechanism that transforms inputs into the reference domain, thereby preserving structural correspondence with the pretrained models. Extensive numerical experiments in one- and two-dimensional settings demonstrate substantial improvements in training efficiency, generalization, and model transferability, highlighting the promise of initialization-based strategies for scalable and modular neural function approximation. The full code is made publicly available on Gitee.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。