优化随机神经网络的激活函数以提升动力系统算子逼近精度
Optimization of randomized neural networks for transfer operator approximation
- 固定随机权重，仅优化激活函数构建更优基函数
- 在随机微分方程与图翁随机游走问题上显著提升逼近性能
- 适合需要高效数据驱动建模的动力系统研究者
RaNNDy 是一种用于复杂动力系统转移算子数据驱动逼近的随机神经网络架构。其隐藏层权重和偏置随机初始化后固定不变，仅训练输出层，具有闭式解和极低训练成本等优势。然而，该方法受限于初始权重和偏置的选择，而这些参数决定了算子逼近所需的基函数。由于基函数由激活函数决定，因此选择合适的激活函数至关重要。本文提出一种算法，在保持随机神经网络权重和偏置固定的条件下，优化激活函数本身，从而提供更适配的字典。通过多个基准问题验证了该方法的有效性，包括随机微分方程和图翁上的随机游走。
原文摘要 · Abstract (English)
RaNNDy is a randomized neural network architecture for the data-driven approximation of transfer operators associated with complex dynamical systems. The weights and biases of the hidden layers of the network are randomly initialized and kept fixed, only the output layer is trained. This has several advantages over fully optimized neural networks, notably a closed-form solution for the output layer and significantly lower training costs. Despite these advantages, RaNNDy is restricted to the initial selection of weights and biases that parametrize the basis functions required for the operator approximation. Since the basis functions are determined by the activation function, choosing an appropriate activation function for the hidden layers is crucial. In this work, we propose an algorithm that optimizes the activation function itself, while keeping the weights and biases in the randomized neural network fixed, providing a more suitable dictionary. We illustrate the efficacy of the approach using various benchmark problems, including stochastic differential equations and random walks on graphons.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。