用极限学习机加速系外行星模拟,训练快10万倍,适合数据少的物理建模。
Extreme Learning Machines for Exoplanet Simulations: A Faster, Lightweight Alternative to Deep Learning
- 不用梯度下降,直接随机初始化权重,极大降低训练开销。
- 在序列数据上训练速度比BIRNN快10万倍,预测快40倍,性能还更好。
- 图像数据需50个EML组合使用,训练快16.4倍,适合算力有限的科研场景。
高分辨率天体与气候数据要求更复杂的模型,但传统基于梯度下降的神经网络训练成本过高。本文探索极限学习机(ELM)作为轻量级非梯度方法,在两类测试中表现优异:(i) 序列数据下,当样本数N远大于输入维度d时,单个ELM训练速度比双向RNN快10万倍,预测快40倍,且测试性能更优;(ii) 图像数据下,因d >> N,单个ELM不足,但50个ELM集成模型达到与卷积神经网络(CNN)相当精度,训练时间减少16.4倍,预测时间增加6.9倍。样本效率方面,(i) 中仅需0.28%数据即达基准效果,(ii) 中需78%数据,体现复杂度与样本效率的权衡。
原文摘要 · Abstract (English)
Increasing resolution and coverage of astrophysical and climate data necessitates increasingly sophisticated models, often pushing the limits of computational feasibility. While emulation methods can reduce calculation costs, the neural architectures typically used--optimised via gradient descent--are themselves computationally expensive to train, particularly in terms of data generation requirements. This paper investigates the utility of the Extreme Learning Machine (ELM) as a lightweight, non-gradient-based machine learning algorithm for accelerating complex physical models. We evaluate ELM surrogate models in two test cases with different data structures: (i) sequentially-structured data, and (ii) image-structured data. For test case (i), where the number of samples $N$ >> the dimensionality of input data $d$, ELMs achieve remarkable efficiency, offering a 100,000$\times$ faster training time and a 40$\times$ faster prediction speed compared to a Bi-Directional Recurrent Neural Network (BIRNN), whilst improving upon BIRNN test performance. For test case (ii), characterised by $d >> N$ and image-based inputs, a single ELM was insufficient, but an ensemble of 50 individual ELM predictors achieves comparable accuracy to a benchmark Convolutional Neural Network (CNN), with a 16.4$\times$ reduction in training time, though costing a 6.9$\times$ increase in prediction time. We find different sample efficiency characteristics between the test cases: in test case (i) individual ELMs demonstrate superior sample efficiency, requiring only 0.28% of the training dataset compared to the benchmark BIRNN, while in test case (ii) the ensemble approach requires 78% of the data used by the CNN to achieve comparable results--representing a trade-off between sample efficiency and model complexity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。