用性能分布学习网络结构,自动推荐更优模型。
ASNN: Learning to Suggest Neural Architectures from Performance Distributions
- 基于架构与精度的映射关系,训练可生成新结构的模型。
- 在2层和3层网络中均找到超越原始数据的最佳结构。
- 适合想自动化设计神经网络的研究者使用。
神经网络的架构对性能影响重大,但目前尚无通用闭式函数描述结构与准确率的关系,导致架构设计多依赖经验或搜索。本文提出架构建议神经网络(ASNN),旨在学习架构与测试准确率之间的关系,并据此推荐改进后的架构。通过构建基于TensorFlow的模型,覆盖不同层数与节点数,在网格配置下进行10次重复实验以应对随机性,将准确率作为输入,架构参数作为输出进行训练。训练后的ASNN通过迭代预测并重新训练,逐步发现更高平均测试准确率的架构。在2层和3层情况下,所提架构均优于原始训练数据中的最佳结果,验证了其泛化能力。该方法为架构优化提供了一种高效替代随机搜索的方案,展现出自动化神经网络设计的潜力。
原文摘要 · Abstract (English)
The architecture of a neural network (NN) plays a critical role in determining its performance. However, there is no general closed-form function that maps between network structure and accuracy, making the process of architecture design largely heuristic or search-based. In this study, we propose the Architecture Suggesting Neural Network (ASNN), a model designed to learn the relationship between NN architecture and its test accuracy, and to suggest improved architectures accordingly. To train ASNN, we constructed datasets using TensorFlow-based models with varying numbers of layers and nodes. Experimental results were collected for both 2-layer and 3-layer architectures across a grid of configurations, each evaluated with 10 repeated trials to account for stochasticity. Accuracy values were treated as inputs, and architectural parameters as outputs. The trained ASNN was then used iteratively to predict architectures that yield higher performance. In both 2-layer and 3-layer cases, ASNN successfully suggested architectures that outperformed the best results found in the original training data. Repeated prediction and retraining cycles led to the discovery of architectures with improved mean test accuracies, demonstrating the model's capacity to generalize the performance-structure relationship. These results suggest that ASNN provides an efficient alternative to random search for architecture optimization, and offers a promising approach toward automating neural network design. "Parts of the manuscript, including text editing and expression refinement, were supported by OpenAI's ChatGPT. All content was reviewed and verified by the authors."
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。