用混合优化算法提升神经网络训练效率与精度
A Study of Hybrid and Evolutionary Metaheuristics for Single Hidden Layer Feedforward Neural Network Architecture
- 结合粒子群与梯度下降，提升局部搜索能力
- 混合方法使均方误差降低90%-95%以上
- 适合追求高精度训练的科研与工程应用
使用随机梯度下降（SGD）训练人工神经网络常面临计算成本高和易陷入局部最优的问题，因其依赖部分权重梯度。本文研究粒子群优化（PSO）与遗传算法（GA）这两种基于种群的元启发式优化器，作为SGD的替代方案。提出一种混合PSO-SGD策略以提高局部搜索效率。结果表明，该混合方法在不同网络规模下相比传统GA与PSO，使中位数训练均方误差（MSE）降低90%至95%（例如，在Sphere函数中从约0.02降至约0.001）。RMHC方法相较GA将MSE降低约85%至90%。而随机搜索（RS）始终表现不佳，误差超过0.3。研究显示，混合与进化方法显著提升训练效率与准确性，且支持构建块假说（BBH），表明有利权重结构可在进化过程中被保留。
原文摘要 · Abstract (English)
Training Artificial Neural Networks (ANNs) with Stochastic Gradient Descent (SGD) frequently encounters difficulties, including substantial computing expense and the risk of converging to local optima, attributable to its dependence on partial weight gradients. Therefore, this work investigates Particle Swarm Optimization (PSO) and Genetic Algorithms (GAs) - two population-based Metaheuristic Optimizers (MHOs) - as alternatives to SGD to mitigate these constraints. A hybrid PSO-SGD strategy is developed to improve local search efficiency. The findings indicate that the hybrid PSO-SGD technique decreases the median training MSE by 90 to 95 percent relative to conventional GA and PSO across various network sizes (e.g., from around 0.02 to approximately 0.001 in the Sphere function). RMHC attains substantial enhancements, reducing MSE by roughly 85 to 90 percent compared to GA. Simultaneously, RS consistently exhibits errors exceeding 0.3, signifying subpar performance. These findings underscore that hybrid and evolutionary procedures significantly improve training efficiency and accuracy compared to conventional optimization methods and imply that the Building Block Hypothesis (BBH) may still be valid, indicating that advantageous weight structures are retained during evolutionary search.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。