Foxtsage优化器比Adam更快收敛且更稳定,但耗时更多。
Foxtsage vs. Adam: Revolution or Evolution in Optimization?
- 融合FOX-TSA全局搜索与SGD稳定性,提升优化效果。
- 训练损失均值降低42.03%,标准差改善42.19%,更稳健。
- 适合追求高精度与稳定性的模型训练场景,不看重速度者优先。
优化方法对神经网络训练的预测性能和收敛效率至关重要。本文提出一种新型混合优化方法Foxtsage,将混合式FOX-TSA与随机梯度下降(SGD)结合,用于多层感知机模型训练。在多个标准数据集上,将Foxtsage与广泛使用的Adam优化器进行对比,评估指标包括训练损失、准确率、精确率、召回率、F1分数及计算时间。实验结果表明,Foxtsage使损失均值降低42.03%(Foxtsage: 9.508, Adam: 16.402),损失标准差改善42.19%(Foxtsage: 20.86, Adam: 36.085),体现更强的一致性与鲁棒性;准确率、精确率、召回率、F1分数均值分别提升0.78%、0.91%、1.02%、0.89%,进一步证明其预测能力。然而,计算成本显著上升,时间均值增加330.87%(Foxtsage: 39.541秒, Adam: 9.177秒)。通过有效结合FOX-TSA的全局搜索能力与SGD的稳定性,Foxtsage为神经网络优化任务提供了一种鲁棒且可行的新选择。
原文摘要 · Abstract (English)
Optimization techniques are pivotal in neural network training, shaping both predictive performance and convergence efficiency. This study introduces Foxtsage, a novel hybrid optimisation approach that integrates the Hybrid FOX-TSA with Stochastic Gradient Descent for training Multi-Layer Perceptron models. The proposed Foxtsage method is benchmarked against the widely adopted Adam optimizer across multiple standard datasets, focusing on key performance metrics such as training loss, accuracy, precision, recall, F1-score, and computational time. Experimental results demonstrate that Foxtsage achieves a 42.03% reduction in loss mean (Foxtsage: 9.508, Adam: 16.402) and a 42.19% improvement in loss standard deviation (Foxtsage: 20.86, Adam: 36.085), reflecting enhanced consistency and robustness. Modest improvements in accuracy mean (0.78%), precision mean (0.91%), recall mean (1.02%), and F1-score mean (0.89%) further underscore its predictive performance. However, these gains are accompanied by an increased computational cost, with a 330.87% rise in time mean (Foxtsage: 39.541 seconds, Adam: 9.177 seconds). By effectively combining the global search capabilities of FOX-TSA with the stability and adaptability of SGD, Foxtsage presents itself as a robust and viable alternative for neural network optimization tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。