用Sigmoid+MSE替代Softmax+交叉熵，提升分类模型抗噪能力
Making Sigmoid-MSE Great Again: Output Reset Challenges Softmax Cross-Entropy in Neural Network Classification
- 采用Sigmoid激活与MSE损失，配合输出重置算法减少误差不一致
- 在MNIST、CIFAR-10等数据集上达到与SCE相当的准确率和收敛速度
- 特别适合噪声数据场景，挑战了传统分类训练范式
本研究对比分析了神经网络分类任务中两种目标函数：均方误差（MSE）与Softmax交叉熵（SCE）。尽管SCE搭配Softmax激活是传统的分类输出概率转换方式，本文探索了使用MSE结合Sigmoid激活的替代方案。提出输出重置（Output Reset）算法，有效降低不一致误差，增强分类器鲁棒性。在MNIST、CIFAR-10和Fashion-MNIST等基准数据集上的大量实验表明，采用Sigmoid+MSE的方法在准确率和收敛速度上可媲美SCE，且在噪声数据场景下表现更优。结果表明，尽管MSE传统上用于回归任务，但它在分类问题中同样具备可行性，对神经网络训练策略的传统认知构成挑战。
原文摘要 · Abstract (English)
This study presents a comparative analysis of two objective functions, Mean Squared Error (MSE) and Softmax Cross-Entropy (SCE) for neural network classification tasks. While SCE combined with softmax activation is the conventional choice for transforming network outputs into class probabilities, we explore an alternative approach using MSE with sigmoid activation. We introduce the Output Reset algorithm, which reduces inconsistent errors and enhances classifier robustness. Through extensive experiments on benchmark datasets (MNIST, CIFAR-10, and Fashion-MNIST), we demonstrate that MSE with sigmoid activation achieves comparable accuracy and convergence rates to SCE, while exhibiting superior performance in scenarios with noisy data. Our findings indicate that MSE, despite its traditional association with regression tasks, serves as a viable alternative for classification problems, challenging conventional wisdom about neural network training strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。