让神经网络逐步学习标签，提升稳定性和泛化能力
Adaptive Class Emergence Training: Enhancing Neural Network Stability and Generalization through Progressive Target Evolution
- 训练时从零向one-hot标签渐进演化目标输出
- 在复杂数据和噪声下收敛更快、准确率更高
- 适合追求稳定训练的深度学习研究者
近年来，人工智能尤其是深度神经网络的发展推动了复杂任务性能的边界。传统分类训练常依赖静态目标输出（如one-hot编码），易导致优化不稳定，难以处理数据中的非线性。本文提出一种新训练方法：在整个训练过程中，将目标输出从零向量逐步演化为one-hot编码。这种渐进过渡使网络能更平滑地适应任务复杂度提升，维持平衡状态，降低过拟合风险，增强泛化能力。该方法受有限元分析中结构平衡思想启发，在合成与真实世界数据集上进行了广泛实验验证。结果表明，该方法在高复杂度与高噪声场景下均实现更快收敛、更高准确率及更好泛化性能。这一渐进式训练框架为经典方法提供了稳健替代方案，为高效稳定训练开辟新路径。
原文摘要 · Abstract (English)
Recent advancements in artificial intelligence, particularly deep neural networks, have pushed the boundaries of what is achievable in complex tasks. Traditional methods for training neural networks in classification problems often rely on static target outputs, such as one-hot encoded vectors, which can lead to unstable optimization and difficulties in handling non-linearities within data. In this paper, we propose a novel training methodology that progressively evolves the target outputs from a null vector to one-hot encoded vectors throughout the training process. This gradual transition allows the network to adapt more smoothly to the increasing complexity of the classification task, maintaining an equilibrium state that reduces the risk of overfitting and enhances generalization. Our approach, inspired by concepts from structural equilibrium in finite element analysis, has been validated through extensive experiments on both synthetic and real-world datasets. The results demonstrate that our method achieves faster convergence, improved accuracy, and better generalization, especially in scenarios with high data complexity and noise. This progressive training framework offers a robust alternative to classical methods, opening new perspectives for more efficient and stable neural network training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。