模型早期就出现鲁棒性，但训练后期会消失，新方法可稳定这一特性。
Robustness Emerges Early in Training Dynamics, but Is Not Preserved
- 在训练初期，浅层网络自动形成鲁棒表征和平坦损失曲面。
- 标准训练下这些鲁棒性在后期被破坏，导致性能下降。
- 无需改架构或参数，通过两种无参策略提升鲁棒性，适合视觉任务优化。
深度神经网络对自然退化仍面临鲁棒性挑战。本文发现一种鲁棒性衰减现象：浅层网络在训练早期自发产生鲁棒表征和平坦损失曲面，但在标准收敛过程中这些特性未能保持。为此，我们提出一个框架，通过针对性干预训练动态来稳定早期涌现的鲁棒先验。该方法包含两项无参策略：早期阶段稳定（EPS）和不对称权重回溯（AWR），可在不修改模型架构或引入可学习参数的前提下，稳定或恢复鲁棒的浅层配置。大量实验表明，该框架在多种基准和架构上均有效，显著提升下游迁移、动态适应及多样化计算机视觉应用的表现。
原文摘要 · Abstract (English)
Robustness to natural corruptions remains a fundamental challenge for deep neural networks. In this paper, we identify a robustness fading phenomenon where shallow layers spontaneously develop robust representations and flat loss landscapes in early training, yet these properties are not preserved during standard convergence. To address this, we propose a framework that performs strategic interventions on training dynamics to stabilize the empirically identified early-emergent robust priors. Our approach includes two parameter-free strategies: Early-Phase Stabilization~(EPS) and Asymmetric Weight Reversion~(AWR), which stabilize or recover robust shallow configurations without modifying the model architecture or introducing learnable parameters. Extensive experiments demonstrate the efficacy of our framework across various benchmarks and architectures, yielding significant gains in downstream transfer, dynamic adaptation, and diverse computer vision applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。