PILOT让优化器根据梯度稳定性动态调整更新策略，提升训练效果。
PILOT: Policy-Informed Learned Optimization for Adaptive Deep Network Training
- 基于梯度方向一致性判断训练稳定性，动态调节更新方式。
- 在FashionMNIST和CIFAR-10上均达到最高准确率，最高达95.71%。
- 适用于各类卷积网络，尤其适合深层模型自适应优化。
尽管优化在深度学习中至关重要，但多数优化器的更新结构在训练前即固定不变，难以应对损失曲面中梯度行为的变化，如稳定、噪声或不一致等阶段。本文提出PILOT（Policy-Informed Learned OpTimizer），一种在线优化器，可在训练过程中自适应调整更新行为。不同于固定平衡动量、归一化与符号更新的机制，PILOT以梯度方向一致性作为局部训练稳定性的信号，据此动态调节更新规则。实验表明，在FashionMNIST和CIFAR-10上，PILOT在多种卷积架构下均表现最优：在CNN上分别达到94.13%和81.94%的准确率；在ResNet-18上进一步提升至95.71%和93.42%。结果表明，学习如何在训练中动态调整更新结构，可显著提升紧凑与深层卷积模型的性能，同时保持一阶优化框架的简洁性。代码已公开于https://github.com/SattamAltwaim/PILOT.git。
原文摘要 · Abstract (English)
Despite the central role of optimization in deep learning, most optimizers rely on update structures whose functional form is fixed before training begins. This static design can limit their ability to respond to changing gradient behavior across the loss landscape, where training may shift between stable, noisy, and inconsistent regimes. This study proposes PILOT (Policy-Informed Learned OpTimizer), an online optimizer that adapts its update behavior during training. Rather than using a fixed balance between momentum, normalization, and sign-based updates, PILOT uses gradient-direction agreement as a signal of local training stability. Conditioning the update rule on this agreement signal allows the optimizer to adjust its behavior when gradients become stable, noisy, or inconsistent. Experiments on FashionMNIST and CIFAR-10 show that PILOT consistently achieves the highest accuracy among the evaluated optimizers across convolutional settings. On the CNN architecture, PILOT reaches 94.13% on FashionMNIST and 81.94% on CIFAR-10. On ResNet-18, it further improves performance, reaching 95.71% on FashionMNIST and 93.42% on CIFAR-10. These results suggest that learning how to adapt the update structure during training can improve performance across both compact and deeper convolutional models while preserving a simple first-order optimization framework. The implementation of PILOT is publicly available at https://github.com/SattamAltwaim/PILOT.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。