动态切换优化器，让训练更快更稳地达到目标精度。
OptiRoulette Optimizer: A New Stochastic Meta-Optimizer for up to 5.3x Faster Convergence
- 训练中随机选最优更新策略，不固定单一优化器。
- 在多个数据集上提升准确率，最高增益达9.73个百分点。
- 适合追求稳定高效训练的深度学习研究者使用。
本文提出OptiRoulette，一种基于随机采样的元优化器，可在训练过程中动态选择更新规则，而非固定单一优化器。该方法结合了预热期优化器锁定、活跃优化器池中的随机采样、过渡期兼容性自适应学习率缩放以及故障感知的池替换机制。OptiRoulette可作为torch.optim.Optimizer兼容组件直接使用，并支持pip安装。我们在五个图像分类任务上完成了10组种子实验：CIFAR-100、CIFAR-100-C、SVHN、Tiny ImageNet和Caltech-256。相比单个优化器AdamW基线，OptiRoulette将测试准确率分别提升至：CIFAR-100（0.6734→0.7656，+9.22）、CIFAR-100-C（0.2904→0.3355，+4.52）、SVHN（0.9667→0.9756，+0.89）、Tiny ImageNet（0.5669→0.6642，+9.73）和Caltech-256（0.5946→0.6920，+9.74）。其核心优势在于高目标下的收敛可靠性：在所有任务中，10次运行均达指定目标（如CIFAR-100 0.75），而基线未达成任何一次。在相同目标下，时间消耗显著减少（如Caltech-256达0.59时：25.7 vs 77.0 epochs）。所有数据集配对种子差值为正，仅CIFAR-100-C ROC-AUC未达统计显著。
原文摘要 · Abstract (English)
This paper presents OptiRoulette, a stochastic meta-optimizer that selects update rules during training instead of fixing a single optimizer. The method combines warmup optimizer locking, random sampling from an active optimizer pool, compatibility-aware learning-rate scaling during optimizer transitions, and failure-aware pool replacement. OptiRoulette is implemented as a drop-in, "torch.optim.Optimizer-compatible" component and packaged for pip installation. We report completed 10-seed results on five image-classification suites: CIFAR-100, CIFAR-100-C, SVHN, Tiny ImageNet, and Caltech-256. Against a single-optimizer AdamW baseline, OptiRoulette improves mean test accuracy from 0.6734 to 0.7656 on CIFAR-100 (+9.22 percentage points), 0.2904 to 0.3355 on CIFAR-100-C (+4.52), 0.9667 to 0.9756 on SVHN (+0.89), 0.5669 to 0.6642 on Tiny ImageNet (+9.73), and 0.5946 to 0.6920 on Caltech-256 (+9.74). Its main advantage is convergence reliability at higher targets: it reaches CIFAR-100/CIFAR-100-C 0.75, SVHN 0.96, Tiny ImageNet 0.65, and Caltech-256 0.62 validation accuracy in 10/10 runs, while the AdamW baseline reaches none of these targets within budget. On shared targets, OptiRoulette also reduces time-to-target (e.g., Caltech-256 at 0.59: 25.7 vs 77.0 epochs). Paired-seed deltas are positive on all datasets; CIFAR-100-C test ROC-AUC is the only metric not statistically significant in the current 10-seed study.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。