不同网络架构需不同批大小调度策略，该研究提出自适应方案提升训练效率。
One Size Does Not Fit All: Architecture-Aware Adaptive Batch Scheduling with DEBA
- 根据梯度方差与损失变化动态调整批大小，实现架构感知的自适应调度。
- 轻量级模型提速45%-62%，准确率提升1%-7%；深层模型则效果不稳定。
- 揭示了批大小自适应需考虑架构特性，适合深度学习训练优化者参考。
自适应批大小方法旨在加速神经网络训练，但现有方法对所有模型采用相同策略，假设‘一法通万理’。本文提出DEBA（动态高效批大小自适应），通过监控梯度方差、梯度范数变化和损失变化来指导批大小调整。在六个架构（ResNet-18/50、DenseNet-121、EfficientNet-B0、MobileNet-V3、ViT-B16）上，基于CIFAR-10和CIFAR-100数据集，每配置运行五次随机种子，系统评估表明：（1）轻量及中等深度模型（MobileNet-V3、DenseNet-121、EfficientNet-B0）实现45%-62%训练提速，同时准确率提升1%-7%；（2）浅层残差网络（ResNet-18）获得+2.4%至+4.0%准确率提升，提速36%-43%；深层残差网络（ResNet-50）则表现波动大，偶有性能下降；（3）已有稳定架构（ViT-B16）仅获6%提速，准确率基本不变，说明适应收益取决于原始优化稳定性。研究引入基于梯度稳定性指标（稳定性评分、梯度范数变化）的基线表征框架，可预测哪些架构能受益于自适应调度。消融实验发现：滑动窗口统计（而非全历史）和每次调整后至少5个周期的冷却期是成功关键。本工作挑战了自适应方法普适性的假设，首次提供系统性证据表明批大小自适应必须依赖架构感知设计。
原文摘要 · Abstract (English)
Adaptive batch size methods aim to accelerate neural network training, but existing approaches apply identical adaptation strategies across all architectures, assuming a one-size-fits-all solution. We introduce DEBA (Dynamic Efficient Batch Adaptation), an adaptive batch scheduler that monitors gradient variance, gradient norm variation and loss variation to guide batch size adaptations. Through systematic evaluation across six architectures (ResNet-18/50, DenseNet-121, EfficientNet-B0, MobileNet-V3, ViT-B16) on CIFAR-10 and CIFAR-100, with five random seeds per configuration, we demonstrate that the architecture fundamentally determines adaptation efficacy. Our findings reveal that: (1) lightweight and medium-depth architectures (MobileNet-V3, DenseNet-121, EfficientNet-B0) achieve a 45-62% training speedup with simultaneous accuracy improvements of 1-7%; (2) shallow residual networks (ResNet-18) show consistent gains of +2.4 - 4.0% in accuracy, 36 - 43% in speedup, while deep residual networks (ResNet-50) exhibit high variance and occasional degradation; (3) already-stable architectures (ViT-B16) show minimal speedup (6%) despite maintaining accuracy, indicating that adaptation benefits vary with baseline optimization characteristics. We introduce a baseline characterization framework using gradient stability metrics (stability score, gradient norm variation) that predicts which architectures will benefit from adaptive scheduling. Our ablation studies reveal critical design choices often overlooked in prior work: sliding window statistics (vs. full history) and sufficient cooldown periods (5+ epochs) between adaptations are essential for success. This work challenges the prevailing assumption that adaptive methods generalize across architectures and provides the first systematic evidence that batch size adaptation requires an architecture-aware design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。