arXiv:2607.15745cs.LG2026-07

通过智能选批让小模型训练更快更准,媲美大模型。

Learning Faster without Deeper Networks: A*-Inspired Batch Selection for Efficient CNN Training

论文配图:Learning Faster without Deeper Networks: A*-Inspired Batch Selection for Efficient CNN Training
图 1 · 摘自论文原文
  • 用类似A*搜索的评分机制挑选难易适中的批次,提升学习效率。
  • 在MedMNIST-v2上,小模型用此法比ResNet-18/50准确率高15%。
  • 无需改网络或优化器,适合想提速降耗的研究者和工程师。

训练卷积神经网络时,常规使用随机打乱的小批次,存在收敛慢、学习信号衰减的问题,因许多样本很快被判定为简单。本文提出一种轻量、模型无关的A*-启发式批次选择(A*-BS),将批次调度建模为启发式搜索问题:每个批次视为搜索空间中的节点,采用结合损失难度与重复惩罚的评分机制进行排序。该策略促进有效梯度更新并保持批次多样性,无需修改网络结构或优化算法,可无缝集成至现有流程。在MedMNIST-v2的十二个2D分类任务上评估,使用约2.25×10⁵参数的简易模型,相比基准的ResNet-18和ResNet-50,在一半任务中达到更高准确率与AUC,相对提升最高达15%。相同架构与超参下,消融实验表明A*-BS在全部十二项任务上优于随机打乱。墙钟时间测量显示,该轻量模型在相同硬件下训练速度远超ResNet-18与ResNet-50。结果表明,智能批次排序可部分弥补模型深度不足,为深度网络提供高效替代方案,且性能稳定可靠。

原文摘要 · Abstract (English)

Common practice when training Convolutional Neural Networks (CNNs) is to use randomly shuffled mini-batches. This creates two limitations: slower convergence, and a diminishing learning signal, since many samples are quickly classified as easy during training. We address these inefficiencies with A*-Inspired Batch Selection (A*-BS), a lightweight, model-agnostic strategy that formulates mini-batch scheduling as a heuristic search problem. Each batch is treated as a node in a search space and ranked using an A*-like score combining a loss-based difficulty measure with a reuse penalty. This encourages informative gradient updates and batch diversity throughout training, without modifying network architectures or optimization algorithms, so it integrates seamlessly into existing pipelines. We evaluate A*-BS on the twelve 2D classification tasks of the MedMNIST-v2 benchmark, using a deliberately simple architecture of approximately 2.25x10^5 parameters, compared against the ResNet-18 and ResNet-50 baselines reported by the benchmark. On half of these tasks, the lightweight model with A*-BS reaches higher accuracy and AUC than both ResNet baselines, with relative gains of up to 15%. An ablation under identical architecture and hyperparameters shows A*-BS outperforms random batch shuffling on all twelve tasks. Wall-clock measurements further show the lightweight CNN with A*-BS trains substantially faster than ResNet-18 and ResNet-50 on identical hardware. These results indicate that intelligent batch ordering can partially compensate for reduced architectural complexity, offering a computationally efficient alternative to deeper models, with reliability reinforced by strong performance even against deeper, more sophisticated architectures.

训练加速小模型批次选择CNN

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。