arXiv:2509.08318cs.CV2025-09

解决轻量级早退出模型训练与推理不匹配问题,提升精度与效率。

CalexNet: Soft Cascade-Aligned Training and Calibration for Lightweight Early-Exit Branches

  • 通过动态采样对齐训练分布,使分支学习更贴近实际推理场景。
  • 在验证集的幸存样本上校准阈值,提升分类精度;在30%-70%计算量减少下效果最优。
  • 无需修改推理结构,可直接部署于任意冻结主干的早退出框架中。

早退出级联在冻结卷积主干上实现自适应推理,但存在三类训练-推理不匹配:分支在推理时不会遇到的样本上训练;分类精度阈值在错误分布上校准;标准交叉熵损失丢弃主干的置信度信号。我们提出CalexNet(Cascade-Aligned Early eXits),仅通过训练策略改进:分支采用连续加权重要性采样,匹配级联幸存者分布;每类精度阈值在验证集的真实幸存子集上校准;分类头通过温度缩放的KL散度目标,以主干全概率输出为监督信号。结合增强型原型池化分支头,CalexNet在ResNet18和ResNet50主干上评估,覆盖CIFAR-100(20类粗粒度,主要难点设置)和CINIC-10(10类,较易跨验证设置)。在准确率-FLOPs帕累托前沿上,其表现优于或持平三个已有基线(PTEEnet、ZTW、BoostNet)及论文内“无对齐、无知识蒸馏”对照组。最大增益出现在30%-70%计算量缩减的实际应用区间,且在n=3个训练种子下保持稳定。CalexNet无需推理时结构变更,可作为即插即用方案适配任意冻结主干的早退出级联。

原文摘要 · Abstract (English)

Early-exit cascades over a frozen convolutional backbone enable adaptive inference but suffer from three sources of train-inference mismatch: branches train on samples they will never see at inference, their per-class precision thresholds are calibrated on the wrong distribution, and the standard cross-entropy target on backbone argmax labels discards the backbone's uncertainty signal. We close all three gaps with CalexNet (Cascade-Aligned Early eXits), a training-recipe-only modification: branches train under continuously-weighted importance sampling that matches the cascade-survivor distribution; per-class precision thresholds are calibrated on the actual cascade-survivor subset of the validation set; and the classification head is trained against the backbone's full softmax via a temperature-scaled KL objective. Combined with an augmented prototype-pooling branch head, CalexNet is evaluated on ResNet18 and ResNet50 backbones across CIFAR-100 (20-superclass coarse, the harder primary setting) and CINIC-10 (10-class, the easier cross-validation counterpart). On the accuracy-FLOPs Pareto frontier, CalexNet matches or exceeds three published baselines (PTEEnet, ZTW, BoostNet) and a within-paper "no-alignment, no-KD" reference. The largest gains appear in the practically relevant 30-70% FLOPs-reduction regime and are stable across n=3 training seeds. CalexNet requires no inference-time architectural change and is a drop-in for any frozen-backbone early-exit cascade.

早退出模型压缩训练优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。