为多任务部署的模型统一训练设计可切换分类头，提升精度与鲁棒性。
SOLAR: Switchable Output Layer for Accuracy and Robustness in Once-for-All Training
- 为每个子网络分配独立分类头，减少特征干扰。
- 在多个数据集上精度最高提升4.71%，鲁棒性提升9.01%。
- 无需修改主干网络，适合资源受限场景下的高效部署。
Once-for-All（OFA）训练使单个超网络能生成适配不同部署场景的多个子网络，在不重新训练的情况下灵活权衡精度、鲁棒性和模型大小。然而，随着支持的子网络数量增加，主干网络中过度的参数共享会限制表征能力，导致校准性能下降，整体表现降低。为此，本文提出SOLAR（Switchable Output Layer for Accuracy and Robustness in Once-for-All Training），一种简单而有效的方法：为每个子网络分配独立的分类头。通过解耦各子网络间的logit学习过程，开关输出层（SOL）减少了表征干扰，改善优化效果，且无需修改共享主干。我们在五个数据集（SVHN、CIFAR-10、STL-10、CIFAR-100、TinyImageNet）上，使用四种超网络主干（ResNet-34、WideResNet-16-8、WideResNet-40-2、MobileNetV2）和两种OFA训练框架（OATS和SNNs）进行了评估。实验表明，相比基线方法，SOLAR在子网络精度上提升最多达1.26%~4.71%，鲁棒性提升最多达9.01%~9.01%；在TinyImageNet上，使用不同主干时精度最高提升2.93%、2.34%和1.35%（共8个子网络）。
原文摘要 · Abstract (English)
Once-for-All (OFA) training enables a single super-net to generate multiple sub-nets tailored to diverse deployment scenarios, supporting flexible trade-offs among accuracy, robustness, and model-size without retraining. However, as the number of supported sub-nets increases, excessive parameter sharing in the backbone limits representational capacity, leading to degraded calibration and reduced overall performance. To address this, we propose SOLAR (Switchable Output Layer for Accuracy and Robustness in Once-for-All Training), a simple yet effective technique that assigns each sub-net a separate classification head. By decoupling the logit learning process across sub-nets, the Switchable Output Layer (SOL) reduces representational interference and improves optimization, without altering the shared backbone. We evaluate SOLAR on five datasets (SVHN, CIFAR-10, STL-10, CIFAR-100, and TinyImageNet) using four super-net backbones (ResNet-34, WideResNet-16-8, WideResNet-40-2, and MobileNetV2) for two OFA training frameworks (OATS and SNNs). Experiments show that SOLAR outperforms the baseline methods: compared to OATS, it improves accuracy of sub-nets up to 1.26 %, 4.71 %, 1.67 %, and 1.76 %, and robustness up to 9.01 %, 7.71 %, 2.72 %, and 1.26 % on SVHN, CIFAR-10, STL-10, and CIFAR-100, respectively. Compared to SNNs, it improves TinyImageNet accuracy by up to 2.93 %, 2.34 %, and 1.35 % using ResNet-34, WideResNet-16-8, and MobileNetV2 backbones (with 8 sub-nets), respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。