小模型红外检测中，ImageNet预训练仍有用但效果递减。
Infrared Object Detection with Ultra Small ConvNets: Is ImageNet Pretraining Still Useful?
- 构建超小卷积网络，研究预训练对红外目标检测的影响。
- 参数少于100万时，预训练收益随模型变小而降低。
- 适合嵌入式设备的模型不宜过小，否则泛化能力差。
许多实际应用需要在不同工作条件和模态下保持鲁棒性的识别模型，同时运行在硬件受限的嵌入式设备上。虽然对常规规模模型而言，预训练能显著提升准确率和鲁棒性，但对于可用于边缘设备的小型模型，其作用尚不明确。本文研究了ImageNet预训练对日益缩小的主干网络（超小模型，参数少于100万）在红外视觉模态下的下游目标检测任务中鲁棒性的影响。基于标准图像识别架构的缩放定律，我们构建了两个超小主干网络族，并系统评估其性能。在三个不同数据集上的实验表明，尽管ImageNet预训练仍有一定帮助，但在超过某一容量阈值后，其在分布外检测鲁棒性方面的增益逐渐减弱。因此，建议从业者仍使用预训练，并尽可能避免模型过小——尽管小模型在域内问题上表现良好，但在工作条件变化时易变得脆弱。
原文摘要 · Abstract (English)
Many real-world applications require recognition models that are robust to different operational conditions and modalities, but at the same time run on small embedded devices, with limited hardware. While for normal size models, pre-training is known to be very beneficial in accuracy and robustness, for small models, that can be employed for embedded and edge devices, its effect is not clear. In this work, we investigate the effect of ImageNet pretraining on increasingly small backbone architectures (ultra-small models, with less than 1M parameters) with respect to robustness in downstream object detection tasks in the infrared visual modality. Using scaling laws derived from standard object recognition architectures, we construct two ultra-small backbone families and systematically study their performance. Our experiments on three different datasets reveal that while ImageNet pre-training is still useful, beyond a certain capacity threshold, it offers diminishing returns in terms of out-of-distribution detection robustness. Therefore, we advise practitioners to still use pre-training and, when possible avoid too small models as while they might work well for in-domain problems, they are brittle when working conditions are different.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。