提出WiSE-OD方法，提升红外目标检测在跨模态下的鲁棒性。
WiSE-OD: Benchmarking Robustness in Infrared Object Detection
- 通过权重空间集成，融合RGB零样本与红外微调模型优势。
- 在合成和真实分布外数据上，检测鲁棒性显著提升，无需额外训练。
- 构建了LLVIP-C与FLIR-C两个跨模态测试基准，推动红外检测评估标准化。
红外图像中的目标检测对低光和夜间应用至关重要。然而，大规模红外数据集的缺乏迫使模型依赖在可见光（RGB）图像上预训练的权重。尽管在红外数据上微调可提高精度，但常因可见光与红外之间的模态差异导致分布外性能下降。为此，我们构建了两个跨模态分布外（OOD）基准LLVIP-C和FLIR-C，通过对标准红外数据集施加噪声等扰动生成。为进一步利用可见光与红外训练模型的互补知识，提出WiSE-OD权重空间集成方法，包含两种变体：WiSE-OD$_{ZS}$结合零样本可见光与红外微调权重；WiSE-OD$_{LP}$融合零样本与线性探测权重。在四个可见光预训练检测器及两个鲁棒基线模型上，于本研究构建的基准和真实世界分布外数据集M3FD进行评估，结果表明，WiSE-OD在跨模态及各类合成/真实分布偏移下均显著提升鲁棒性，且无需额外训练或推理开销。代码已开源。
原文摘要 · Abstract (English)
Object detection (OD) in infrared (IR) imagery is critical for low-light and nighttime applications. However, the scarcity of large-scale IR datasets forces models to rely on weights pre-trained on RGB images. While fine-tuning on IR improves accuracy, it often compromises robustness under distribution shifts due to the inherent modality gap between RGB and IR. To address this, we introduce LLVIP-C and FLIR-C, two cross-modality out-of-distribution (OOD) benchmarks built by applying corruptions to standard IR datasets. Additionally, to fully leverage the complementary knowledge from RGB and infrared-trained models, we propose WiSE-OD, a weight-space ensembling method with two variants: WiSE-OD$_{ZS}$, which combines RGB zero-shot and IR fine-tuned weights, and WiSE-OD$_{LP}$, which blends zero-shot and linear probing. Evaluated using four RGB-pretrained detectors and two robust baselines on our benchmark and in the real-world out-of-distribution M3FD dataset, our WiSE-OD improves robustness across modalities and to corruption in synthetic and real-world distribution shifts without any additional training or inference costs. Our code is available at: https://github.com/heitorrapela/wiseod.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。