通过查询融合实现红热图像解耦检测，提升极端环境下的目标识别鲁棒性。
Modality-Decoupled RGB-Thermal Object Detector via Query Fusion
- 红热图像分别处理，通过高质量查询跨模态传递纠正预测。
- 在无配对数据下仍可训练，避免依赖成对标注数据。
- 极端光照/天气下表现更优，适合夜间或恶劣环境应用。
RGB-Thermal（RGB-T）检测的优势在于能融合多模态信息，实现不同光照与天气条件下的鲁棒检测。然而，在极端条件下某一模态质量差且干扰检测时，需进行模态分离以降低噪声影响。为此，我们提出一种基于查询融合的模态解耦检测框架（MDQF），在每个优化阶段间穿插双分支（RGB与TIR）间的查询融合。具体而言，通过选择并适配高质量分支的查询，注入到另一分支中进行修正，有效排除劣质模态的干扰。该解耦结构允许仅使用未配对的RGB或TIR图像分别优化各分支，无需依赖配对数据。大量实验表明，该方法在性能上优于现有RGB-T检测器，并实现了更强的模态独立性。
原文摘要 · Abstract (English)
The advantage of RGB-Thermal (RGB-T) detection lies in its ability to perform modality fusion and integrate cross-modality complementary information, enabling robust detection under diverse illumination and weather conditions. However, under extreme conditions where one modality exhibits poor quality and disturbs detection, modality separation is necessary to mitigate the impact of noise. To address this problem, we propose a Modality-Decoupled RGB-T detection framework with Query Fusion (MDQF) to balance modality complementation and separation. In this framework, DETR-like detectors are employed as separate branches for the RGB and TIR images, with query fusion interspersed between the two branches in each refinement stage. Herein, query fusion is performed by feeding the high-quality queries from one branch to the other one after query selection and adaptation. This design effectively excludes the degraded modality and corrects the predictions using high-quality queries. Moreover, the decoupled framework allows us to optimize each individual branch with unpaired RGB or TIR images, eliminating the need for paired RGB-T data. Extensive experiments demonstrate that our approach delivers superior performance to existing RGB-T detectors and achieves better modality independence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。