针对异构热成像与可见光传感器,提出两种自适应融合方法提升无人机检测性能。
Alignment-Aware and Reliability-Gated Multimodal Fusion for Unmanned Aerial Vehicle Detection Across Heterogeneous Thermal-Visual Sensors
- 用仿射配准与引导滤波保持热成像显著性并增强结构细节。
- 通过可靠性加权注意力机制,实现热对比与视觉锐度的动态平衡。
- 在14.7万帧数据上验证,最高召回率达98.64%,显著优于单模态检测器。
可靠的无人飞行器(UAV)检测对自主空域监控至关重要,但在分辨率、视角和视场差异较大的异构传感器融合中仍具挑战。传统融合方法如小波、拉普拉斯及决策级融合常无法保持模态间空间对应,且受标注不一致影响,真实场景鲁棒性差。本文提出两种新策略:注册感知引导图像融合(RGIF)与可靠性门控模态注意力融合(RGMAF)。RGIF采用基于增强相关系数(ECC)的仿射配准结合引导滤波,在保留热成像显著性的同时增强结构细节;RGMAF融合仿射与光流配准,引入可靠性加权注意力机制,自适应平衡热对比度与视觉清晰度。实验基于包含147,417帧空中对空标注数据的多传感器多视角固定翼无人机(MMFW-UAV)数据集进行。单模态检测中YOLOv10x跨域表现最稳定,被选为检测主干。RGIF使视觉基线提升2.13% mAP@50(达97.65%),而RGMAF达到最高召回率98.64%。结果表明,注册感知与可靠性自适应融合可有效提升异构模态集成能力,显著增强多模态环境下的无人机检测性能。
原文摘要 · Abstract (English)
Reliable unmanned aerial vehicle (UAV) detection is critical for autonomous airspace monitoring but remains challenging when integrating sensor streams that differ substantially in resolution, perspective, and field of view. Conventional fusion methods-such as wavelet-, Laplacian-, and decision-level approaches-often fail to preserve spatial correspondence across modalities and suffer from annotation of inconsistencies, limiting their robustness in real-world settings. This study introduces two fusion strategies, Registration-aware Guided Image Fusion (RGIF) and Reliability-Gated Modality-Attention Fusion (RGMAF), designed to overcome these limitations. RGIF employs Enhanced Correlation Coefficient (ECC)-based affine registration combined with guided filtering to maintain thermal saliency while enhancing structural detail. RGMAF integrates affine and optical-flow registration with a reliability-weighted attention mechanism that adaptively balances thermal contrast and visual sharpness. Experiments were conducted on the Multi-Sensor and Multi-View Fixed-Wing (MMFW)-UAV dataset comprising 147,417 annotated air-to-air frames collected from infrared, wide-angle, and zoom sensors. Among single-modality detectors, YOLOv10x demonstrated the most stable cross-domain performance and was selected as the detection backbone for evaluating fused imagery. RGIF improved the visual baseline by 2.13% mAP@50 (achieving 97.65%), while RGMAF attained the highest recall of 98.64%. These findings show that registration-aware and reliability-adaptive fusion provides a robust framework for integrating heterogeneous modalities, substantially enhancing UAV detection performance in multimodal environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。