arXiv:2411.03728cs.CV2024-11中稿 · TGRS 2025被引 10

提出轻量级网络,实现无人机红外与可见光图像的实时显著目标检测。

Efficient Fourier Filtering Network with Contrastive Learning for AAV-based Unaligned Bimodal Salient Object Detection

  • 用语义对比损失实现双模态无参数对齐,提升特征互补性。
  • 通过傅里叶滤波机制,参数减少70%,计算量降低49.4%,推理速度提升152.5%。
  • 在多个数据集上超越19个先进模型,适合资源受限的无人机部署。

基于自主飞行器(AAV)的双模态显著目标检测(BSOD)旨在利用未对齐的RGB与热成像配对中的互补信息分割显著目标。然而,现有模型计算开销大,难以在真实AAV设备上应用。为此,本文提出高效傅里叶滤波网络与对比学习结合的方法,实现实时且高精度检测。首先设计语义对比对齐损失,在不增加参数的前提下实现双模态语义对齐,促进特征互训。其次,受快速傅里叶变换启发,提出同步对齐融合机制,通过分层滤波在通道与空间维度对齐并融合双模特征。所提模型AlignSal相较前沿模型MROS,参数减少70.0%,浮点运算量下降49.4%,推理速度提升152.5%。在AAV RGB-T 2400及七个双模态密集预测数据集上的实验表明,AlignSal在多数评估指标上优于19个先进模型,兼具实时性与强泛化能力。消融实验进一步验证其可有效提升现有对齐型BSOD模型在未对齐数据上的性能。代码已开源。

原文摘要 · Abstract (English)

Autonomous aerial vehicle (AAV)-based bi-modal salient object detection (BSOD) aims to segment salient objects in a scene utilizing complementary cues in unaligned RGB and thermal image pairs. However, the high computational expense of existing AAV-based BSOD models limits their applicability to real-world AAV devices. To address this problem, we propose an efficient Fourier filter network with contrastive learning that achieves both real-time and accurate performance. Specifically, we first design a semantic contrastive alignment loss to align the two modalities at the semantic level, which facilitates mutual refinement in a parameter-free way. Second, inspired by the fast Fourier transform that obtains global relevance in linear complexity, we propose synchronized alignment fusion, which aligns and fuses bi-modal features in the channel and spatial dimensions by a hierarchical filtering mechanism. Our proposed model, AlignSal, reduces the number of parameters by 70.0%, decreases the floating point operations by 49.4%, and increases the inference speed by 152.5% compared to the cutting-edge BSOD model (i.e., MROS). Extensive experiments on the AAV RGB-T 2400 and seven bi-modal dense prediction datasets demonstrate that AlignSal achieves both real-time inference speed and better performance and generalizability compared to nineteen state-of-the-art models across most evaluation metrics. In addition, our ablation studies further verify AlignSal's potential in boosting the performance of existing aligned BSOD models on AAV-based unaligned data. The code is available at: https://github.com/JoshuaLPF/AlignSal.

显著目标检测双模态轻量化无人机

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。