提出空间频率联合重建方法,提升无人机多光谱目标检测精度
SFFR: Spatial-Frequency Feature Reconstruction for Multispectral Aerial Object Detection
- 用KAN网络在频域和空域同时重建特征,增强跨模态互补性
- 在三个数据集上实现更优检测性能,尤其在尺度变化下更鲁棒
- 适合做无人机多光谱感知的算法研究者和工程应用开发者
现有多光谱目标检测方法主要基于CNN或Transformer在空间域进行特征融合,而频域特征潜力尚未充分挖掘。本文提出一种新的空间-频率特征重建方法(SFFR),利用柯尔莫哥洛夫-阿诺德网络(KAN)的空间-频率表征机制,在特征融合前重建时空域互补表示。核心组件包括频率分量交换KAN(FCEKAN)模块和多尺度高斯KAN(MSGKAN)模块。FCEKAN通过选择性频域分量交换策略,显著增强RGB与红外图像间跨模态特征的互补性与一致性;MSGKAN利用多尺度高斯基函数,有效捕捉不同无人机飞行高度下的尺度变化引起的特征差异,大幅提升模型对尺度变化的适应性与鲁棒性。实验验证了两个模块的互补性,能分别有效捕获频域与空间语义特征,促进更优融合。在SeaDroneSee、DroneVehicle和DVTOD数据集上的大量实验表明,所提方法在无人机多光谱目标感知任务中表现卓越,具有显著优势。代码将开源于https://github.com/qchenyu1027/SFFR。
原文摘要 · Abstract (English)
Recent multispectral object detection methods have primarily focused on spatial-domain feature fusion based on CNNs or Transformers, while the potential of frequency-domain feature remains underexplored. In this work, we propose a novel Spatial and Frequency Feature Reconstruction method (SFFR) method, which leverages the spatial-frequency feature representation mechanisms of the Kolmogorov-Arnold Network (KAN) to reconstruct complementary representations in both spatial and frequency domains prior to feature fusion. The core components of SFFR are the proposed Frequency Component Exchange KAN (FCEKAN) module and Multi-Scale Gaussian KAN (MSGKAN) module. The FCEKAN introduces an innovative selective frequency component exchange strategy that effectively enhances the complementarity and consistency of cross-modal features based on the frequency feature of RGB and IR images. The MSGKAN module demonstrates excellent nonlinear feature modeling capability in the spatial domain. By leveraging multi-scale Gaussian basis functions, it effectively captures the feature variations caused by scale changes at different UAV flight altitudes, significantly enhancing the model's adaptability and robustness to scale variations. It is experimentally validated that our proposed FCEKAN and MSGKAN modules are complementary and can effectively capture the frequency and spatial semantic features respectively for better feature fusion. Extensive experiments on the SeaDroneSee, DroneVehicle and DVTOD datasets demonstrate the superior performance and significant advantages of the proposed method in UAV multispectral object perception task. Code will be available at https://github.com/qchenyu1027/SFFR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。