解决遥感中光学与雷达图像缺失时的检测难题,动态融合更鲁棒。
Towards Robust Optical-SAR Object Detection under Missing Modalities: A Dynamic Quality-Aware Fusion Framework
- 用可学习参考令牌动态评估特征可靠性,自适应融合。
- 在部分模态损坏时仍保持高精度,空间网6和OGSOD-2.0上领先。
- 适合需要跨模态融合的遥感目标检测场景。
光学与合成孔径雷达(SAR)融合的目标检测在遥感领域受到广泛关注,因其能提供全天候监测的互补信息。然而,实际部署受限于成像机制差异、时间不同步及配准困难,导致高质量光学-SAR图像对难以获取,常出现模态缺失或退化。尽管已有方法尝试应对,但仍缺乏对随机缺失模态的鲁棒性,且无法保证融合检测性能持续提升。为此,本文提出一种新型质量感知动态融合网络(QDFNet),利用可学习参考令牌动态评估特征可靠性并指导自适应融合。设计了动态模态质量评估(DMQA)模块,通过可学习参考令牌迭代优化特征可靠性评估,精准识别退化区域,并为后续融合提供质量引导。同时,提出正交约束归一化融合(OCNF)模块,利用正交约束保持模态独立性,基于可靠性得分动态调整融合权重,有效抑制不可靠特征传播。在SpaceNet6-OTD和OGSOD-2.0数据集上的大量实验表明,相比现有先进方法,QDFNet在部分模态损坏或缺失场景下表现出更强的优越性与有效性。
原文摘要 · Abstract (English)
Optical and Synthetic Aperture Radar (SAR) fusion-based object detection has attracted significant research interest in remote sensing, as these modalities provide complementary information for all-weather monitoring. However, practical deployment is severely limited by inherent challenges. Due to distinct imaging mechanisms, temporal asynchrony, and registration difficulties, obtaining well-aligned optical-SAR image pairs remains extremely difficult, frequently resulting in missing or degraded modality data. Although recent approaches have attempted to address this issue, they still suffer from limited robustness to random missing modalities and lack effective mechanisms to ensure consistent performance improvement in fusion-based detection. To address these limitations, we propose a novel Quality-Aware Dynamic Fusion Network (QDFNet) for robust optical-SAR object detection. Our proposed method leverages learnable reference tokens to dynamically assess feature reliability and guide adaptive fusion in the presence of missing modalities. In particular, we design a Dynamic Modality Quality Assessment (DMQA) module that employs learnable reference tokens to iteratively refine feature reliability assessment, enabling precise identification of degraded regions and providing quality guidance for subsequent fusion. Moreover, we develop an Orthogonal Constraint Normalization Fusion (OCNF) module that employs orthogonal constraints to preserve modality independence while dynamically adjusting fusion weights based on reliability scores, effectively suppressing unreliable feature propagation. Extensive experiments on the SpaceNet6-OTD and OGSOD-2.0 datasets demonstrate the superiority and effectiveness of QDFNet compared to state-of-the-art methods, particularly under partial modality corruption or missing data scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。