arXiv:2606.01173cs.CV2026-06

提出一种无需参数的光谱可靠性描述符,提升红外-可见融合检测鲁棒性。

Bridging Multimodal Fusion and Expert Routing via Spectral Reliability Descriptors for Robust Object Detection

论文配图:Bridging Multimodal Fusion and Expert Routing via Spectral Reliability Descriptors for Robust Object Detection
图 1 · 摘自论文原文
  • 设计7维光谱可靠性描述符,融合多模态能量与相位一致性信息
  • 在DroneVehicle数据集上,合成退化下平均保留率95.0%,优于对比方法
  • 适用于需要抗模态缺失的红外-可见目标检测场景

RGB-红外检测器通常忽略跨模态融合过程产生的统计信息,导致下游模块无法判断当前交互是否可靠。本文提出一种无参数、7维的光谱可靠性描述符,综合了频带能量、幅度比、相位一致性和跨模态相关性,并将其复用于融合后阶段。该描述符驱动两种机制:光谱可靠性融合(SRF)通过门控策略将光谱残差与保守空间基底结合;可靠性条件专家路由(RCER)则将描述符与池化内容联合,引导稀疏的后融合专家。在匹配消融实验中,基于描述符的门控使mAP50优于仅依赖内容的自适应门控;2×2因子分析显示,在参数量相近时,描述符条件路由带来的增益大于专家架构本身。在DroneVehicle上的六种合成退化测试中,平均保留率提升至95.0%,高于仅内容的MoE(92.0%)和拼接法(87.9%),尤其在模态缺失时增益最大;同一模型在自然昼夜划分上,mAP50分别提升+5.2/+5.3。结果表明,将融合阶段的可靠性作为显式信号,有助于增强自适应融合与后融合条件计算。

原文摘要 · Abstract (English)

RGB-infrared detectors typically discard the statistics generated during cross-modal fusion, leaving downstream modules unaware of whether the current interaction is reliable. We propose to extract a parameter-free, 7-dimensional spectral reliability descriptor -- summarizing band energy, amplitude ratio, phase consistency, and cross-modal correlation -- and to reuse it beyond the fusion stage. The descriptor drives both Spectral Reliability Fusion (SRF), which gates a spectral residual against a conservative spatial base, and Reliability-Conditioned Expert Routing (RCER), which combines the descriptor with pooled content to steer sparse post-fusion experts. Under matched ablations, descriptor-aware gating improves mAP50 over content-only adaptive gating; a $2{\times}2$ factorial analysis further shows that descriptor-conditioned routing provides the larger marginal gain over expert architecture alone at near-equal parameter count. Under six synthetic degradations on DroneVehicle, average retention rises to 95.0%, versus 92.0% for content-only MoE and 87.9% for concatenation, with the largest gain under modality drop; the same model also improves mAP50 by +5.2/+5.3 on the natural day/night split. These results suggest that preserving fusion-time reliability as an explicit signal benefits both adaptive fusion and post-fusion conditional computation.

多模态融合目标检测可靠性建模红外视觉

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。