arXiv:2601.14776cs.CV2026-01

用超图注意力提升多模态目标检测,解决跨模态对齐难题。

M2I2HA: Multi-modal Object Detection Based on Intra- and Inter-Modal Hypergraph Attention

  • 基于超图建模模态内与跨模态的高阶关系
  • 在多个公开数据集上超越现有方法,实现最佳检测性能
  • 适合关注多模态感知与复杂依赖建模的研究者

近年来,多模态目标检测在低光、过曝等挑战性环境中的表现显著提升。通过融合RGB与热成像、深度等模态,多模态融合增强了数据冗余性与系统鲁棒性。然而,如何有效提取模态内与跨模态的任务相关特征,以及实现精确的跨模态对齐仍是关键挑战。传统CNN受限于感受野和归纳偏置,难以捕捉长程依赖;Transformer虽具全局上下文能力,但存在二次计算开销,且仅支持成对相关性建模。Mamba等状态空间模型则因序列扫描机制破坏2D空间拓扑结构,限制了复杂高阶依赖的建模。为此,本文提出基于超图理论的多模态感知网络M2I2HA,包含模态内超图增强模块(Intra-Hypergraph Enhancement),用于捕捉各模态内的全局多对多高阶关系;以及跨模态超图融合模块(Inter-Hypergraph Fusion),以弥合不同数据源间的配置与空间差异,实现特征对齐与融合。此外,引入M2-FullPAD模块,支持网络内多层级自适应融合,增强特征分布与流动。在多个公开数据集上的大量实验表明,M2I2HA在多模态目标检测任务中达到领先性能。

原文摘要 · Abstract (English)

Recent advances in multi-modal detection have significantly improved detection accuracy in challenging environments (e.g., low light, overexposure). By integrating RGB with modalities such as thermal and depth, multi-modal fusion increases data redundancy and system robustness. However, significant challenges remain in effectively extracting task-relevant information both within and across modalities, as well as in achieving precise cross-modal alignment. While CNNs excel at feature extraction, they are limited by constrained receptive fields, strong inductive biases, and difficulty in capturing long-range dependencies. Transformer-based models offer global context but suffer from quadratic computational complexity and are confined to pairwise correlation modeling. Mamba and other State Space Models (SSMs), on the other hand, are hindered by their sequential scanning mechanism, which flattens 2D spatial structures into 1D sequences, disrupting topological relationships and limiting the modeling of complex higher-order dependencies. To address these issues, we propose a multi-modal perception network based on hypergraph theory called M2I2HA. Our architecture includes an Intra-Hypergraph Enhancement module to capture global many-to-many high-order relationships within each modality, and an Inter-Hypergraph Fusion module to align, enhance, and fuse cross-modal features by bridging configuration and spatial gaps between data sources. We further introduce a M2-FullPAD module to enable adaptive multi-level fusion of multi-modal enhanced features within the network, meanwhile enhancing data distribution and flow across the architecture. Extensive object detection experiments on multiple public datasets against baselines demonstrate that M2I2HA achieves state-of-the-art performance in multi-modal object detection tasks.

多模态检测超图注意力目标检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。