arXiv:2505.14014cs.CV2025-05被引 1

EGFormer高效融合多模态信息,显著降低参数与计算量。

EGFormer: Towards Efficient and Generalizable Multimodal Semantic Segmentation

  • 动态评估各模态重要性,按需选择保留关键特征
  • 参数减少88%,计算量降低50%仍保持高精度
  • 适合资源受限场景,尤其擅长跨域泛化

近期研究探索了基于不同主干网络的多模态语义分割方法。然而,多数方法侧重提升精度,而对计算效率关注不足。为此,我们提出EGFormer,一种高效且可扩展的多模态语义分割框架,能灵活融合任意数量模态,显著减少模型参数与推理时间,同时不牺牲性能。该框架引入两个新模块:首先,任意模态评分模块(ASM)独立为各模态分配重要性分数,实现基于特征图的动态排序;其次,模态丢弃模块(MDM)在每阶段过滤低信息量模态,仅保留并聚合最有价值特征。这一设计使模型能有效利用所有可用模态信息,同时剔除冗余,保障高质量分割。此外,我们在合成到真实迁移任务中评估EGFormer的泛化能力。大量实验表明,其性能具有竞争力,参数最多减少88%,GFLOPs降低50%。在无监督域适应设置下,相比现有方法达到最先进的迁移性能。

原文摘要 · Abstract (English)

Recent efforts have explored multimodal semantic segmentation using various backbone architectures. However, while most methods aim to improve accuracy, their computational efficiency remains underexplored. To address this, we propose EGFormer, an efficient multimodal semantic segmentation framework that flexibly integrates an arbitrary number of modalities while significantly reducing model parameters and inference time without sacrificing performance. Our framework introduces two novel modules. First, the Any-modal Scoring Module (ASM) assigns importance scores to each modality independently, enabling dynamic ranking based on their feature maps. Second, the Modal Dropping Module (MDM) filters out less informative modalities at each stage, selectively preserving and aggregating only the most valuable features. This design allows the model to leverage useful information from all available modalities while discarding redundancy, thus ensuring high segmentation quality. In addition to efficiency, we evaluate EGFormer on a synthetic-to-real transfer task to demonstrate its generalizability. Extensive experiments show that EGFormer achieves competitive performance with up to 88 percent reduction in parameters and 50 percent fewer GFLOPs. Under unsupervised domain adaptation settings, it further achieves state-of-the-art transfer performance compared to existing methods.

多模态语义分割高效模型域泛化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。