提出均衡模态贡献的分割框架，提升真实场景下多模态模型鲁棒性。
Robust Multimodal Semantic Segmentation with Balanced Modality Contributions
- 通过等编码设计实现多模态特征平衡融合。
- 在多个数据集上显著提升分割性能，缓解模态失衡问题。
- 适合应对传感器失效或环境变化的工业级视觉任务。
多模态语义分割通过利用跨模态互补性增强模型鲁棒性。然而，现有方法常因模态间依赖不均，在实际场景中主导模态退化时整体性能显著下降。因此，模态平衡成为实用多模态分割的关键挑战。为此，我们提出EQUISeg框架，通过模态等编码机制实现贡献均衡。基于四阶段交叉模态变换块（CMTB），EQUISeg支持高效多模态融合与分层选择。此外，设计自引导模块（SGM），引入相互引导机制，使各模态可自适应调整贡献，提升在退化条件下的鲁棒性。在多个数据集上的大量实验表明，EQUISeg取得显著性能提升，有效缓解了模态不平衡带来的负面影响。
原文摘要 · Abstract (English)
Multimodal semantic segmentation enhances model robustness by exploiting cross-modal complementarities. However, existing methods often suffer from imbalanced modal dependencies, where overall performance degrades significantly once a dominant modality deteriorates in real-world scenarios. Thus, modality balance has become acritical challenge for practical multimodal segmentation. To address this issue, we propose EQUISeg, a multimodal segmentation framework that balances modality contributions through equal encoding of modalities. Built upon a four-stage Cross-modal Transformer Block(CMTB), EQUISeg enables efficient multimodal fusion and hierarchical selection. Furthermore, we design a Self-guided Module(SGM) that mitigates modality imbalance by introducing a mutual guidance mechanism, enabling each modality to adaptively adjust its contribution and enhance robustness under degraded conditions. Extensive experiments on multiple datasets demonstrate that EQUISeg achieves significant performance gains and effectively alleviates the adverse effects of modality imbalance in segmentation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。