统一重参数网络提升RGB-D语义分割效率与精度
URNet: A Unified Reparameterized Network for Efficient RGB-D Semantic Segmentation
- 单编码器同步提取多模态特征并融合
- 轻量化设计使推理速度更快，精度达最新水平
- 适合需要高效部署的实时分割场景
以往的RGB-D语义分割方法通常采用双编码器分别处理彩色图像和深度图，再通过专用模块进行跨模态特征融合。然而，这类设计往往难以充分捕捉深度信息，限制了跨模态交互效果，且额外的编码分支引入冗余计算，不利于轻量化部署。为此，我们提出URNet——一种统一重参数化的RGB-D网络，在单一编码器中实现多模态特征的同步提取与跨模态融合。具体地，采用重参数化策略压缩网络结构，加速推理；在每个重参数化模块（RepBlock）中引入线性门控注意力（LGA），以充分利用不同特征尺度下的互补颜色与深度线索。此外，针对现有模型解码器设计不足的问题，我们提出一个简洁高效的通用解码器——金字塔融合解码器（PMD）。在多个RGB-D分割基准上的大量实验表明，URNet在保持高效率的同时实现了最先进的性能。代码将发布于https://github.com/Wild-Stephen/URNet。
原文摘要 · Abstract (English)
Previous RGB-D semantic segmentation methods commonly employ dual encoders to separately process RGB and depth inputs, followed by dedicated modules for cross-modal feature fusion. However, such designs often inadequately capture depth representations and consequently limit effective cross-modal interaction, while the additional encoder branch introduces redundant computation that hinders lightweight execution. To tackle these challenges, we propose URNet, a Unified Reparameterized RGB-D Network that performs simultaneous multi-modal feature extraction and cross-modal fusion within a single encoder. Specifically, we adopt a reparameterization strategy to compact the network architecture and facilitate fast inference. Within each Reparameterized Block (RepBlock), a Linear Gated Attention (LGA) module is introduced to fully exploit complementary RGB and depth cues across different feature scales. Furthermore, considering that decoder design has been relatively underexplored in existing RGB-D segmentation models, we develop a concise yet effective universal decoder, termed the Pyramid Merging Decoder (PMD). Extensive experiments on multiple RGB-D segmentation benchmarks demonstrate that URNet achieves state-of-the-art performance while maintaining high efficiency. Code will be available at https://github.com/Wild-Stephen/URNet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。