用单目深度图提升轻量分割模型，仅用1.1万样本达更高精度
Efficient Segment Anything with Depth-Aware Fusion and Limited Training Data
- 融合预训练深度图与RGB特征，中间层进行深度信息融合
- 仅用11.2k样本训练，精度超过依赖百万图像的EfficientViT-SAM
- 适合资源受限场景下的高精度通用分割任务
通用分割模型SAM虽性能出色，但需海量数据（如1100万张图像）且仅依赖RGB输入。现有轻量变体虽降低计算量，仍依赖大规模训练。本文提出一种轻量级RGB-D融合框架，将EfficientViT-SAM与单目深度先验结合。深度图由预训练估计器生成，并通过专用深度编码器在中层与RGB特征融合。仅在11.2k样本（不足SA-1B的0.1%）上训练，本方法精度已超越EfficientViT-SAM，表明深度线索能提供强大的几何先验。
原文摘要 · Abstract (English)
Segment Anything Models (SAM) achieve impressive universal segmentation performance but require massive datasets (e.g., 11M images) and rely solely on RGB inputs. Recent efficient variants reduce computation but still depend on large-scale training. We propose a lightweight RGB-D fusion framework that augments EfficientViT-SAM with monocular depth priors. Depth maps are generated with a pretrained estimator and fused mid-level with RGB features through a dedicated depth encoder. Trained on only 11.2k samples (less than 0.1\% of SA-1B), our method achieves higher accuracy than EfficientViT-SAM, showing that depth cues provide strong geometric priors for segmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。