arXiv:2608.02284cs.CV2026-08

让SAM3单次运行实现高效开放词汇分割,速度提升338倍。

EOVSAM: Efficient Open-Vocabulary Segmentation with SAM 3 in One Pass

论文配图:EOVSAM: Efficient Open-Vocabulary Segmentation with SAM 3 in One Pass
图 1 · 摘自论文原文
  • 去掉提示条件,用注意力聚合实现端到端分类
  • 在所有数据集上提升精度,最快比原版快338倍
  • 适合需要高速高精度分割的实时应用

开放词汇分割可识别并分割任意文本描述的对象。SAM 3 支持名词短语引导分割,通过遍历全部词表实现良好性能,但随类别增多计算开销剧增。本文提出 EOVSAM 框架,将 SAM 3 改造成单次推理的高效开放词汇分割模型。该方法移除提示条件,使 SAM 3 成为高效掩码生成器,并引入新的注意力聚合策略,实现开放词汇分类的端到端优化。此设计避免多阶段流程和后处理启发式操作,缓解直接优化分类时的封闭集坍缩问题。EOVSAM 在所有评估数据集上均优于原始 SAM 3,推理速度最高提升 338 倍。此外,其在低分辨率下仍保持高精度,且加速效果更显著。标准语义与全景分割基准测试表明,EOVSAM 在保持竞争性或顶尖准确率的同时,相较现有开放词汇分割模型具备显著速度优势。代码与模型见 https://github.com/hustvl/EOVSAM。

原文摘要 · Abstract (English)

Open-vocabulary segmentation identifies and segments objects from arbitrary textual descriptions. SAM 3 supports noun-phrase-guided segmentation and achieves competitive open-vocabulary performance through exhaustive vocabulary traversal, yet suffers from prohibitive computational overhead as target categories scale. In this paper, we propose an Efficient Open-Vocabulary segmentation framework with SAM 3 (EOVSAM), which adapts SAM 3 for single-pass prediction. EOVSAM removes prompt conditioning to turn SAM 3 into an efficient mask generator and introduces a new Attentional Aggregation strategy to optimize open-vocabulary classification end-to-end. This formulation avoids the multi-stage pipelines and post-processing heuristics commonly used by existing methods, while mitigating the closed-set collapse that can arise when classification is optimized directly. EOVSAM consistently improves segmentation accuracy over vanilla SAM 3 on all evaluated datasets and accelerates inference by up to 338$\times$. Furthermore, EOVSAM maintains high accuracy at lower resolutions while achieving even more remarkable inference speeds. Experiments on standard semantic and panoptic segmentation benchmarks show that EOVSAM combines competitive or state-of-the-art accuracy with a substantial speed advantage over existing open-vocabulary segmentation models. Code and models are available at https://github.com/hustvl/EOVSAM.

开放词汇分割SAM3高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。