YOLOE-26实现实时开放词汇实例分割,兼顾高效与灵活识别。
YOLOE-26: Integrating YOLO26 with YOLOE for Real-Time Open-Vocabulary Instance Segmentation
- 用对象嵌入替代固定类别分类,通过文本或图像提示实现开放词汇识别。
- 支持零开销文本提示、示例引导和免提示推理,统一在嵌入空间中切换模式。
- 兼容Ultralytics生态,适用于动态现实场景的实时部署。
本文提出YOLOE-26,一种将部署优化的YOLO26架构与YOLOE的开放词汇学习范式结合的统一框架,实现实时开放词汇实例分割。基于无NMS、端到端设计的YOLOv26,该方法保持了YOLO家族的高效性与确定性,同时突破封闭集识别限制。YOLOE-26采用带PAN/FPN式多尺度特征聚合的卷积主干,后接端到端回归与实例分割头。关键创新在于以对象嵌入头替代固定类别逻辑,将分类转化为与文本描述、视觉样例或内置词汇生成的提示嵌入的相似度匹配。为实现高效开放词汇推理,引入可重参数化区域-文本对齐(RepRTA)实现零开销文本提示,语义激活视觉提示编码器(SAVPE)支持示例引导分割,以及懒惰区域提示对比(Lazy Region Prompt Contrast)实现免提示推理。所有提示模式均在统一对象嵌入空间中运行,可无缝切换文本提示、视觉提示与完全自主分割。大量实验表明,不同模型规模下均具一致扩展性及良好的准确率-效率权衡。训练策略融合大规模检测与定位数据集,采用多任务优化,且完全兼容Ultralytics生态系统用于训练、验证与部署。总体而言,YOLOE-26为动态真实环境中的实时开放词汇实例分割提供了实用且可扩展的解决方案。
原文摘要 · Abstract (English)
This paper presents YOLOE-26, a unified framework that integrates the deployment-optimized YOLO26(or YOLOv26) architecture with the open-vocabulary learning paradigm of YOLOE for real-time open-vocabulary instance segmentation. Building on the NMS-free, end-to-end design of YOLOv26, the proposed approach preserves the hallmark efficiency and determinism of the YOLO family while extending its capabilities beyond closed-set recognition. YOLOE-26 employs a convolutional backbone with PAN/FPN-style multi-scale feature aggregation, followed by end-to-end regression and instance segmentation heads. A key architectural contribution is the replacement of fixed class logits with an object embedding head, which formulates classification as similarity matching against prompt embeddings derived from text descriptions, visual examples, or a built-in vocabulary. To enable efficient open-vocabulary reasoning, the framework incorporates Re-Parameterizable Region-Text Alignment (RepRTA) for zero-overhead text prompting, a Semantic-Activated Visual Prompt Encoder (SAVPE) for example-guided segmentation, and Lazy Region Prompt Contrast for prompt-free inference. All prompting modalities operate within a unified object embedding space, allowing seamless switching between text-prompted, visual-prompted, and fully autonomous segmentation. Extensive experiments demonstrate consistent scaling behavior and favorable accuracy-efficiency trade-offs across model sizes in both prompted and prompt-free settings. The training strategy leverages large-scale detection and grounding datasets with multi-task optimization and remains fully compatible with the Ultralytics ecosystem for training, validation, and deployment. Overall, YOLOE-26 provides a practical and scalable solution for real-time open-vocabulary instance segmentation in dynamic, real-world environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。