解决视频分割中目标丢失、碎片化和误识别问题
ENEAS: Embedding-guided Neural Ensemble for Adaptive Segmentation

- 用文本提示引导的神经集成架构,结合几何鲁棒性与时序记忆
- 在物体消失后仍能精准追踪且不误判相似物,错误率降低40%
- 适合需要高精度语义分割的3D重建、视频库分析等场景
我们提出ENEAS,一种统一的、可接受文本提示的实例追踪与语义发现方法。当前基于文本提示的分割模型(如SAM 3)仍存在时间幻觉、空间碎片化和语义误分类问题:当目标离开视野时无法报告其消失,极端近景下仅分割局部纹理而非完整对象,且过度依赖视觉特征,导致雕像、画作或反射等视觉相似物被误判为目标实体。ENEAS通过单一方法实现两类功能:精确追踪唯一实例并高质量分割,以及开放概念发现文本查询所指的任意实例。追踪方面,扩展了此前仅支持点交互的几何鲁棒的SeC架构,加入文本提示适配器并利用其时序记忆,使目标在消失后仍能保持定位且不漂移至干扰物,即使填满画面也保持完整性。发现方面,验证层结合高速视觉嵌入匹配与条件式VLM精炼,仅对模糊候选进行语义推理,有效过滤视觉模型无法区分的本体错误,同时保持低延迟。专为3D重建设计,单个误分类干扰物即可破坏资产质量,ENEAS实现了高质量语义追踪与分割,适用于大规模视频库及时间或空间无序数据集,并能准确区分外观相似但本质不同的实体。代码与模型已开源。
原文摘要 · Abstract (English)
We present ENEAS, a unified, text-promptable method for instance tracking and semantic discovery. Text-promptable segmentation models, including the latest foundation models such as SAM 3, still suffer from temporal hallucinations, spatial fragmentation, and semantic misclassification: they fail to report target absence when an object leaves the field of view, segment local textures instead of the complete object during extreme close-ups, and prioritize visual features over ontological reality, so that visually similar artifacts such as statues, paintings, or reflections are segmented as target entities. ENEAS works two ways from a single method: precise tracking and high-quality segmentation of a unique instance, and open-concept discovery of every instance a text query names, resolved by a semantic verification layer. For tracking, we extend the geometrically robust SeC architecture, previously limited to point interactions, with a text-prompting adapter and leverage its temporal memory, so that the target is held through disappearance without drifting to distractors and kept whole even when it fills the entire view. For discovery, the verification layer combines high-speed visual embedding matching with conditional VLM refinement, invoking semantic reasoning only for ambiguous candidates, which filters out the ontological errors that visual-only models cannot distinguish while keeping latency low. Designed with 3D reconstruction in mind, where a single misclassified distractor corrupts the asset, ENEAS unlocks high-quality semantic tracking and segmentation of video, of broad libraries, and of collections of temporally or spatially unordered data, together with the discrimination to tell true instances from their doppelgangers: things that look alike but are not the same. The code and models are available at https://github.com/speridlabs/eneas
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。