BYOCL通过分层聚类实现视频图像一致分割，无需训练且高效。
BYOCL: Build Your Own Consistent Latent with Hierarchical Representative Latent Clustering
- 基于SAM编码器提取特征，分批进行内外聚类
- 时间复杂度呈指数下降，显存消耗大幅减少
- 无需微调，适配任意潜在空间，即插即用
为解决SAM等单图分割模型在处理图像序列时的语义不一致性问题，本文提出BYOCL。该模型利用SAM图像编码器提取特征，结合批次内与批次间聚类算法，实现分层原型建模，在CLIP等多类表征下表现优异。实验表明，BYOCL在多项指标上超越现有最先进单图分割模型，显著降低时间和空间开销，相比以往方法实现指数级时间缩减。本工作首次在不需训练的前提下，通过可插拔模块实现基于基础模型的一致分割，适用于任意潜在空间，具有高度通用性与效率。代码已开源。
原文摘要 · Abstract (English)
To address the semantic inconsistency issue with SAM or other single-image segmentation models handling image sequences, we introduce BYOCL. This novel model outperforms SAM in extensive experiments, showcasing its Hierarchical prototype capabilities across CLIP and other representations. BYOCL significantly reduces time and space consumption by dividing inputs into smaller batches, achieving exponential time reduction compared to previous methods. Our approach leverages the SAM image encoder for feature extraction, followed by Intra-Batch and Inter-Batch clustering algorithms. Extensive experiments demonstrate that BYOCL far exceeds the previous state-of-the-art single image segmentation model. Our work is the first to apply consistent segmentation using foundation models without requiring training, utilizing plug-and-play modules for any latent space, making our method highly efficientModels are available at \href{https://github.com/cyt1202/BYOCL.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。