仅需一张标注图,就能自动分割同类生物的精细特征。
Static Segmentation by Tracking: A Label-Efficient Approach for Fine-Grained Specimen Image Segmentation
- 将多张同物种图像拼成伪视频,用追踪方式传播分割掩码。
- 仅用1张标注图,即可在多个物种上实现高质量特征分割。
- 适合生物分类、形态学研究等需要高效标注的领域。
我们研究生物领域的图像分割任务,特别是从标本图像中分割出精细性状(如蝶翼条纹、甲虫鞘翅)。这类细粒度任务对理解生物特性至关重要,但传统方法需为每个物种手动标注数百张图像的分割掩码,极为耗时。为此,我们提出一种标签高效的方案——静态分割追踪(SST),核心思路是:同一物种的标本虽有自然变异,但目标性状具有一致性。因此,我们将标本图像拼接为“伪视频”,将性状分割重构为追踪问题。SST通过传播“伪前帧”中的已标注或预测掩码,生成未标注图像的分割结果。基于最新视频分割模型(如Segment Anything Model 2),SST仅需每物种1张标注图即可实现高质量分割,实现标本图像分析的突破。为进一步提升质量,我们引入循环一致性损失进行微调,仍仅需1张标注图。此外,SST还展示了在自然图像中的一次性实例分割和基于性状的图像检索潜力。
原文摘要 · Abstract (English)
We study image segmentation in the biological domain, particularly trait segmentation from specimen images (e.g., butterfly wing stripes, beetle elytra). This fine-grained task is crucial for understanding the biology of organisms, but it traditionally requires manually annotating segmentation masks for hundreds of images per species, making it highly labor-intensive. To address this challenge, we propose a label-efficient approach, Static Segmentation by Tracking (SST), based on a key insight: while specimens of the same species exhibit natural variation, the traits of interest show up consistently. This motivates us to concatenate specimen images into a ``pseudo-video'' and reframe trait segmentation as a tracking problem. Specifically, SST generates masks for unlabeled images by propagating annotated or predicted masks from the ``pseudo-preceding'' images. Built upon recent video segmentation models, such as Segment Anything Model 2, SST achieves high-quality trait segmentation with only one labeled image per species, marking a breakthrough in specimen image analysis. To further enhance segmentation quality, we introduce a cycle-consistent loss for fine-tuning, again requiring only one labeled image. Additionally, we demonstrate the broader potential of SST, including one-shot instance segmentation in natural images and trait-based image retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。