自动分割深海章鱼,克服伪装与遮挡难题
HideAndSeg: an AI-based tool with automated prompting for octopus segmentation in natural habitats
- 用SAM2+YOLOv11实现从点标注到全自动分割
- 在无真值数据下仍保持高一致性,减少噪声80%以上
- 适合野外生物行为研究者,尤其擅长遮挡后重识别
分析自然栖息地中章鱼的挑战源于其伪装能力、快速皮肤变色、非刚性形变及频繁遮挡,叠加水下光照与浑浊度变化。针对缺乏大规模标注数据的问题,本文提出HideAndSeg——一种最小监督的AI分割工具,用于视频中章鱼的语义分割,并建立该任务的定量基准。该方法融合SAM2与自训练的YOLOv11检测器:用户仅需提供初始点坐标生成分割掩码,作为YOLO模型训练数据;随后通过边界框提示实现全流程自动化,无需进一步人工干预。提出两个无监督评估指标——时间一致性DICE_t与新组件数量NC_t,用于定量评价分割质量并指导掩码优化。结果表明,相较于人工提示方法,HideAndSeg显著降低分割噪声。更重要的是,即使经历完全遮挡,该方法仍可成功重识别并分割章鱼,而人工提示模型在此场景下失效。本工作大幅减少真实环境下的手动分析需求,为野生头足类动物行为研究提供高效实用工具。
原文摘要 · Abstract (English)
Analyzing octopuses in their natural habitats is challenging due to their camouflage capability, rapid changes in skin texture and color, non-rigid body deformations, and frequent occlusions, all of which are compounded by variable underwater lighting and turbidity. Addressing the lack of large-scale annotated datasets, this paper introduces HideAndSeg, a novel, minimally supervised AI-based tool for segmenting videos of octopuses. It establishes a quantitative baseline for this task. HideAndSeg integrates SAM2 with a custom-trained YOLOv11 object detector. First, the user provides point coordinates to generate the initial segmentation masks with SAM2. These masks serve as training data for the YOLO model. After that, our approach fully automates the pipeline by providing a bounding box prompt to SAM2, eliminating the need for further manual intervention. We introduce two unsupervised metrics - temporal consistency $DICE_t$ and new component count $NC_t$ - to quantitatively evaluate segmentation quality and guide mask refinement in the absence of ground-truth data, i.e., real-world information that serves to train, validate, and test AI models. Results show that HideAndSeg achieves satisfactory performance, reducing segmentation noise compared to the manually prompted approach. Our method can re-identify and segment the octopus even after periods of complete occlusion in natural environments, a scenario in which the manually prompted model fails. By reducing the need for manual analysis in real-world scenarios, this work provides a practical tool that paves the way for more efficient behavioral studies of wild cephalopods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。