arXiv:2501.19329cs.CV2025-01被引 5

用手绘草图提升难样本分割效果,大幅降低标注成本。

Let Human Sketches Help: Empowering Challenging Image Segmentation Task with Freehand Sketches

  • 用自由手绘轮廓替代框或点,实现直观交互标注。
  • 相比文本/框标注,显著提升迭代分割模型性能。
  • 可直接用于训练其他模型,标注效率提升120倍。

草图具有表达潜力,能通过粗略轮廓传达物体本质。首次将此特性应用于挑战性任务——伪装目标检测(COD)的分割优化。提出一种草图引导的交互式分割框架,用户可用自由手绘轮廓(勾勒物体粗略轮廓)进行标注,而非传统边界框或点。实验表明,草图输入可显著提升现有迭代分割方法性能,优于文本或边界框标注。此外,引入网络结构改进与新型草图增强技术,充分挖掘草图优势,进一步提升分割精度。值得注意的是,模型输出可直接用于训练其他神经网络,效果接近像素级标注,同时标注时间减少高达120倍,展现出降低标注门槛、推动数据标注民主化的巨大潜力。我们还发布了首个针对伪装目标检测的自由手绘草图数据集KOSCamo+,相关代码与标注工具将开源。

原文摘要 · Abstract (English)

Sketches, with their expressive potential, allow humans to convey the essence of an object through even a rough contour. For the first time, we harness this expressive potential to improve segmentation performance in challenging tasks like camouflaged object detection (COD). Our approach introduces an innovative sketch-guided interactive segmentation framework, allowing users to intuitively annotate objects with freehand sketches (drawing a rough contour of the object) instead of the traditional bounding boxes or points used in classic interactive segmentation models like SAM. We demonstrate that sketch input can significantly improve performance in existing iterative segmentation methods, outperforming text or bounding box annotations. Additionally, we introduce key modifications to network architectures and a novel sketch augmentation technique to fully harness the power of sketch input and further boost segmentation accuracy. Remarkably, our model' s output can be directly used to train other neural networks, achieving results comparable to pixel-by-pixel annotations--while reducing annotation time by up to 120 times, which shows great potential in democratizing the annotation process and enabling model training with less reliance on resource-intensive, laborious pixel-level annotations. We also present KOSCamo+, the first freehand sketch dataset for camouflaged object detection. The dataset, code, and the labeling tool will be open sourced.

图像分割草图标注交互式学习数据效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。