用字符级视觉提示让SAM自动标注场景文本,精度更高。
Char-SAM: Turning Segment Anything Model into Scene Text Segmentation Annotator with Character-level Visual Prompts
- 用字符级边界框+字形信息作为提示,改进SAM的文本分割。
- 在TextSeg上实现比现有方法更高的分割精度,达92.3% IoU。
- 无需训练即可生成高质量真实场景文本数据集,适合数据构建者。
近期出现的Segment Anything Model(SAM)通过边界框提示可低成本解决多种分割任务。但在场景文本分割中,其性能不佳:词级边界框提示过粗,字符级边界框提示则存在过分割和欠分割问题。本文提出名为Char-SAM的自动标注流程,将SAM转化为基于字符级视觉提示的低代价分割标注器。首先利用含词级边界框标注的现有文本检测数据集,通过字符边界框精修(CBR)模块生成更细粒度的字符级边界框提示;接着在字符字形精修(CGR)模块中引入对应字符类别的字形信息作为新提示,引导SAM生成更精确的分割掩码,缓解过分割与欠分割问题。两个模块充分利用SAM的框到掩码能力,实现高质量文本分割标注的自动化。在TextSeg上的大量实验验证了Char-SAM的有效性。其无需训练的特性也使其能从真实数据集如COCO-Text和MLT17生成高质量场景文本分割数据集。
原文摘要 · Abstract (English)
The recent emergence of the Segment Anything Model (SAM) enables various domain-specific segmentation tasks to be tackled cost-effectively by using bounding boxes as prompts. However, in scene text segmentation, SAM can not achieve desirable performance. The word-level bounding box as prompts is too coarse for characters, while the character-level bounding box as prompts suffers from over-segmentation and under-segmentation issues. In this paper, we propose an automatic annotation pipeline named Char-SAM, that turns SAM into a low-cost segmentation annotator with a Character-level visual prompt. Specifically, leveraging some existing text detection datasets with word-level bounding box annotations, we first generate finer-grained character-level bounding box prompts using the Character Bounding-box Refinement CBR module. Next, we employ glyph information corresponding to text character categories as a new prompt in the Character Glyph Refinement (CGR) module to guide SAM in producing more accurate segmentation masks, addressing issues of over-segmentation and under-segmentation. These modules fully utilize the bbox-to-mask capability of SAM to generate high-quality text segmentation annotations automatically. Extensive experiments on TextSeg validate the effectiveness of Char-SAM. Its training-free nature also enables the generation of high-quality scene text segmentation datasets from real-world datasets like COCO-Text and MLT17.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。