通过图文融合与伪标签,提升小样本实例分割性能。
Integrated Image-Text Based on Semi-supervised Learning for Small Sample Instance Segmentation
- 利用图文特征融合增强分类精度。
- 生成伪标签扩大可用样本数,提升掩码准确率。
- 无需预训练,适用于多种场景与框架。
小样本实例分割是一项极具挑战性的任务,现有方法多采用元学习策略,先在支持集上预训练模型,再在查询集上微调。预训练阶段高度依赖任务相关数据,需大量额外训练时间,且数据集需与目标任务相近才有效。本文提出一种新方法,旨在不增加标注负担和训练成本的前提下,最大化利用已有信息。设计两个模块:一是通过学习生成伪标签,充分挖掘未标注数据,扩充样本数量;二是融合图像与文本特征,提升分类准确性。这两个模块适用于无框和有框两类框架。实验在陆地、水下和显微镜三类不同场景的数据集上进行。结果表明,图文融合可校准分类置信度,伪标签有助于模型生成更精确的掩码。所有结果验证了该方法的有效性与优越性。
原文摘要 · Abstract (English)
Small sample instance segmentation is a very challenging task, and many existing methods follow the training strategy of meta-learning which pre-train models on support set and fine-tune on query set. The pre-training phase, which is highly task related, requires a significant amount of additional training time and the selection of datasets with close proximity to ensure effectiveness. The article proposes a novel small sample instance segmentation solution from the perspective of maximizing the utilization of existing information without increasing annotation burden and training costs. The proposed method designs two modules to address the problems encountered in small sample instance segmentation. First, it helps the model fully utilize unlabeled data by learning to generate pseudo labels, increasing the number of available samples. Second, by integrating the features of text and image, more accurate classification results can be obtained. These two modules are suitable for box-free and box-dependent frameworks. In the way, the proposed method not only improves the performance of small sample instance segmentation, but also greatly reduce reliance on pre-training. We have conducted experiments in three datasets from different scenes: on land, underwater and under microscope. As evidenced by our experiments, integrated image-text corrects the confidence of classification, and pseudo labels help the model obtain preciser masks. All the results demonstrate the effectiveness and superiority of our method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。