用特征嵌入和尺寸比自动清理昆虫图像数据集,提升质量。
Efficient Curation of Invertebrate Image Datasets Using Feature Embeddings and Automatic Size Comparison
- 用预训练模型提取特征嵌入,找视觉最不同的图片作为代表
- 通过面积比较发现大量错误图像,如残肢或分类错误样本
- 提供新评估指标和公开工具,适合生态监测数据清洗
近年来,随着计算机视觉方法在环境监测中的兴起,用于监测的图像数据集数量迅速增加。高质量的数据集对计算机视觉应用至关重要,但数据清洗通常随意进行,且方法很少被发表。本文提出一种针对包含同物种多图像、背景相对一致的无脊椎动物图像数据集的清洗方法:利用预训练深度神经网络提取特征嵌入,通过将图像嵌入与群体原型嵌入比较,筛选出视觉上最独特的图像。此外,我们证明基于面积的简单尺寸比较可有效识别大量常见错误图像,如分离的肢体或误分类样本。我们还提出了新的评估指标来衡量人工参与的异常检测效果。相关代码和包含标注错误图像的基准数据集已在 https://github.com/mikkoim/taxonomist-studio 公开。
原文摘要 · Abstract (English)
The amount of image datasets collected for environmental monitoring purposes has increased in the past years as computer vision assisted methods have gained interest. Computer vision applications rely on high-quality datasets, making data curation important. However, data curation is often done ad-hoc and the methods used are rarely published. We present a method for curating large-scale image datasets of invertebrates that contain multiple images of the same taxa and/or specimens and have relatively uniform background in the images. Our approach is based on extracting feature embeddings with pretrained deep neural networks, and using these embeddings to find visually most distinct images by comparing their embeddings to the group prototype embedding. Also, we show that a simple area-based size comparison approach is able to find a lot of common erroneous images, such as images containing detached body parts and misclassified samples. In addition to the method, we propose using novel metrics for evaluating human-in-the-loop outlier detection methods. The implementations of the proposed curation methods, as well as a benchmark dataset containing annotated erroneous images, are publicly available in https://github.com/mikkoim/taxonomist-studio.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。