用验证任务训练模型,实现未知类别在新数据中的精准分类
VeriCam: A Verification Baseline for the Classification of Unknown Data
- 基于图像配对验证学习细粒度特征,构建类间关系图谱
- 跨设备场景下验证基线F1-Score达93.45,聚类V-Measure达80.13
- 无需标签即可识别摄像头设备,适合真实监控场景的公平评估
基础模型推动了零样本分类的新时代,但其在细粒度、微小特征区分方面仍存在表示能力不足的问题。为弥补现有研究空白,我们提出VeriCam,一种用于未见数据中未知类别分类的流水线。该方法利用专为验证任务训练的图像模型,通过判别同类别与异类别图像对,学习到包含细微差异的复杂特征空间,并构建类间关系图。我们提出两种图聚类方法:朴素算法与针对Leiden算法的特定配置。在包含真实交通监控图像的LPLCv2数据集上验证,该数据集存在显著采集设备偏差,对下游车牌识别任务构成泛化挑战。我们采用无标签方法动态识别设备,构建公平基准。在跨设备场景中,验证基线F1-Score达93.45,聚类步骤的V-Measure为80.13。所有代码已开源。
原文摘要 · Abstract (English)
The advent of foundation models have enabled a new era in zero-shot classification. Yet, key challenges persist. Despite their impressive generalization power that leverages the immense pre-training knowledge, both foundation models for image and text as well as vision-text hybrids lack the representational power needed for fine-grained, minutiae-based class separation that some real-world tasks require. To address the current gaps in the literature, we propose VeriCam, a pipeline designed to learn highly specialized features that enable classification of unknown classes in unseen data. VeriCam works by leveraging the representation power of image models trained for the verification task, where the model develops an intricate feature space that incorporates fine-grained details. By training a model to discriminate between pairs of images from the same and different classes, a relational graph is constructed, representing the class relationships between data points. We then present two approaches for graph clustering: a naive algorithm and a specific setup for the Leiden graph clustering algorithm. The pipeline is validated on the LPLCv2 dataset, which comprises real-world traffic surveillance images. We show that the dataset carries an inherent capture device bias that is posed as a generalization challenge for downstream License Plate recognition tasks such as OCR. As such, we dynamically identify capture devices with a label-agnostic approach, enabling the construction of a fair and unbiased benchmark. In the cross-device scenario, our pipeline reaches an F1-Score of 93.45 in the verification baseline and a V-Measure score of 80.13 in the clustering step. All code is publicly available at https://github.com/lmlwojcik/VeriCam
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。