用自监督ViT提取局部特征，提升古籍作者识别准确率
Self-Supervised Vision Transformers for Writer Retrieval
- 用ViT+VLAD编码提取局部前景特征，无需标签训练
- 在Historical-WI达83.1% mAP，HisIR19达95.0% mAP
- 可直接迁移至CVL数据集，无需微调
尽管视觉变换器（ViT）在多个领域表现卓越，但在作者检索任务中尚未取得成功。该领域长期依赖手工特征或卷积神经网络提取的特征。本文提出一种新方法，利用ViT提取特征并结合VLAD编码进行聚合，模型采用自监督方式训练，无需任何标注数据。实验表明，在作者检索任务中，提取局部前景特征优于使用ViT的分类令牌。我们在两个历史文献数据集上进行了评估：在Historical-WI数据集上达到83.1% mAP，HisIR19数据集上达到95.0% mAP。此外，我们的ViT特征提取器可直接应用于现代数据集如CVL，无需微调，取得98.6% mAP。
原文摘要 · Abstract (English)
While methods based on Vision Transformers (ViT) have achieved state-of-the-art performance in many domains, they have not yet been applied successfully in the domain of writer retrieval. The field is dominated by methods using handcrafted features or features extracted from Convolutional Neural Networks. In this work, we bridge this gap and present a novel method that extracts features from a ViT and aggregates them using VLAD encoding. The model is trained in a self-supervised fashion without any need for labels. We show that extracting local foreground features is superior to using the ViT's class token in the context of writer retrieval. We evaluate our method on two historical document collections. We set a new state-at-of-art performance on the Historical-WI dataset (83.1\% mAP), and the HisIR19 dataset (95.0\% mAP). Additionally, we demonstrate that our ViT feature extractor can be directly applied to modern datasets such as the CVL database (98.6\% mAP) without any fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。