arXiv:2605.15599cs.CVcs.AI2026-05

在极低数据量下,预训练目标比主干网络更重要,影响细粒度分类效果。

Pretraining Objective Matters in Extreme Low-Data FGVC: A Backbone-Controlled Study

论文配图:Pretraining Objective Matters in Extreme Low-Data FGVC: A Backbone-Controlled Study
图 1 · 摘自论文原文
  • 控制主干相同,对比四种预训练方法对特征质量的影响。
  • 监督与对比学习在线性分类器下表现最优,重建类方法在非线性下更优。
  • 数据极少时应选增强边界的预训练方法,非线性分类可考虑重建型模型。

极端低数据量的细粒度分类在专业领域常见,标注成本高,但缺乏选择预训练编码器的明确指导。本文基于自建三类别宝石分级图像数据集,研究在相同主干容量下,不同预训练目标对下游表征质量的影响。比较了四种冻结的ViT-B/16编码器:监督分类、对比学习(SigLIP2)、掩码重建(MAE)和自蒸馏(DINOv3),采用留一法交叉验证并使用线性和非线性探测器评估。为降低低样本条件下的统计噪声,通过1000次置换测试分析宏AUC。结果显示,监督与对比学习编码器在线性探测中表现最佳(逻辑回归AUC: 0.768, 0.735;SVM AUC: 0.739, 0.697),MAE在非线性探测中更优(XGBoost AUC: 0.713)。DINOv3在各类探测器中均表现较差。结论建议:数据稀缺时优先选择边界增强型预训练目标,若可用非线性分类器,则可考虑重建类方法。

原文摘要 · Abstract (English)

Extreme low-data fine-grained classification is common in expert domains where labeling is expensive, yet practitioners still need principled guidance for selecting pretrained encoders. We study emerald inclusion grading with a custom dataset of labeled images across three classes and ask: under matched backbone capacity, how does pretraining objective affect downstream representation quality? We compare four frozen ViT-B/16 encoders trained with supervised classification, contrastive learning (SigLIP2), masked reconstruction (MAE), and self-distillation (DINOv3), and evaluate them with leave-one-out cross-validation using linear and nonlinear probes. To control statistical noise in the low-N regime, we use permutation testing (N=1000) on macro one-vs-rest AUC. Supervised and contrastive encoders provide the strongest linear separability (logistic AUC: 0.768 and 0.735; SVM AUC: 0.739 and 0.697), while MAE improves under nonlinear probes (XGBoost AUC: 0.713). We find that DINOv3 underperforms across probe families in this domain. These results support a practical recommendation for extreme low-data FGVC: prioritize margin-enforcing pretraining objectives when data scarcity restricts probing to linear decision rules, and consider reconstruction-style encoders when nonlinear classifiers are feasible given dataset constraints.

细粒度分类低数据预训练目标ViT

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。