提出分层对比学习，解决多粒度视觉分类中的层级矛盾问题。
Beyond Flat Labels: Level-Restricted Contrastive Learning for Hierarchical Fine-Grained Vision Classification
- 限定对比学习在同一层级内进行，避免跨级误判。
- 在iNaturalist 2021上提升各层级平均准确率30.47%。
- 适合需要精准层级分类的生物图像识别任务。
多模态对比学习通过对齐图像与文本类别实现了零样本视觉分类。然而，在具有层次结构的标签空间中，现有方法常产生跨层级不一致的预测结果，例如细粒度类别与其父类别的预测存在矛盾。分析表明，问题源于跨层级对比时引入的错误负样本。为此，我们提出将对比学习限制在同级类别间，并采用组平衡设计，确保每一层级获得充分优化。所提框架显著提升了从粗到细粒度的层级一致性与分类准确率。基于BioCLIP在TreeOfLife-10M上训练，并在多个层次化分类基准上评估，模型在欧氏与双曲空间中均表现出更强的层级一致性。尤其在iNaturalist 2021（iNat21）上，跨层级平均准确率较基线提升30.47%，验证了其在层次化零样本分类中的有效性。
原文摘要 · Abstract (English)
Multimodal contrastive learning has enabled zero-shot visual classification by aligning images with textual categories. However, in hierarchically structured label spaces, existing methods often produce predictions that are inconsistent across taxonomic levels. For example, a model may predict a fine-grained category whose parent category contradicts its simultaneously predicted higher-level label. By analysis, the issue originates from false negative labels when contrastive comparison involves multiple taxonomic levels. To this end, we propose to restrict contrastive comparisons to categories within the same taxonomic level. In addition, we adopt a group-balanced design, ensuring each taxonomic level receives adequate optimization. As a result, the proposed framework improves both hierarchical consistency and classification accuracy from coarse to fine granularity. We train our model with TreeOfLife-10M based on BioCLIP and evaluate it across multiple hierarchical classification benchmarks, where the model demonstrates significantly improved hierarchical consistency in both Euclidean and hyperbolic spaces. Notably, on iNaturalist 2021 (iNat21), our method improves average accuracy across levels by 30.47% over the baseline, highlighting its effectiveness for hierarchical zero-shot classification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。