增加语义类别能提升ViT分类准确率，但超过临界点后效果反而下降。
Scaling Semantic Categories: Investigating the Impact on Vision Transformer Labeling Performance
- 通过指数增加冗余语义类别测试ViT性能变化
- 类别数超过临界值后准确率开始下降或波动
- 适合研究视觉模型标签策略优化的读者
本研究探讨了语义类别数量对视觉变换器（ViT）图像分类性能的影响。实验使用Jina AI提供的CLIP服务器，假设随着真实类别和人为引入的语义等价类别的增加，ViT的标注准确率将提升，直至达到理论极限。研究选取多种图像数据集，通过自定义Python函数处理并评估模型准确率，调整不同数据格式差异。通过指数方式逐步引入新冗余类别，观察准确率变化趋势直至趋于平稳、下降或不规则波动。结果表明，语义扩展初期可提升模型表现，但超过临界阈值后收益递减甚至逆转，揭示了类别标注策略在ViT中的局限性与优化可能。
原文摘要 · Abstract (English)
This study explores the impact of scaling semantic categories on the image classification performance of vision transformers (ViTs). In this specific case, the CLIP server provided by Jina AI is used for experimentation. The research hypothesizes that as the number of ground truth and artificially introduced semantically equivalent categories increases, the labeling accuracy of ViTs improves until a theoretical maximum or limit is reached. A wide variety of image datasets were chosen to test this hypothesis. These datasets were processed through a custom function in Python designed to evaluate the model's accuracy, with adjustments being made to account for format differences between datasets. By exponentially introducing new redundant categories, the experiment assessed accuracy trends until they plateaued, decreased, or fluctuated inconsistently. The findings show that while semantic scaling initially increases model performance, the benefits diminish or reverse after surpassing a critical threshold, providing insight into the limitations and possible optimization of category labeling strategies for ViTs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。