arXiv:2410.15980cs.CV2024-10CVPR被引 16

通过引入视觉相似的辅助类别,提升长尾数据中稀有类别的表征能力。

Learning from Neighbors: Category Extrapolation for Long-Tail Learning

  • 用大语言模型生成视觉相似的辅助类别,增强数据粒度。
  • 在三个基准数据集上显著优于现有方法,尤其在尾部类别表现更优。
  • 适合关注长尾学习、数据不平衡问题的研究者与应用开发者。

在深度学习中,长尾数据分布的训练平衡仍是一个长期挑战。尽管重加权和重采样等方法缓解了数据不平衡问题,但样本多样性有限仍阻碍模型学习鲁棒且泛化的特征表示,尤其是对尾部类别。本文受观察启发:粒度更细的数据集受数据不平衡影响较小。通过定量与定性分析,我们发现增加粒度能提升尾部类别的特征泛化能力。基于此,提出通过类别外推(category extrapolation)增加数据粒度的方法:引入与现有类别视觉相似的开放集辅助类别,以增强头尾类别的表征学习。为自动构建辅助数据,利用大语言模型(LLMs)作为知识库,通过网络爬取检索相关图像。为防止辅助类别主导训练,设计邻居静默损失(neighbor-silencing loss),促使模型聚焦于目标数据集内的类别判别。推理时屏蔽辅助类别分类权重,仅保留目标类别权重。在三个标准长尾基准上的大量实验与消融研究证明了该方法的有效性,显著优于使用相同数据量的强基线方法。代码将公开。

原文摘要 · Abstract (English)

Balancing training on long-tail data distributions remains a long-standing challenge in deep learning. While methods such as re-weighting and re-sampling help alleviate the imbalance issue, limited sample diversity continues to hinder models from learning robust and generalizable feature representations, particularly for tail classes. In contrast to existing methods, we offer a novel perspective on long-tail learning, inspired by an observation: datasets with finer granularity tend to be less affected by data imbalance. In this paper, we investigate this phenomenon through both quantitative and qualitative studies, showing that increased granularity enhances the generalization of learned features in tail categories. Motivated by these findings, we propose a method to increase dataset granularity through category extrapolation. Specifically, we introduce open-set auxiliary classes that are visually similar to existing ones, aiming to enhance representation learning for both head and tail classes. This forms the core contribution and insight of our approach. To automate the curation of auxiliary data, we leverage large language models (LLMs) as knowledge bases to search for auxiliary categories and retrieve relevant images through web crawling. To prevent the overwhelming presence of auxiliary classes from disrupting training, we introduce a neighbor-silencing loss that encourages the model to focus on class discrimination within the target dataset. During inference, the classifier weights for auxiliary categories are masked out, leaving only the target class weights for use. Extensive experiments and ablation studies on three standard long-tail benchmarks demonstrate the effectiveness of our approach, notably outperforming strong baseline methods that use the same amount of data. The code will be made publicly available.

长尾学习类别外推大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。