利用场景信息提升细粒度类别发现准确率
MOS: Modeling Object-Scene Associations in Generalized Category Discovery
- 设计MLP模块建模物体与场景关联，缓解类别歧义问题
- 在细粒度数据集上提升4%平均准确率，超越现有方法
- 适合关注开放世界分类与场景先验的开发者
广义类别发现（GCD）旨在用已标注数据集的知识对未标注图像中的基础类和新类别进行分类。以往研究忽视场景信息或视其为噪声，削弱了模型训练效果。本文认为场景信息应作为推断新类别的强先验。我们指出，导致场景信息被误读的关键原因是GCD中的歧义挑战：基础场景中的新物体可能被误判为基类，而新场景中的基类物体可能被误识为新类。一旦解决此歧义问题，场景信息可充分发挥作用，显著提升模型性能。为此，提出物体-场景关联建模（MOS）框架，采用基于MLP的场景感知模块增强模型表现。在具有挑战性的细粒度数据集上，相比当前最优方法，平均准确率提升4%，充分验证其在细粒度GCD任务中的优越性。代码已公开于https://github.com/JethroPeng/MOS。
原文摘要 · Abstract (English)
Generalized Category Discovery (GCD) is a classification task that aims to classify both base and novel classes in unlabeled images, using knowledge from a labeled dataset. In GCD, previous research overlooks scene information or treats it as noise, reducing its impact during model training. However, in this paper, we argue that scene information should be viewed as a strong prior for inferring novel classes. We attribute the misinterpretation of scene information to a key factor: the Ambiguity Challenge inherent in GCD. Specifically, novel objects in base scenes might be wrongly classified into base categories, while base objects in novel scenes might be mistakenly recognized as novel categories. Once the ambiguity challenge is addressed, scene information can reach its full potential, significantly enhancing the performance of GCD models. To more effectively leverage scene information, we propose the Modeling Object-Scene Associations (MOS) framework, which utilizes a simple MLP-based scene-awareness module to enhance GCD performance. It achieves an exceptional average accuracy improvement of 4% on the challenging fine-grained datasets compared to state-of-the-art methods, emphasizing its superior performance in fine-grained GCD. The code is publicly available at https://github.com/JethroPeng/MOS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。