利用大模型聚类提示，挖掘相似类别共享特征提升分割精度
Prompt Categories Cluster for Weakly Supervised Semantic Segmentation
- 用大模型根据提示词自动聚类相似类别，发现隐含语义关联
- 在PASCAL VOC 2012上达到新最优，优于现有弱监督分割方法
- 适合研究弱监督学习与跨类别语义关系的学者参考
弱监督语义分割（WSSS）利用图像级标签，因其成本低而受到广泛关注。以往方法主要强化类别间差异以缓解语义模糊，但忽略了相似类别间共享信息的积极作用。同一类别簇内的类别具有部分相似特征，若模型能识别这些特征，可进一步缓解类别间的语义混淆。为此，本文提出一种名为提示类别聚类（Prompt Categories Clustering, PCC）的新框架。具体而言，我们探索大型语言模型（LLM）通过提示词生成类别簇的能力，这些簇有效表征了类别间的内在关联。将此关系信息融入训练网络后，模型能更好学习类别间的隐藏联系。实验结果表明，该方法在PASCAL VOC 2012数据集上表现优异，超越现有最先进弱监督分割方法。
原文摘要 · Abstract (English)
Weakly Supervised Semantic Segmentation (WSSS), which leverages image-level labels, has garnered significant attention due to its cost-effectiveness. The previous methods mainly strengthen the inter-class differences to avoid class semantic ambiguity which may lead to erroneous activation. However, they overlook the positive function of some shared information between similar classes. Categories within the same cluster share some similar features. Allowing the model to recognize these features can further relieve the semantic ambiguity between these classes. To effectively identify and utilize this shared information, in this paper, we introduce a novel WSSS framework called Prompt Categories Clustering (PCC). Specifically, we explore the ability of Large Language Models (LLMs) to derive category clusters through prompts. These clusters effectively represent the intrinsic relationships between categories. By integrating this relational information into the training network, our model is able to better learn the hidden connections between categories. Experimental results demonstrate the effectiveness of our approach, showing its ability to enhance performance on the PASCAL VOC 2012 dataset and surpass existing state-of-the-art methods in WSSS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。