arXiv:2607.09104cs.CVcs.AI2026-07

用大模型提升图像分类中图结构质量,让无监督学习更准

Integrating Large Language Models and Graph Convolutional Networks for Semi-Supervised Image Classification

论文配图:Integrating Large Language Models and Graph Convolutional Networks for Semi-Supervised Image Classification
图 1 · 摘自论文原文
  • 用视觉语言模型生成图文描述,再用大模型算语义相似度
  • 基于语义相似度修剪kNN图,使连接更合理,准确率最高提升4.2%
  • 适合想改进图像图神经网络的开发者和研究者

尽管图像数据日益丰富,但标注成本高昂。半监督方法如图卷积网络(GCN)通过利用少量标签和大量无标签数据成为有效解决方案。然而,将GCN应用于图像分类的核心挑战在于图构建——图像缺乏预定义结构。现有方法多基于预训练模型提取的特征向量进行相似性计算,采用kNN或互kNN算法构建图。尽管大语言模型(LLM)在捕捉高层语义方面表现优异,其与GCN在图像分类中的融合仍不充分。本文提出使用视觉语言模型(VLM)生成图像文本描述,并由LLM估算图像间语义相似度,指导对kNN与互kNN图的边裁剪,移除语义无关连接。实验表明,利用LLM进行图优化可显著提升分类准确率,尤其在kNN图及部分骨干网络下效果明显。代码已公开于 http://gcnllm.lucasvalem.com。

原文摘要 · Abstract (English)

While the growing availability of image data has driven significant advances, labeling datasets remains costly and time-consuming. Therefore, semi-supervised approaches such as Graph Convolutional Networks (GCNs), which learn from both labeled and unlabeled data, have emerged as a promising solution. One of the primary challenges in applying GCNs to image classification is graph construction, since, unlike in citation networks or similar domains, images typically do not come with a predefined structural representation. For visual data, most studies construct graphs based on the similarity between feature vectors from pretrained deep learning backbones, typically by employing kNN or reciprocal kNN algorithms. Although Large Language Models (LLMs) have shown remarkable capability in capturing high-level semantics, their integration with GCNs for image classification remains underexplored. Aiming to fill this gap, our approach uses a Vision Language Model (VLM) to generate textual image descriptions, which are then processed by an LLM to estimate semantic similarity scores between connected images. These scores guide the pruning of edges in kNN and reciprocal kNN graphs, filtering out semantically irrelevant neighbors. Experimental results reveal that leveraging LLMs for graph refinement can improve classification accuracy, particularly for kNN graphs and some backbones. The source code is publicly available at http://gcnllm.lucasvalem.com.

图像分类图神经网络大模型融合半监督学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。