arXiv:2410.12096cs.LGcs.AI2024-10被引 4

融合大模型与图学习,提升噪声环境下的表示学习鲁棒性

Bridging Large Language Models and Graph Structure Learning Models for Robust Representation Learning

  • 用大模型清理原始数据噪声,生成高质量节点特征
  • 大模型与图结构学习模型双向互训,共同优化特征与结构
  • 适用于含噪图数据的各类下游任务,尤其适合复杂场景

图表示学习同时依赖节点特征与图结构,在真实场景中常受噪声干扰。现有方法通常分别用大语言模型(LLMs)处理节点特征、图结构学习模型(GSLMs)处理图结构。本文提出LangGSL框架,整合预训练语言模型与GSLM的优势,联合提升节点特征与图结构学习。首先利用LLM过滤原始数据噪声,提取清洁信息作为特征;在互训阶段,小型语言模型处理局部属性,生成可靠伪标签与节点嵌入,注入GSLM预测过程,增强全局上下文;同时,GSLM基于语言模型输出重构图结构,并将更新标签反馈给语言模型,形成有效互训。二者在变分信息最大化框架下协同互补,显著提升特征质量与图结构鲁棒性。在多种规模与任务场景的图数据集上实验验证了该方法的可扩展性与有效性。

原文摘要 · Abstract (English)

Graph representation learning, involving both node features and graph structures, is crucial for real-world applications but often encounters pervasive noise. State-of-the-art methods typically address noise by focusing separately on node features with large language models (LLMs) and on graph structures with graph structure learning models (GSLMs). In this paper, we introduce LangGSL, a robust framework that integrates the complementary strengths of pre-trained language models and GSLMs to jointly enhance both node feature and graph structure learning. In LangGSL, we first leverage LLMs to filter noise in the raw data and extract valuable cleaned information as features, enhancing the synergy of downstream models. During the mutual learning phase in LangGSL, the core idea is to leverage the relatively small language model (LM) to process local attributes and generate reliable pseudo-labels and informative node embeddings, which are then integrated into the GSLM's prediction phase. This approach enriches the global context and enhances overall performance. Meanwhile, GSLM refines the evolving graph structure constructed from the LM's output, offering updated labels back to the LM as additional guidance, thus facilitating a more effective mutual learning process. The LM and GSLM work synergistically, complementing each other's strengths and offsetting weaknesses within a variational information-maximizing framework, resulting in enhanced node features and a more robust graph structure. Extensive experiments on diverse graph datasets of varying scales and across different task scenarios demonstrate the scalability and effectiveness of the proposed approach.

图学习大模型噪声鲁棒

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。