arXiv:2409.01145cs.SIcs.AI2024-09被引 7

用大模型生成文本增强,提升图文图对比学习效果

LATEX-GCL: Large Language Models (LLMs)-Based Data Augmentation for Text-Attributed Graph Contrastive Learning

  • 用大模型直接对文本做增强,避免编码损失
  • 在4个数据集上超越现有方法,最高提升6.2%
  • 适合做图文图学习、需要文本增强的研究者

图对比学习(GCL)是一种强大的自监督图学习范式,已广泛应用于多种场景。然而,针对文本属性图(TAGs)的GCL研究尚未深入。传统增强方法如特征掩码无法直接处理文本属性。简单将文本通过语言模型编码为向量再输入GCL模块,存在三大挑战:一、信息丢失;二、文本编码阶段语义损失;三、隐含增强约束导致结果不可控且难以理解。本文提出LATEX-GCL框架,利用大语言模型(LLMs)生成文本增强,并借助其强大的自然语言处理能力解决上述问题,为在文本属性图上应用GCL铺平道路。在四个高质量的TAG数据集上的实验表明,该方法显著优于现有基线。代码与数据集已公开,便于复现。

原文摘要 · Abstract (English)

Graph Contrastive Learning (GCL) is a potent paradigm for self-supervised graph learning that has attracted attention across various application scenarios. However, GCL for learning on Text-Attributed Graphs (TAGs) has yet to be explored. Because conventional augmentation techniques like feature embedding masking cannot directly process textual attributes on TAGs. A naive strategy for applying GCL to TAGs is to encode the textual attributes into feature embeddings via a language model and then feed the embeddings into the following GCL module for processing. Such a strategy faces three key challenges: I) failure to avoid information loss, II) semantic loss during the text encoding phase, and III) implicit augmentation constraints that lead to uncontrollable and incomprehensible results. In this paper, we propose a novel GCL framework named LATEX-GCL to utilize Large Language Models (LLMs) to produce textual augmentations and LLMs' powerful natural language processing (NLP) abilities to address the three limitations aforementioned to pave the way for applying GCL to TAG tasks. Extensive experiments on four high-quality TAG datasets illustrate the superiority of the proposed LATEX-GCL method. The source codes and datasets are released to ease the reproducibility, which can be accessed via this link: https://anonymous.4open.science/r/LATEX-GCL-0712.

图学习大模型文本增强对比学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。