arXiv:2606.10461cs.LGcs.AI2026-06中稿 · ICML

用能量模型对齐图神经网络与大模型的文本表示,提升跨任务表现。

ERAlign: Energy-based Representation Alignment of GNNs and LLMs on Text-attributed Graphs

论文配图:ERAlign: Energy-based Representation Alignment of GNNs and LLMs on Text-attributed Graphs
图 1 · 摘自论文原文
  • 构建能量模型,在共享空间中对齐图结构与文本嵌入
  • 在8个数据集上实现当前最优性能,支持弱监督与跨任务迁移
  • 提出能量差异机制,降低训练开销并保证优化稳定性

文本属性图(TAGs)通过结合文本节点属性与图结构来描述丰富的关系语义。近期将图神经网络(GNN)与大语言模型(LLM)融合的研究在处理TAGs上展现出潜力,但实现良好对齐的表示仍具挑战。以往方法多依赖启发式策略,进行粗粒度匹配,缺乏充分约束且忽略分布对齐,导致表示漂移和泛化能力受限。本文基于能量模型(EBM),提出能量表示对齐(ERAlign)框架,将GNN编码的图结构与LLM生成的文本嵌入投影至共享隐空间,实现分布一致性。具体而言,通过距离度量量化逐层对齐,并以EBM目标函数优化。通过降低能量值,获得适用于下游任务的良好对齐表示。训练中引入能量差异(ED),避免难以计算的归一化带来的高采样成本,同时具备更高的训练效率和更少的能量景观扭曲的理论保障。在八个TAG数据集上的实证评估表明,ERAlign在不同监督水平和跨任务迁移场景下均达到当前最优性能。

原文摘要 · Abstract (English)

Text-attributed Graphs (TAGs) incorporate textual node attributes with graph structures to describe rich relational semantics. Recent efforts to integrate Graph Neural Networks (GNNs) and Large Language Models (LLMs) have shown promise for learning on TAGs, yet achieving well-aligned representations remains challenging. Prior studies largely rely on heuristics that perform coarse-grained matching. They lack sufficient constraints and ignore distributional alignment, leading to representation drift and limited generalization. Building on Energy-based Models (EBMs), we propose an Energy-based Representation Alignment (ERAlign) framework that projects GNN-encoded graph structure and LLM-derived text embeddings in a shared latent space to achieve distribution consistency. Concretely, layer-wise alignment is quantified by a distance metric and optimized via an EBM objective. By decreasing energy values, our framework yields well-aligned representations for downstream tasks. During training, we introduce Energy Discrepancy (ED) to avoid high sampling costs associated with intractable normalization. ED also carries theoretical guarantees of higher training efficiency and reduced energy landscape distortion. Empirical evaluations on eight TAG datasets demonstrate that ERAlign obtains state-of-the-art performance across varying levels of supervision and cross-task transfer scenarios.

图神经网络大模型对齐文本属性图能量模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。