arXiv:2601.22657cs.CLcs.AI2026-01

让语言模型直接理解图结构,无需额外网络

NAG: A Unified Native Architecture for Encoder-free Text-Graph Modeling in Language Models

  • 用自注意力机制内化图结构处理,统一文本与拓扑建模
  • 在多个图任务上达到先进性能,且无需外部图神经网络
  • 适合需要高效融合文本与图数据的研究者

现有将图结构融入语言模型的方法通常采用分离架构:外部图神经网络(GNN)编码结构拓扑,语言模型(LM)处理文本语义。我们指出这种设计对文本-图任务不理想,因它造成概念割裂的交互模式。通过分离结构编码与语义处理,系统需在抽象图标记与具体文本元素间进行复杂隐式对齐。为挑战外部编码器的必要性,我们提出NAG(Native Architecture for Graphs),一种将图处理内置于语言模型原生空间的统一框架。NAG利用自注意力机制强制拓扑依赖关系,并重设位置编号以确保结构等价性,使模型能同时理解节点/边内容与整体拓扑。我们提出两种高效实现:NAG-Zero保持基础模型语言能力绝对不变;NAG-LoRA增强结构适应性。在多种图任务上的实验表明,NAG在无需外部编码器的前提下实现稳健的图理解,提供更简洁、连贯的文本-图建模范式。

原文摘要 · Abstract (English)

Prevailing methods for integrating graphs into Language Models (LMs) typically rely on a segregated architecture: external Graph Neural Networks (GNNs) encode structural topology, while LMs process textual semantics. We argue this approach is suboptimal for text-graphs: it creates a conceptually disjointed interaction paradigm. By segregating structural encoding from semantic processing, these systems must perform a complex implicit alignment between abstract graph tokens and concrete textual elements. Challenging the necessity of external encoders, we propose NAG (Native Architecture for Graphs), a unified framework that internalizes graph processing within the LM's native manifold. Instead of bridging disparate embedding spaces, NAG repurposes the self-attention mechanism to enforce topological dependencies and recalibrates positional IDs to ensure structural equivalence. This allows the model to harness its intrinsic linguistic capability to simultaneously comprehend node and edge content alongside structural topology. We introduce two efficient implementations: NAG-Zero for absolute preservation of the base model's linguistic capabilities, and NAG-LoRA for enhanced structural adaptation. Experiments across diverse graph tasks validate that NAG achieves robust graph comprehension without the overhead of external encoders, offering a simpler, more coherent paradigm for text-graph modeling.

图神经网络语言模型统一架构文本图

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。