让大模型理解结构化输入，提升语义图转文本效果
SAFT: Structure-Aware Fine-Tuning of LLMs for AMR-to-Text Generation
- 用磁拉普拉斯计算方向感知位置编码，注入预训练大模型
- 在AMR 3.0上比基线高3.5的BLEU，复杂图表现更优
- 无需改架构，适用于各类图结构数据，适合做结构化生成
大语言模型（LLMs）在处理图等结构化输入的任务中日益重要。抽象语义表示（AMR）以有向图形式编码丰富语义，是检验模型从结构化输入生成文本能力的理想基准。然而，现有方法常随意线性化AMR，丢失关键结构信息，或依赖与标准LLM不兼容的架构。本文提出SAFT——一种无需改变架构即可将图拓扑注入预训练大模型的结构感知微调方法。通过转换后的AMR的磁拉普拉斯计算方向敏感的位置编码，并将其投影到LLM的嵌入空间。尽管该方法可推广至任意图结构输入，本文聚焦于作为代表性挑战任务的AMR-to-text生成。SAFT在AMR 3.0数据集上取得新最佳性能，相比基线提升3.5的BLEU。性能提升随图复杂度增加而扩大，凸显结构感知表征对增强LLM表现的价值。SAFT为连接结构化数据与语言模型提供了一条通用且高效的新路径。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly applied to tasks involving structured inputs such as graphs. Abstract Meaning Representations (AMRs), which encode rich semantics as directed graphs, offer a rigorous testbed for evaluating LLMs on text generation from such structures. Yet, current methods often arbitrarily linearize AMRs, discarding key structural cues, or rely on architectures incompatible with standard LLMs. We introduce SAFT, a structure-aware fine-tuning approach that injects graph topology into pretrained LLMs without architectural changes. We compute direction-sensitive positional encodings from the magnetic Laplacian of transformed AMRs and project them into the embedding space of the LLM. While possibly applicable to any graph-structured inputs, we focus on AMR-to-text generation as a representative and challenging benchmark. SAFT sets a new state-of-the-art on AMR 3.0 with a 3.5 BLEU improvement over baselines. Gains scale with graph complexity, highlighting the value of structure-aware representations in enhancing LLM performance. SAFT offers a general and effective pathway for bridging structured data and language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。