用新数据集提升大模型生成图文本的能力，解决复杂图结构理解难题。
Evaluating and Improving Graph to Text Generation with Large Language Models
- 设计新样本选择策略，优化少样本提示效果
- 构建标注重排与归因任务的PlanGTG数据集，提升生成质量
- 适合研究知识图谱文本生成、大模型推理能力的学者
大语言模型在众多任务中展现出巨大潜力，但在解析图结构方面的研究仍有限。为此，我们全面评估了当前开源大模型在图到文本生成任务上的表现。尽管探索了最优提示策略并提出基于多样性和难度的少样本样本选择方法，但发现无需调参的方法提升有限，因大模型在复杂图（尤其三元组较多时）上规划能力不足。为进一步提升模型在图序列规划与事实对齐方面的能力，我们引入新数据集PlanGTG，包含重排和归因两个子任务。通过大量自动与人工评估，证明使用PlanGTG数据集在少样本学习与微调下均显著提升生成文本质量。本研究为图到文本生成开辟了新方向，PlanGTG数据集已开源于https://github.com/probe2/kg_text。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated immense potential across various tasks. However, research for exploring and improving the capabilities of LLMs in interpreting graph structures remains limited. To address this gap, we conduct a comprehensive evaluation of prompting current open-source LLMs on graph-to-text generation tasks. Although we explored the optimal prompting strategies and proposed a novel and effective diversity-difficulty-based few-shot sample selection method, we found that the improvements from tuning-free approaches were incremental, as LLMs struggle with planning on complex graphs, particularly those with a larger number of triplets. To further improve LLMs in planning with graph sequences and grounding in truth, we introduce a new graph-to-text dataset, PlanGTG, annotated with two sub-tasks: reordering and attribution. Through extensive automatic and human evaluations, we demonstrate significant improvements in the quality of generated text from both few-shot learning and fine-tuning perspectives using the PlanGTG dataset. Our study paves the way for new research directions in graph-to-text generation. PlanGTG datasets can be found in https://github.com/probe2/kg_text.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。