用高质量数据训练轻量模型,实现一次生成知识图谱。
InvertiTune: High-Quality Data Synthesis for Cost-Effective Single-Shot Text-to-Knowledge Graph Generation
- 构建数据生成流水线,从知识库提取子图并生成自然语言描述
- 在CE12k和CrossEval-1200上优于大模型与现有方法
- 适合需要高效、高精度文本转知识图谱的场景
大型语言模型(LLMs)推动了从文本自动构建知识图谱(Text2KG)的发展。然而,许多方法依赖迭代提示,计算成本高且易忽略分散在文本中的复杂关系。为此,我们提出InvertiTune框架,结合受控数据生成与监督微调(SFT)。该框架通过从大规模知识库中系统提取子图,经噪声过滤后,利用LLM生成对应自然语言描述,更契合模型能力。生成的数据集包含更长文本与更大知识图谱,更贴近真实场景,支持轻量模型的单次生成。在自建的CE12k数据集上,InvertiTune超越更大非微调模型及现有先进Text2KG方法,并在由三个基准数据集与CE12k组成的CrossEval-1200上展现更强跨数据集泛化能力。结果表明,真实、高质量训练数据对提升高效高精度Text2KG系统至关重要。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have revolutionized the ability to understand and generate text, enabling significant progress in automatic knowledge graph construction from text (Text2KG). Many Text2KG methods, however, rely on iterative LLM prompting, making them computationally expensive and prone to overlooking complex relations distributed throughout the text. To address these limitations, we propose InvertiTune, a framework that combines a controlled data generation pipeline with supervised fine-tuning (SFT). Within this framework, the data-generation pipeline systematically extracts subgraphs from large knowledge bases, applies noise filtering, and leverages LLMs to generate corresponding natural text descriptions, a task more aligned with LLM capabilities than direct KG generation from text. This pipeline enables generating datasets composed of longer texts paired with larger KGs that better reflect real-world scenarios compared to existing benchmarks, thus supporting effective SFT of lightweight models for single-shot KG construction. Experimental results on CE12k, a dataset generated using the introduced pipeline, show that InvertiTune outperforms larger non-fine-tuned LLMs as well as state-of-the-art Text2KG approaches, while also demonstrating stronger cross-dataset generalization on CrossEval-1200, a test set created from three established benchmark datasets and CE12k. These findings highlight the importance of realistic, high-quality training data for advancing efficient and high-performing Text2KG systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。