用知识图谱增强大模型推理,提升实体理解与逻辑一致性
Knowledge Graph-Infused Fine-Tuning for Structured Reasoning in Large Language Models
- 将知识图谱嵌入预训练模型,融合结构化语义与上下文表示
- 在实体识别、问答等任务上显著提升准确率,改善逻辑连贯性
- 适合需要复杂推理和精准实体理解的场景,如智能客服、知识问答
本文针对大语言模型在处理需结构化知识的任务时存在的推理链缺失和实体级语义理解不足问题,提出一种基于知识图谱注入的微调算法框架。该方法在预训练语言模型基础上,引入结构化图信息进行辅助学习,利用图神经网络编码实体及其关系,构建基于图的语义表示,并设计融合机制联合建模知识图谱嵌入与语言模型的上下文表示。为增强知识整合的鲁棒性,引入门控机制动态平衡语言语义与结构知识的贡献,有效缓解不同表征空间间的冲突。训练时采用联合损失函数,兼顾任务性能与结构对齐目标,显著提升实体预测准确率与语义推理能力。通过系统敏感性实验,验证了学习率、图覆盖度及结构扰动对模型表现的影响,结果表明该方法在实体识别、问答和语言生成等任务中具有优异的性能与稳定性。
原文摘要 · Abstract (English)
This paper addresses the problems of missing reasoning chains and insufficient entity-level semantic understanding in large language models when dealing with tasks that require structured knowledge. It proposes a fine-tuning algorithm framework based on knowledge graph injection. The method builds on pretrained language models and introduces structured graph information for auxiliary learning. A graph neural network is used to encode entities and their relations, constructing a graph-based semantic representation. A fusion mechanism is then designed to jointly model the knowledge graph embeddings with the contextual representations from the language model. To enhance the robustness of knowledge integration, a gating mechanism is introduced to dynamically balance the contributions of linguistic semantics and structural knowledge. This effectively mitigates conflicts between different representational spaces. During training, a joint loss function is constructed to account for both task performance and structural alignment objectives. This helps improve the accuracy of entity prediction and semantic reasoning. The study also includes a series of systematic sensitivity experiments. It evaluates the effects of learning rate, graph coverage, and structural perturbations on model performance. The results further validate the effectiveness and stability of the proposed method across tasks such as entity recognition, question answering, and language generation. Experimental findings show that the proposed structure-aware fine-tuning framework significantly enhances the model's ability to represent complex semantic units. It demonstrates better semantic consistency and contextual logic modeling in scenarios involving structural reasoning and entity extraction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。