让大模型更懂知识图谱结构,提升缺失链接预测准确率
SLiNT: Structure-aware Language Model with Injection and Contrastive Training for Knowledge Graph Completion
- 通过注入图结构信息增强大模型对稀疏实体的感知能力
- 在WN18RR和FB15k-237上超越多数基线模型
- 适合需要高精度链接预测的知识图谱应用
知识图谱中的链接预测需融合结构信息与语义上下文以推断缺失实体。尽管大语言模型具备强大的生成推理能力,但其对结构信号的利用有限,常导致结构稀疏与语义模糊,尤其在不完整或零样本设置下。为此,我们提出SLiNT(结构感知语言模型,结合注入与对比训练),一个模块化框架,通过轻量级LoRA适配,在冻结的LLM主干上注入知识图谱衍生的结构上下文,实现鲁棒的链接预测。具体而言,结构引导邻域增强(SGNE)检索伪邻居以丰富稀疏实体,缓解上下文缺失;动态硬对比学习(DHCL)通过插值硬正负样本引入细粒度监督,解决实体级歧义;梯度解耦双注入(GDDI)在令牌层面进行结构感知干预,同时保持核心LLM参数不变。在WN18RR和FB15k-237上的实验表明,SLiNT在性能上优于或媲美基于嵌入和生成的基线模型,验证了结构感知表征学习在可扩展知识图谱补全中的有效性。
原文摘要 · Abstract (English)
Link prediction in knowledge graphs requires integrating structural information and semantic context to infer missing entities. While large language models offer strong generative reasoning capabilities, their limited exploitation of structural signals often results in structural sparsity and semantic ambiguity, especially under incomplete or zero-shot settings. To address these challenges, we propose SLiNT (Structure-aware Language model with Injection and coNtrastive Training), a modular framework that injects knowledge-graph-derived structural context into a frozen LLM backbone with lightweight LoRA-based adaptation for robust link prediction. Specifically, Structure-Guided Neighborhood Enhancement (SGNE) retrieves pseudo-neighbors to enrich sparse entities and mitigate missing context; Dynamic Hard Contrastive Learning (DHCL) introduces fine-grained supervision by interpolating hard positives and negatives to resolve entity-level ambiguity; and Gradient-Decoupled Dual Injection (GDDI) performs token-level structure-aware intervention while preserving the core LLM parameters. Experiments on WN18RR and FB15k-237 show that SLiNT achieves superior or competitive performance compared with both embedding-based and generation-based baselines, demonstrating the effectiveness of structure-aware representation learning for scalable knowledge graph completion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。