用大模型指导压缩时序知识图谱模型,提升效率与准确率。
LLM-Guided Knowledge Distillation for Temporal Knowledge Graph Reasoning
- 引入大模型作为辅助教师,提供时间相关知识增强训练
- 在多个数据集上实现更优链接预测性能,模型更轻量
- 适合需要高效推理的时序知识图谱应用
时序知识图谱(TKG)支持随时间演化的事实推理,但现有先进模型通常计算复杂、部署成本高。传统压缩与蒸馏方法主要针对静态图设计,直接用于时序场景可能忽略时间依赖关系,导致性能下降。本文提出一种面向时序知识图谱推理的LLM辅助蒸馏框架。除了常规的高容量时序教师外,还引入大语言模型作为辅助指导者,提供丰富背景知识和时序信号,使轻量学生模型能在不增加推理开销的情况下更好建模事件动态。通过分阶段对齐策略,联合优化监督与蒸馏目标进行训练。在多个公共TKG基准上的实验表明,该方法在多种骨干架构下均显著优于强基线蒸馏模型,同时保持紧凑高效的模型结构。结果验证了大语言模型作为有效教师,在传递时序推理能力至资源受限系统中的潜力。
原文摘要 · Abstract (English)
Temporal knowledge graphs (TKGs) support reasoning over time-evolving facts, yet state-of-the-art models are often computationally heavy and costly to deploy. Existing compression and distillation techniques are largely designed for static graphs; directly applying them to temporal settings may overlook time-dependent interactions and lead to performance degradation. We propose an LLM-assisted distillation framework specifically designed for temporal knowledge graph reasoning. Beyond a conventional high-capacity temporal teacher, we incorporate a large language model as an auxiliary instructor to provide enriched supervision. The LLM supplies broad background knowledge and temporally informed signals, enabling a lightweight student to better model event dynamics without increasing inference-time complexity. Training is conducted by jointly optimizing supervised and distillation objectives, using a staged alignment strategy to progressively integrate guidance from both teachers. Extensive experiments on multiple public TKG benchmarks with diverse backbone architectures demonstrate that the proposed approach consistently improves link prediction performance over strong distillation baselines, while maintaining a compact and efficient student model. The results highlight the potential of large language models as effective teachers for transferring temporal reasoning capability to resource-efficient TKG systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。