arXiv:2601.00202cs.CL2026-01

用大模型指导小模型,让轻量推理更准更快。

Knowledge Distillation for Temporal Knowledge Graph Reasoning with Large Language Models

  • 大模型做老师,教小模型理解时间动态关系
  • 在多个数据集上准确率超基线,计算开销更低
  • 适合部署在低功耗设备上的实时推理场景

时序知识图谱推理对提升智能决策系统的效率与可靠性至关重要,是未来人工智能应用的关键技术基础。尽管已有进展,现有模型通常参数量大、计算密集,导致硬件成本高、能耗大,难以在资源受限、低功耗、分布式平台上实现实时推理。此外,多数现有模型压缩与蒸馏方法针对静态知识图谱设计,无法有效捕捉时序依赖,常导致推理性能下降。为此,我们提出一种专为时序知识图谱推理设计的蒸馏框架。该方法利用大语言模型作为教师模型,引导学生模型学习结构与时序推理能力,通过融合大规模公开知识与任务特定时序信息,增强学生模型对时序动态的建模能力,同时保持紧凑高效的架构。在多个公开基准数据集上的大量实验表明,该方法持续优于强基线,实现了推理准确率、计算效率与实际可部署性的良好平衡。

原文摘要 · Abstract (English)

Reasoning over temporal knowledge graphs (TKGs) is fundamental to improving the efficiency and reliability of intelligent decision-making systems and has become a key technological foundation for future artificial intelligence applications. Despite recent progress, existing TKG reasoning models typically rely on large parameter sizes and intensive computation, leading to high hardware costs and energy consumption. These constraints hinder their deployment on resource-constrained, low-power, and distributed platforms that require real-time inference. Moreover, most existing model compression and distillation techniques are designed for static knowledge graphs and fail to adequately capture the temporal dependencies inherent in TKGs, often resulting in degraded reasoning performance. To address these challenges, we propose a distillation framework specifically tailored for temporal knowledge graph reasoning. Our approach leverages large language models as teacher models to guide the distillation process, enabling effective transfer of both structural and temporal reasoning capabilities to lightweight student models. By integrating large-scale public knowledge with task-specific temporal information, the proposed framework enhances the student model's ability to model temporal dynamics while maintaining a compact and efficient architecture. Extensive experiments on multiple publicly available benchmark datasets demonstrate that our method consistently outperforms strong baselines, achieving a favorable trade-off between reasoning accuracy, computational efficiency, and practical deployability.

知识图谱模型蒸馏时序推理大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。