用残差量化让知识图谱嵌入与大模型更好结合，提升推理性能。
ReaLM: Residual Quantization Bridging Knowledge Graph Embeddings and Large Language Models
- 通过残差向量量化将知识图谱嵌入转为离散码序列，融入大模型词汇表
- 在两个基准数据集上达到当前最佳性能，显著提升知识补全准确率
- 适合需要融合结构化知识与语言理解的AI系统开发者
大语言模型（LLMs）在知识图谱补全（KGC）中展现出强大推理与泛化能力，超越传统嵌入方法。然而，现有基于LLM的方法难以充分利用结构化语义表示，因为预训练知识图谱模型的连续嵌入空间与大模型的离散标记空间存在根本性错位，阻碍了语义有效传递并限制性能表现。为此，我们提出ReaLM，一种新型有效框架，通过残差向量量化机制实现知识图谱嵌入与大模型分词之间的桥梁连接。ReaLM将预训练的知识图谱嵌入离散化为紧凑码序列，并作为可学习标记集成至大模型词汇表中，实现符号化与上下文知识的无缝融合。此外，引入本体引导的类别约束以保证语义一致性，基于类别级兼容性优化实体预测。在两个广泛使用的基准数据集上的大量实验表明，ReaLM实现了最先进性能，验证了其在对齐结构化知识与大规模语言模型方面的有效性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have recently emerged as a powerful paradigm for Knowledge Graph Completion (KGC), offering strong reasoning and generalization capabilities beyond traditional embedding-based approaches. However, existing LLM-based methods often struggle to fully exploit structured semantic representations, as the continuous embedding space of pretrained KG models is fundamentally misaligned with the discrete token space of LLMs. This discrepancy hinders effective semantic transfer and limits their performance. To address this challenge, we propose ReaLM, a novel and effective framework that bridges the gap between KG embeddings and LLM tokenization through the mechanism of residual vector quantization. ReaLM discretizes pretrained KG embeddings into compact code sequences and integrates them as learnable tokens within the LLM vocabulary, enabling seamless fusion of symbolic and contextual knowledge. Furthermore, we incorporate ontology-guided class constraints to enforce semantic consistency, refining entity predictions based on class-level compatibility. Extensive experiments on two widely used benchmark datasets demonstrate that ReaLM achieves state-of-the-art performance, confirming its effectiveness in aligning structured knowledge with large-scale language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。