小模型也能精准检索知识图谱子图，性能媲美大模型。
Less is More: Making Smaller Language Models Competent Subgraph Retrievers for Multi-hop KGQA
- 用小语言模型把子图检索转为条件生成任务，仅需220M参数
- 220M模型检索效果接近7B大模型，3B模型在双基准上达新SOTA
- 适合追求高效、低资源的多跳知识图谱问答应用
检索增强生成（RAG）被广泛用于将外部非参数化知识注入大语言模型（LLM）。近期研究表明，知识图谱（KG）包含对LLM有价值的外部知识。从KG中检索信息不同于从文档集合中提取信息。现有方法大多直接检索相关子图，避免了传统语义解析方法所需的大量SPARQL标注。本文将子图检索任务建模为由小型语言模型处理的条件生成任务。具体而言，我们定义子图标识符为一系列关系序列，每个关系以特殊标记形式存储在语言模型中。我们的基础生成式子图检索模型仅含220M参数，其检索性能与依赖7B参数的顶尖模型相当，证明小语言模型具备执行子图检索任务的能力。此外，我们最大的3B模型在接入LLM阅读器后，在WebQSP和CWQ两个基准上实现了新的端到端性能纪录。模型与数据将公开：https://github.com/hwy9855/GSR。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) is widely used to inject external non-parametric knowledge into large language models (LLMs). Recent works suggest that Knowledge Graphs (KGs) contain valuable external knowledge for LLMs. Retrieving information from KGs differs from extracting it from document sets. Most existing approaches seek to directly retrieve relevant subgraphs, thereby eliminating the need for extensive SPARQL annotations, traditionally required by semantic parsing methods. In this paper, we model the subgraph retrieval task as a conditional generation task handled by small language models. Specifically, we define a subgraph identifier as a sequence of relations, each represented as a special token stored in the language models. Our base generative subgraph retrieval model, consisting of only 220M parameters, achieves competitive retrieval performance compared to state-of-the-art models relying on 7B parameters, demonstrating that small language models are capable of performing the subgraph retrieval task. Furthermore, our largest 3B model, when plugged with an LLM reader, sets new SOTA end-to-end performance on both the WebQSP and CWQ benchmarks. Our model and data will be made available online: https://github.com/hwy9855/GSR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。