用推理和查询策略从大模型提取知识，构建闭源知识库
REAP: Relation-Aware Elicitation and Parsing for Closed-Book Knowledge Base Construction from LLMs
- 通过结构化推理与特定关系查询，激发模型内部知识
- 在测试集上宏平均F1达0.62，部分关系高达0.95
- 无需微调、参数不超过320亿，适合资源受限场景
我们提出REAP系统，用于2026年AKBC共享任务中在闭源设置下从语言模型构建知识库，参数量上限为320亿，且不进行模型微调。系统结合结构化思维链推理、关系特异性查询策略及基于推理的空集门机制，以激发模型参数知识，并直接提取为有效JSON数组。在测试集上，基于Mistral-Small-24B-Instruct-2501模型的系统取得0.62的宏平均F1，其中countryLandBordersCountry（F1=0.95）、companyTradesAtStockExchange（F1=0.73）和hasArea（F1=0.77）表现尤为突出。代码已公开于https://github.com/yammdd/AKBC-Shared-Task-2026。
原文摘要 · Abstract (English)
We present the REAP system for the AKBC Shared Task 2026 on constructing knowledge bases from language models in a closed-book setting, subject to a budget of at most 32B parameters and no model fine-tuning. Our system combines structured chain-of-thought reasoning, relation-specific query strategies, and a reasoning-based empty-set gate to elicit parametric knowledge, followed by direct extraction into valid JSON arrays. On the test set, the system, built on the Mistral-Small-24B-Instruct-2501 model, achieves a macro-F1 score of 0.62, with particularly strong results on countryLandBordersCountry (F1 = 0.95), companyTradesAtStockExchange (F1 = 0.73), and hasArea (F1 = 0.77). Our code is publicly available at https://github.com/yammdd/AKBC-Shared-Task-2026.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。