用知识图谱和符号验证器让大模型数学推理更准确
MedRule-KG: A Knowledge-Graph--Steered Scaffold for Mathematical Reasoning with a Lightweight Verifier
- 构建带类型的知识图谱，嵌入数学规则约束
- 在FDA基准上准确率从76.7%提至100%
- 适合需要高可靠数学推理的医疗或金融场景
大型语言模型常生成看似合理但违反基础数学逻辑的推理。我们提出MedRule-KG，一个轻量级有类型知识图谱结合符号验证器，用于在推理中强制可解释的数学规则。该系统编码实体、关系及三条领域启发式规则，验证器检查预测并进行最小修正以保证一致性。在90个来自FDA的基准测试中，引入MedRule-KG使精确匹配（EM）从0.767提升至0.900，加入验证器后达到1.000的EM且完全消除规则违规。我们展示了MedRule-KG作为安全数学推理通用框架的潜力，并进行消融分析，开源代码与数据以促进可复现性。
原文摘要 · Abstract (English)
Large language models (LLMs) often produce fluent reasoning steps while violating simple mathematical or logical constraints. We introduce MedRule-KG, a compact typed knowledge graph coupled with a symbolic verifier, designed to enforce mathematically interpretable rules in reasoning tasks. MedRule-KG encodes entities, relations, and three domain-inspired rules, while the verifier checks predictions and applies minimal corrections to guarantee consistency. On a 90-example FDA-derived benchmark, grounding in MedRule-KG improves exact match (EM) from 0.767 to 0.900, and adding the verifier yields 1.000 EM while eliminating rule violations entirely. We demonstrate how MedRule-KG provides a general scaffold for safe mathematical reasoning, discuss ablations, and release code and data to encourage reproducibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。