让大模型用Prolog做逻辑推理,更准且可检查。
PrologMCP: A Standardized Prolog Tool Interface for LLM Agents
- 用PrologMCP接口让大模型调用逻辑求解器,实现可复用的推理流程。
- 在复杂推理任务上准确率接近1.0,远超普通模型(0.762)。
- 适合需要高可靠性推理的智能体,如数学证明或规则系统。
前沿推理优化的语言模型在深度演绎任务中仍会失败,而通过延长内部推理来提升性能的成本过高。符号化委托提供了一种互补路径:语言模型负责问题翻译,求解器执行推理。然而,当前用于逻辑编程的自动形式化管道通常是针对特定任务或智能体的定制集成。我们提出PrologMCP,一个与任务无关的开源服务器,通过模型上下文协议(MCP)将Prolog作为有状态工具暴露。其简洁的工具接口、结构化的错误报告和每会话隔离机制,使‘翻译-运行-检查-修复’循环成为MCP兼容智能体的可复用基础。我们在PARARULE-Plus的两个子集上评估了增强后的形式化智能体:通用样本和针对自然语言推理特定缺陷的挑战样本。在通用样本上,形式化智能体准确率达到1.00,与推理型大模型(Claude Sonnet 4.6, GPT-4.1, o4-mini)持平或超越(分别为1.00 / 0.998),对标准模型的提升最大达0.762(GPT-4.1)。在挑战样本上,形式化智能体保持近完美表现(1.00 / 0.99),而推理型大模型下降至0.95 / 0.94。结果表明,通过MCP将推理委派给Prolog,是比延长自然语言推理更鲁棒、可检查的替代方案。
原文摘要 · Abstract (English)
Frontier reasoning-tuned language models still fail on deductive tasks at depth, and the cost of improved performance through extended internal reasoning scales poorly. Symbolic delegation offers a complementary route: a language model translates the problem, while a solver performs the inference. However, current autoformalization pipelines for logic programming are typically bespoke integrations tied to particular tasks or agents. We introduce PrologMCP, a task-agnostic, open-source server that exposes Prolog as a stateful tool through the Model Context Protocol (MCP). Its compact tool interface, structured error reporting, and per-session isolation make the translate-run-inspect-repair loop a reusable primitive for MCP-capable agents. We evaluate a formalizer agent enhanced with PrologMCP against standard and reasoning LLMs (Claude Sonnet 4.6, GPT-4.1, and o4-mini) on two subsets of PARARULE-Plus: a general-purpose sample and a more challenging one targeting a specific failure mode of natural-language reasoning. On the general sample, the formalizer matches or exceeds reasoning LLMs (accuracy 1.00 vs.\ 1.00 / 0.998), with the largest gains over standard models (0.762 for GPT-4.1). On the challenging subset, the formalizer remains near-perfect (1.00 / 0.99) while reasoning LLMs drop to 0.95 / 0.94. These results suggest that delegating inference to Prolog via MCP is a robust and inspectable alternative to extended natural-language reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。