让大模型自己检查数学答案,准确率最高提升8.3%。
AMTFV: Agentic Mathematical Tool-Flow Verification for LLM Self-Correction
- 用工具流分离验证思路与执行,支持精确计算
- 在5个数据集上平均准确率超越基线,最高提8.3个百分点
- 适合需要高精度数学推理的AI系统开发者
大型语言模型虽具备强大的数学求解能力,但其候选答案的可靠验证仍具挑战。现有方法多依赖自然语言反思或直接生成验证程序,前者难以保障精确计算,后者过早耦合数学建模与底层实现。本文提出AMTFV(Agentic Mathematical Tool-Flow Verification),引入数学工具流(MTF)作为中断-执行-恢复接口,将验证建模与具体执行解耦,并通过数学工具箱实现精确计算。验证代理首先构建验证流程,将需可靠执行的数学对象与计算意图编码为MTF请求,发送至数学工具箱代理。后者解析请求,生成可执行调用并分发至后端进行精确计算。工具输出用于候选答案判定、答案修正及验证流程优化。我们在五个具有挑战性的数学推理数据集上,使用DeepSeek、GPT和Gemini的七种模型配置进行评估。结果表明,AMTFV整体优于本研究中评估的代表性基线;在单个模型配置下,平均准确率相较最强基线最高提升8.3个百分点,且在中高复杂度样本上增益更显著。
原文摘要 · Abstract (English)
Large language models have demonstrated strong mathematical problem-solving capabilities, yet reliably verifying their candidate answers remains challenging. Existing representative methods mainly revise outputs through natural-language reflection or assist verification by directly generating verification programs; the former may not reliably support exact computation, whereas the latter prematurely couples mathematical modeling with low-level implementation. We propose AMTFV (Agentic Mathematical Tool-Flow Verification). By introducing Mathematical Tool Flow (MTF) as an interrupt--execute--resume interface, AMTFV decouples verification modeling from concrete execution and supports exact computation through a mathematical toolbox. Specifically, the verification agent first constructs a verification workflow, encodes the mathematical objects and computational intent requiring reliable execution in an MTF request, and sends it to the mathematical toolbox agent. The latter parses the request, generates executable calls, and dispatches them to the backend for exact computation. Tool outputs then support candidate-answer adjudication, answer revision, and verification-workflow revision. We evaluate AMTFV on five challenging mathematical reasoning datasets with seven model configurations from DeepSeek, GPT, and Gemini. Experimental results show that AMTFV outperforms the representative baselines evaluated in this study overall; under an individual model configuration, it improves average accuracy over the strongest baseline by up to 8.3 percentage points, with larger gains on samples of medium and high verification complexity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。