通过多思维链投票提升大模型几何推理能力
Beyond Symbolic Solving: Multi Chain-of-Thought Voting for Geometric Reasoning in Large Language Models
- 生成多条并行推理路径并用代码验证数值结果
- 8条推理路径时在Geometry3K上达88.8%准确率
- 适合需要高精度数学推理的AI研究者
几何问题求解(GPS)是提升大语言模型数学推理能力的核心挑战,因其需结合图示理解、符号运算与逻辑推理。现有方法主要聚焦于图文描述对齐与求解,多采用神经、符号或神经符号方法,但仅解决了图示理解与符号运算,逻辑推理仍依赖单一思维链(CoT)。为此,本文提出MARS-GPS:生成多条并行推理路径,结合Python代码执行进行数值验证,利用分词级熵作为置信度信号进行排序,并通过多阶段投票与自验证机制聚合答案。实验表明,8条并行推理路径下,MARS-GPS在Geometry3K上达到88.8%准确率,较先前最佳方法提升近11%;且随着推理路径数从1增至16,准确率持续上升,在消融子集上提升6.0%。代码与数据已公开于匿名仓库:https://anonymous.4open.science/r/MARS-GPS-DE55。
原文摘要 · Abstract (English)
Geometric Problem Solving (GPS) remains at the heart of enhancing mathematical reasoning in large language models because it requires the combination of diagrammatic understanding, symbolic manipulation and logical inference. In existing literature, researchers have chiefly focused on synchronising the diagram descriptions with text literals and solving the problem. In this vein, they have either taken a neural, symbolic or neuro-symbolic approach. But this solves only the first two of the requirements, namely diagrammatic understanding and symbolic manipulation, while leaving logical inference underdeveloped. The logical inference is often limited to one chain-of-thought (CoT). To address this weakness in hitherto existing models, this paper proposes MARS-GPS, that generates multiple parallel reasoning rollouts augmented with Python code execution for numerical verification, ranks them using token-level entropy as a confidence signal, and aggregates answers through a multi-stage voting and self-verification pipeline. Empirical results show that MARS-GPS with 8 parallel rollouts achieves 88.8% on Geometry3K, a nearly +11% improvement over the prior state-of-the-art, with accuracy scaling consistently as the number of rollouts increases from 1 to 16 (+6.0% on ablation subset). We provide our code and data in an anonymous repository: https://anonymous.4open.science/r/MARS-GPS-DE55.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。