通过重写数学题型提升大模型解题能力
RM-PoT: Reformulating Mathematical Problems and Solving via Program of Thoughts
- 先重写题目表面形式,减少模型偏见
- 从题库中检索5个相似例题提供参考
- 生成可执行代码确保计算准确
近期大语言模型在解决复杂数值推理任务时取得了显著进展,但问题的表述方式对模型表现影响巨大。我们发现,仅改变数学题的表面形式,就会显著影响答案分布和求解率,暴露出大模型在复杂推理中的脆弱性。为此,本文提出RM-PoT框架,包含三个阶段:问题重写(RM)以消除结构偏见;从预构建的领域特定题库中检索5个语义对齐的示例提供上下文引导;最后生成可执行的Python代码进行精确计算。该方法有效提升了模型在多样化问题上的鲁棒性和准确率。
原文摘要 · Abstract (English)
Recently, substantial advancements have been made in training language models to carry out step-by-step reasoning for solving intricate numerical reasoning tasks. Beyond the methods used to solve these problems, the structure and formulation of the problems themselves also play a crucial role in determining the performance of large language models. We observe that even small changes in the surface form of mathematical problems can have a profound impact on both the answer distribution and solve rate. This highlights the vulnerability of LLMs to surface-level variations, revealing its limited robustness when reasoning through complex problems. In this paper, we propose RM-PoT, a three-stage framework that integrates problem reformulation (RM), code-aided reasoning (PoT), and domain-aware few-shot learning to address these limitations. Our approach first reformulates the input problem into diverse surface forms to reduce structural bias, then retrieves five semantically aligned examples from a pre-constructed domain-specific question bank to provide contextual guidance, and finally generates executable Python code for precise computation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。