用简洁几何描述让大模型搞定平面几何题,效果媲美顶级多模态模型。
Concise Geometric Description as a Bridge: Unleashing the Potential of LLM for Plane Geometry Problem Solving
- 将图形转为简洁文本描述,让现成大模型直接推理。
- 仅用5.5k数据训练,性能超越多个开源和闭源多模态模型。
- 设计匹配奖励机制,提升几何描述生成精度,适合教育与AI推理研究者。
平面几何问题求解(PGPS)是一项多模态推理任务,需基于几何图示和文字描述解决几何问题。尽管大语言模型(LLM)具备强大推理能力,但其无法直接处理视觉图示。现有方法通常在大规模PGPS数据上端到端微调多模态大模型(MLLM),以同时增强视觉理解与推理,但可能损害基础LLM的推理能力。本文发现,若将视觉信息恰当转化为文本描述,大模型本身即可成为强效求解器。为此,我们训练一个MLLM解释器生成几何描述(采用条件声明语言CDL,因其简洁性利于训练),再由现成的LLM执行推理。解释器通过思维链增强的SFT结合GRPO进行微调,生成CDL;我们设计了基于CDL匹配的奖励机制,替代传统以解法结果为基准的奖励,提供更直接、密集的指导。为支持训练,我们构建新数据集Formalgeo7k-Rec-CoT,通过人工审核Formalgeo7k v2并加入思维链标注。在Formalgeo7k-Rec-CoT、Unigeo和MathVista上的实验表明,本方法仅用5.5k数据即达到领先开源与闭源多模态模型的性能。
原文摘要 · Abstract (English)
Plane Geometry Problem Solving (PGPS) is a multimodal reasoning task that aims to solve a plane geometric problem based on a geometric diagram and problem textual descriptions. Although Large Language Models (LLMs) possess strong reasoning skills, their direct application to PGPS is hindered by their inability to process visual diagrams. Existing works typically fine-tune Multimodal LLMs (MLLMs) end-to-end on large-scale PGPS data to enhance visual understanding and reasoning simultaneously. However, such joint optimization may compromise base LLMs' inherent reasoning capability. In this work, we observe that LLM itself is potentially a powerful PGPS solver when appropriately formulating visual information as textual descriptions. We propose to train a MLLM Interpreter to generate geometric descriptions for the visual diagram, and an off-the-shelf LLM is utilized to perform reasoning. Specifically, we choose Conditional Declaration Language (CDL) as the geometric description as its conciseness eases the MLLM Interpreter training. The MLLM Interpreter is fine-tuned via CoT (Chain-of-Thought)-augmented SFT followed by GRPO to generate CDL. Instead of using a conventional solution-based reward that compares the reasoning result with the ground-truth answer, we design CDL matching rewards to facilitate more effective GRPO training, which provides more direct and denser guidance for CDL generation. To support training, we construct a new dataset, Formalgeo7k-Rec-CoT, by manually reviewing Formalgeo7k v2 and incorporating CoT annotations. Extensive experiments on Formalgeo7k-Rec-CoT, Unigeo, and MathVista show our method (finetuned on only 5.5k data) performs favorably against leading open-source and closed-source MLLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。