用强化学习让小模型学会精准画辅助线解几何题。
GeometryZero: Advancing Geometry Solving via Group Contrastive Policy Optimization
- 通过上下文判断构造动作好坏，只奖励有用操作。
- 在几何数据集上准确率超基线模型，且用更小模型。
- 适合想低成本做几何推理的开发者或研究者。
大语言模型在数学推理上进展迅速，但几何问题仍具挑战性，常需辅助线构造。现有方法或性能不足，或依赖超大规模模型（如GPT-4o），成本高昂。我们提出一种基于可验证奖励的强化学习框架——分组对比策略优化（GCPO），包含两个部分：(1) 分组对比掩码，根据上下文效用区分正负构造奖励；(2) 长度奖励，鼓励更长的推理链。在此基础上构建了GeometryZero，一套可负担的几何推理模型家族，能选择性使用辅助线构造。在Geometry3K和MathVista数据集上的实验表明，GeometryZero持续优于其他强化学习基线（如GRPO、ToRL）。代码已开源于https://github.com/ekonwang/GeometryZero。
原文摘要 · Abstract (English)
Recent progress in large language models (LLMs) has boosted mathematical reasoning, yet geometry remains challenging where auxiliary construction is often essential. Prior methods either underperform or depend on very large models (e.g., GPT-4o), making them costly. We argue that reinforcement learning with verifiable rewards (e.g., GRPO) can train smaller models to couple auxiliary construction with solid geometric reasoning. However, naively applying GRPO yields unconditional rewards, encouraging indiscriminate and sometimes harmful constructions. We propose Group Contrastive Policy Optimization (GCPO), an RL framework with two components: (1) Group Contrastive Masking, which assigns positive/negative construction rewards based on contextual utility, and (2) a Length Reward that encourages longer reasoning chains. On top of GCPO, we build GeometryZero, an affordable family of geometry reasoning models that selectively use auxiliary construction. Experiments on Geometry3K and MathVista show GeometryZero consistently outperforms RL baselines (e.g., GRPO, ToRL). The code has been available at https://github.com/ekonwang/GeometryZero.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。