用可验证的分解方法提升模型解数学题能力，效果显著
VERIFY-RL: Verifiable Recursive Decomposition for Reinforcement Learning in Mathematical Reasoning
- 基于微分规则构建可验证的子问题分解框架
- 复杂度下降+解可包含+形式推导三条件确保有效性
- 淘汰无效分解后难题准确率从32%升至68%
训练语言模型解决复杂数学问题时，课程学习通过逐步处理简单子问题带来收益。但现有分解方法多为启发式，无法保证子问题更简单、解它们有助于父任务，或关系具有数学依据。我们发现符号微分天然提供了可验证的分解结构：微积分规则明确定义表达式如何简化为更小成分，并具备可证明性质。本文提出Verify-RL框架，要求每个父子分解满足三个可验证条件：结构复杂度严格降低、解包含性、形式规则推导。与启发式方法中大量无效分解不同，本框架可通过符号计算实现自动验证，实现“构造即验证”。实验表明，剔除无效分解后，最难问题准确率从32%提升至68%，整体相对提升40%。
原文摘要 · Abstract (English)
Training language models to solve complex mathematical problems benefits from curriculum learning progressively training on simpler subproblems. However, existing decomposition methods are often heuristic, offering no guarantees that subproblems are simpler, that solving them aids the parent task, or that their relationships are mathematically grounded. We observe that symbolic differentiation provides a natural structure for verified decomposition: calculus rules explicitly define how expressions reduce to simpler components with provable properties. We introduce Verify-RL, a framework where every parent-child decomposition satisfies three verifiable conditions: strictly decreasing structural complexity, solution containment, and formal rule derivation. Unlike heuristic methods where a significant fraction of decompositions are invalid our properties admit automatic verification through symbolic computation, achieving "verification by construction" Experiments demonstrate that eliminating invalid decompositions yields sizable gains, accuracy on the hardest problems more than doubles from 32% to 68%, with a 40% relative improvement overall.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。