用符号化方法动态生成数学题,让小模型更会解题。
Adaptive Problem Generation via Symbolic Representations
- 在符号空间修改题目,控制结构并自动生成答案
- 闭环优化策略使题目难度随模型能力调整,提升解题率
- 适合想提升小模型数学能力的研究者或开发者
我们提出一种基于符号表示的自适应问题生成方法,用于强化学习训练,以提升小型开放权重语言模型在数学任务上的表现。现有数据生成方法依赖开环流程和固定修改,无法根据模型能力动态调整,且直接操作文字题目,难以控制问题结构。为此,我们采用符号化问题空间,将每个问题表示为符号变量与约束集合(如通过 SymPy 或 SMT 框架),实现对问题结构的精确控制、自动生成真值解,并将数学推理与语言表达解耦。该方法还带来更丰富的题目多样性。进一步地,我们设计闭环框架,通过符号空间中的提示优化学习修改策略,使问题难度可随模型表现自适应调整。实验表明,符号化修改与自适应生成均显著提升了模型的数学求解能力。
原文摘要 · Abstract (English)
We present a method for generating training data for reinforcement learning with verifiable rewards to improve small open-weights language models on mathematical tasks. Existing data generation approaches rely on open-loop pipelines and fixed modifications that do not adapt to the model's capabilities. Furthermore, they typically operate directly on word problems, limiting control over problem structure. To address this, we perform modifications in a symbolic problem space, representing each problem as a set of symbolic variables and constraints (e.g., via algebraic frameworks such as SymPy or SMT formulations). This representation enables precise control over problem structure, automatic generation of ground-truth solutions, and decouples mathematical reasoning from linguistic realization. We also show that this results in more diverse generations. To adapt the problem difficulty to the model, we introduce a closed-loop framework that learns modification strategies through prompt optimization in symbolic space. Experimental results demonstrate that both adaptive problem generation and symbolic representation modifications contribute to improving the model's math solving ability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。