通过提问增强提升大模型推理能力，显著改善数学难题解题效果。
QuestA: Expanding Reasoning Capacity in LLMs via Question Augmentation
- 训练时引入部分解法降低题目难度，提供更有效的学习信号。
- 在AIME24等数据集上达72.50%准确率，较基线提升10.73%。
- 适用于希望提升开源模型推理能力的研究者和开发者。
强化学习（RL）已成为训练大语言模型（LLM）进行推理任务的核心范式。然而近期研究质疑其能否在基础模型之上有效激励推理能力的提升。为应对这一挑战，本文提出一种简单但高效的方法——提问增强（QuestA）：在训练中引入部分解法，以降低问题难度并提供更丰富的学习信号。该方法在数学推理任务的RL训练中应用后，不仅提升了pass@1指标，尤其在标准RL难以进步的问题上显著改善了pass@k表现。通过持续优化，超越DeepScaleR与OpenMath Nemotron等强开源模型。使用1.5B参数模型，在数学基准测试中取得新纪录：AIME24达72.50%（+10.73%）、AIME25达62.29%（+12.79%）、HMMT25达41.67%（+10.11%）。代码、数据与模型已公开于https://github.com/foreverlasting1202/QuestA。
原文摘要 · Abstract (English)
Reinforcement learning (RL) has emerged as a central paradigm for training large language models (LLMs) in reasoning tasks. Yet recent studies question RL's ability to incentivize reasoning capacity beyond the base model. This raises a key challenge: how can RL be adapted to solve harder reasoning problems more effectively? To address this challenge, we propose a simple yet effective strategy via Question Augmentation: introduce partial solutions during training to reduce problem difficulty and provide more informative learning signals. Our method, QuestA, when applied during RL training on math reasoning tasks, not only improves pass@1 but also pass@k-particularly on problems where standard RL struggles to make progress. This enables continual improvement over strong open-source models such as DeepScaleR and OpenMath Nemotron, further enhancing their reasoning capabilities. We achieve new state-of-the-art results on math benchmarks using 1.5B-parameter models: 72.50% (+10.73%) on AIME24, 62.29% (+12.79%) on AIME25, and 41.67% (+10.11%) on HMMT25. Code, data and model are available at https://github.com/foreverlasting1202/QuestA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。