让大模型自主选择最优推理路径，提升数学科学题解答准确率。
Group Pattern Selection Optimization: Let LRMs Pick the Right Pattern for Reasoning
- 用强化学习让模型从多种推理模式中选最佳路径。
- 在多个基准上显著提升准确率，效果稳定且可迁移。
- 适合想提升模型推理能力的研究者与开发者。
大型推理模型（LRMs）表现出多样化的高层推理模式（如直接求解、反思验证、多方案探索），但现有训练方法隐式偏向少数主导模式。系统分析发现，不同推理模式在数学与科学基准上的准确率差异显著，模型默认模式往往不适用于特定问题。为此，我们提出分组模式选择优化（GPSO），一种基于强化学习的框架，通过多模式回溯、验证器引导的每题最优模式选择，以及优化过程中的注意力掩码，防止显式模式后缀泄露至策略中。通过探索多样化推理策略并优化最有效的模式组合，GPSO使模型能内化问题特征到最优推理模式的映射。大量实验表明，GPSO在多种模型架构与基准上均带来一致且显著的性能提升，有效缓解模式次优性，增强推理鲁棒性与适应性。所有数据与代码已开源：https://github.com/wanghanbinpanda/GPSO。
原文摘要 · Abstract (English)
Large reasoning models (LRMs) exhibit diverse high-level reasoning patterns (e.g., direct solution, reflection-and-verification, and exploring multiple solutions), yet prevailing training recipes implicitly bias models toward a limited set of dominant patterns. Through a systematic analysis, we identify substantial accuracy variance across these patterns on mathematics and science benchmarks, revealing that a model's default reasoning pattern is often sub-optimal for a given problem. To address this, we introduce Group Pattern Selection Optimization (GPSO), a reinforcement learning framework that extends GRPO by incorporating multi-pattern rollouts, verifier-guided optimal pattern selection per problem, and attention masking during optimization to prevent the leakage of explicit pattern suffixes into the learned policy. By exploring a portfolio of diverse reasoning strategies and optimizing the policy on the most effective ones, GPSO enables the model to internalize the mapping from problem characteristics to optimal reasoning patterns. Extensive experiments demonstrate that GPSO delivers consistent and substantial performance gains across various model backbones and benchmarks, effectively mitigating pattern sub-optimality and fostering more robust, adaptable reasoning. All data and codes are available at https://github.com/wanghanbinpanda/GPSO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。