通过显式对齐三大推理能力，提升大模型推理的稳定性和可扩展性。
Beyond 'Aha!': Toward Systematic Meta-Abilities Alignment in Large Reasoning Models
- 分三阶段对齐演绎、归纳、类比三种元推理能力
- 相比指令微调基线，性能提升超10%且在多领域持续有效
- 适合需要可靠长链推理的数学、编程与科学任务场景
大型推理模型（LRMs）已具备潜在的长链思维推理能力。以往研究显示，基于结果的强化学习（RL）可偶然激发自我修正、回溯和验证等高级推理行为，常被称为模型的‘顿悟时刻’。然而，这些涌现行为的发生时机与一致性难以预测和控制，限制了LRMs推理能力的可扩展性与可靠性。为此，本文摒弃对提示词和偶然‘顿悟时刻’的依赖，转而通过自验证任务，显式对齐模型的三种元能力：演绎、归纳与类比。采用三阶段流水线——个体对齐、参数空间融合、领域特定强化学习，使7B与32B模型在数学、编码与科学基准上性能相对指令微调基线提升超过10%。此外，从对齐检查点开始的领域特定强化学习进一步提升了性能上限，证明显式元能力对齐能为推理提供可扩展且可靠的底层基础。代码开源于：https://github.com/zhiyuanhubj/Meta-Ability-Alignment
原文摘要 · Abstract (English)
Large reasoning models (LRMs) already possess a latent capacity for long chain-of-thought reasoning. Prior work has shown that outcome-based reinforcement learning (RL) can incidentally elicit advanced reasoning behaviors such as self-correction, backtracking, and verification phenomena often referred to as the model's "aha moment". However, the timing and consistency of these emergent behaviors remain unpredictable and uncontrollable, limiting the scalability and reliability of LRMs' reasoning capabilities. To address these limitations, we move beyond reliance on prompts and coincidental "aha moments". Instead, we explicitly align models with three meta-abilities: deduction, induction, and abduction, using automatically generated, self-verifiable tasks. Our three stage-pipeline individual alignment, parameter-space merging, and domain-specific reinforcement learning, boosting performance by over 10\% relative to instruction-tuned baselines. Furthermore, domain-specific RL from the aligned checkpoint yields an additional gain in performance ceiling for both 7B and 32B models across math, coding, and science benchmarks, demonstrating that explicit meta-ability alignment offers a scalable and dependable foundation for reasoning. Code is available at: https://github.com/zhiyuanhubj/Meta-Ability-Alignment
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。