让大模型学会自我判断何时停止推理，避免无意义重复。
From "Aha Moments" to Controllable Thinking: Toward Meta-Cognitive Reasoning in Large Reasoning Models via Decoupled Reasoning and Control
- 将思考与控制分离，独立优化决策机制。
- 在多个基准上提升推理效率和准确率，减少冗余计算。
- 适合需要高效推理的部署场景，如实时问答系统。
大型推理模型（LRMs）虽具备逐步推理、反思和回溯能力，但这些行为常缺乏调控，导致在达到高置信度结论后仍持续生成冗余推理，增加推理成本与延迟，限制实际应用。根本原因在于缺乏内在机制来监测推理状态并决定是否继续、回溯或停止。本文提出MERA框架，通过解耦推理与控制，实现控制策略的独立优化。MERA采用接管式流水线构建高质量推理-控制监督数据，并将长序列转换为结构化的推理-控制交替序列用于训练。模型通过监督微调学习这种结构化分离，再经由控制段策略优化（CSPO）进一步优化，该方法结合分段GRPO与控制掩码，聚焦于控制段的学习。实验表明，MERA在多个推理基准上同时提升了效率与准确率。
原文摘要 · Abstract (English)
Large Reasoning Models (LRMs) can exhibit step-by-step reasoning, reflection, and backtracking, but these behaviors are often unregulated, leading to overthinking. As a result, LRMs continue generating redundant reasoning even after reaching high-confidence conclusions. This increases inference cost and latency, limiting practical deployment. The root cause is the absence of an intrinsic mechanism to monitor the reasoning state and decide when to continue, backtrack, or stop. We propose MERA, a meta-cognitive reasoning framework that decouples reasoning from control to enable independent optimization of control strategies. MERA constructs high-quality reasoning-control supervision data via a takeover-based pipeline, and transforms long-horizon traces into structured reasoning-control alternating sequences for training. The model is trained with supervised fine-tuning to internalize the structured separation, and further optimized with Control-Segment Policy Optimization (CSPO), which combines segment-wise GRPO with control masking to focus learning on control segments. Experiments across reasoning benchmarks show that MERA improves both efficiency and accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。