让大模型自我进化，自动优化推理能力。
AERO: Autonomous Evolutionary Reasoning Optimization via Endogenous Dual-Loop Feedback
- 构建双环反馈系统，内化自问、自答与自评机制。
- 在九个基准上提升4.57%~5.10%，超越现有方法。
- 适合追求自进化推理能力的AI研究者使用。
大型语言模型在复杂推理任务中表现卓越，但受限于对专家标注数据和外部验证器的依赖。现有自进化范式常无法定位最优学习区间，且易因错误内部反馈加剧集体幻觉与错误先验。为此，我们提出自主进化推理优化框架AERO，通过内化自问、自答与批评的协同双环系统实现无监督推理进化。受最近发展区（ZPD）理论启发，AERO利用熵值定位“可解差距”，并采用独立反事实修正进行稳健验证。此外，引入分阶段训练策略以同步各功能角色的能力增长，防止课程坍塌。在涵盖三个领域的九个基准上的广泛评估表明，AERO在Qwen3-4B-Base上平均提升4.57%，在Qwen3-8B-Base上提升5.10%，优于竞争性基线。代码已公开于https://github.com/mira-ai-lab/AERO。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have achieved significant success in complex reasoning but remain bottlenecked by reliance on expert-annotated data and external verifiers. While existing self-evolution paradigms aim to bypass these constraints, they often fail to identify the optimal learning zone and risk reinforcing collective hallucinations and incorrect priors through flawed internal feedback. To address these challenges, we propose \underline{A}utonomous \underline{E}volutionary \underline{R}easoning \underline{O}ptimization (AERO), an unsupervised framework that achieves autonomous reasoning evolution by internalizing self-questioning, answering, and criticism within a synergistic dual-loop system. Inspired by the \textit{Zone of Proximal Development (ZPD)} theory, AERO utilizes entropy-based positioning to target the ``solvability gap'' and employs Independent Counterfactual Correction for robust verification. Furthermore, we introduce a Staggered Training Strategy to synchronize capability growth across functional roles and prevent curriculum collapse. Extensive evaluations across nine benchmarks spanning three domains demonstrate that AERO achieves average performance improvements of 4.57\% on Qwen3-4B-Base and 5.10\% on Qwen3-8B-Base, outperforming competitive baselines. Code is available at https://github.com/mira-ai-lab/AERO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。