arXiv:2609.08183cs.CL2026-09

让AI通过自我评估持续进化,用反馈循环提升能力。

NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness

论文配图:NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness
图 1 · 摘自论文原文
  • 构建异构模型池与智能路由,记录用户交互全过程。
  • 4B模型性能从58.94升至64.87,9B模型从65.60升至69.04。
  • 适合研究自进化系统、强化学习与智能体架构的开发者。

递归自我改进(RSI)需要一个明确机制,使AI系统能观察自身能力并将其转化为下一轮学习的依据。我们提出NeoHorse-1,一种面向智能体的后训练模型家族,探索这一路径。系统结合异构模型池与智能路由,记录每个用户回合中的能力需求预测、服务层级选择及后续交互。这些记录被转化为包含交错推理、工具调用和上下文信息的训练样本,并通过结构验证、六维语义评估与子场景级标注筛选。路由信号将监督微调组织为三阶段课程,并扩展至路由引导的在线策略蒸馏,教师模型在相同演进路径下监督学生生成响应。能力引导的分配将评估反馈转化为下一阶段训练混合策略,形成评估-选择-更新闭环,使系统学习的内容决定其下一阶段的学习内容。在涵盖基于Harness的智能体、工具使用、编码与指令遵循等十一项基准测试中,后训练使4B模型的宏观平均得分从58.94提升至64.87,9B模型从65.60提升至69.04,显著缩小了后训练4B模型与9B基线模型间的整体差距。NeoHorse-1为反馈驱动的迭代式自进化提供了初始原型,铺就了跨迭代实现调度媒介型RSI的路径。

原文摘要 · Abstract (English)

Recursive self-improvement (RSI) requires a concrete mechanism through which an AI system observes its capabilities and converts that evidence into the next round of learning. We present NeoHorse-1, a family of agent-native models developed to explore this path through agentic post-training. Our system combines a heterogeneous model pool with intelligent routing, recording the predicted capability demand, selected service tier, and subsequent interaction for each user turn. These records are converted into training examples that preserve interleaved reasoning, tool calls, and harness context, and are admitted through structural validation, six-dimensional semantic evaluation, and subscene-level labeling. Routing signals organize supervised fine-tuning into a three-stage curriculum and extend to routing-guided on-policy distillation, where a teacher supervises student-generated responses under the same progression. Capability-guided allocation then converts evaluation feedback into the next training mixture, closing an evaluation-selection-update loop in which what the system learns to do shapes what it learns from next. Across eleven benchmarks covering harness-based agents, tool use, coding, and instruction following, post-training raises the macro-average from 58.94 to 64.87 at 4B and from 65.60 to 69.04 at 9B, substantially narrowing the aggregate gap between the post-trained 4B model and the 9B base model. NeoHorse-1 provides an initial prototype of this feedback-driven process and a path toward harness-mediated RSI across successive iterations.

自进化智能体后训练强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。