让AI自动生成学习路径,高效解决复杂决策问题。
EvoCurr: Self-evolving Curriculum with Behavior Code Generation for Complex Decision-making
- 通过自进化课程与可执行策略协同优化,动态生成中间任务。
- 在星际争霸2极端场景下,胜率高达96.7%~99.2%,无需微调大模型。
- 适用于代码策略与多智能体强化学习,适合复杂任务求解研究者。
复杂决策常需代理逐步完成中间任务,而非直接解决最终目标。现有LLM自改进方法通常在固定目标上迭代,而课程学习方法多依赖人工设计进度、训练时优化或领域特定难度指标。为实现自适应课程设计以平滑学习曲线,我们提出EvoCurr——一种通用的推理时框架,能协同演化课程与可执行策略。设计师基于最新成功进展和近期失败结果生成可验证的中间任务;求解器则生成或训练对应策略。仅当任务-策略对通过硬性可行性检验并达到指定性能阈值时才被接受,同时保留最新验证检查点的接受底限约束,防止失败的高难度尝试覆盖已掌握行为。我们在StarCraft II超晚盘微操作、最严压力测试场景下评估EvoCurr,三种LLM骨干模型均实现96.7%–99.2%的宏观平均胜率,且无需对LLM参数进行训练。在Flatland、POGEMA、Overcooked和LiveCodeBench Hard等其他实验中表现提升,体现了EvoCurr的通用性。
原文摘要 · Abstract (English)
Complex decision-making often requires agents to progress through intermediate tasks rather than solve the final target directly. Existing LLM self-refinement methods typically iterate on a fixed target, while curriculum-learning methods often rely on hand-designed schedules, training-time optimization, or domain-specific difficulty metrics. To smooth out the learning curve with adaptive curriculum desgin, we introduce EvoCurr, a general inference-time framework that co-evolves curricula and executable policies. A Designer proposes verifiable intermediate tasks from the latest accepted progress and recent failures, while a Solver generates or trains policies for these tasks. A task-policy pair is accepted only when it passes hard feasibility checks and reaches a specified performance threshold meanwhile an accepted-floor constraint preserves the latest verified checkpoint and prevents failed harder attempts from overwriting mastered behavior. We evaluate EvoCurr across code-as-policy and closed-loop MARL settings on StarCraft II super-late-game micromanagement, hardest stress test, where EvoCurr achieves 96.7--99.2% macro-average win rates across three LLM backbones without LLM parameter training. The performance improvments on other experiments, Flatland, POGEMA, Overcooked, and LiveCodeBench Hard, reflect the generality of our EvoCurr.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。