用大模型自动设计机器人训练课程，零样本部署到真实机器人
AURA: Autonomous Upskilling with Retrieval-Augmented Agents
- 用大模型将用户指令转为可执行的训练流程，支持多阶段课程生成
- 在人形行走和操作任务中，成功率显著高于传统方法
- 通过检索历史训练结果持续优化，适合想快速试错的研究者
为敏捷机器人设计强化学习课程通常需要大量手动调整奖励函数、环境随机化和训练配置。我们提出AURA（自主提升的检索增强型代理），一个基于大语言模型的分阶段课程强化学习框架。AURA将用户提示转化为包含完整奖励函数、域随机化策略和训练配置的YAML工作流，并在使用GPU前进行静态验证，确保高效可靠执行。通过检索增强的反馈循环，专业LLM代理可根据向量数据库中存储的历史训练结果，自动设计、执行并优化课程阶段，实现持续改进。定量实验表明，AURA在生成成功率、人形机器人行走和操作任务中均优于基线方法。消融实验证明了模式验证与检索对课程质量的重要性。AURA能直接从用户提示训练端到端策略，并在多个环境中零样本部署于定制人形机器人——这是以往人工设计控制器无法实现的能力。通过抽象课程设计复杂性，AURA实现了可扩展、自适应的策略学习流水线。
原文摘要 · Abstract (English)
Designing reinforcement learning curricula for agile robots traditionally requires extensive manual tuning of reward functions, environment randomizations, and training configurations. We introduce AURA (Autonomous Upskilling with Retrieval-Augmented Agents), a schema-validated curriculum reinforcement learning (RL) framework that leverages Large Language Models (LLMs) as autonomous designers of multi-stage curricula. AURA transforms user prompts into YAML workflows that encode full reward functions, domain randomization strategies, and training configurations. All files are statically validated before any GPU time is used, ensuring efficient and reliable execution. A retrieval-augmented feedback loop allows specialized LLM agents to design, execute, and refine curriculum stages based on prior training results stored in a vector database, enabling continual improvement over time. Quantitative experiments show that AURA consistently outperforms LLM-guided baselines in generation success rate, humanoid locomotion, and manipulation tasks. Ablation studies highlight the importance of schema validation and retrieval for curriculum quality. AURA successfully trains end-to-end policies directly from user prompts and deploys them zero-shot on a custom humanoid robot in multiple environments - capabilities that did not exist previously with manually designed controllers. By abstracting the complexity of curriculum design, AURA enables scalable and adaptive policy learning pipelines that would be complex to construct by hand. Project page: https://aura-research.org/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。