用大模型自动生成机器人复杂技能的渐进训练任务，减少人工设计负担。
CurricuLLM: Automatic Task Curricula Design for Learning Complex Robot Skills using Large Language Models
- 利用大模型生成自然语言描述的子任务序列，实现任务自动分解。
- 将子任务转化为可执行代码，包含奖励函数和目标分布设定。
- 在多种仿真环境及真实人形机器人上验证，提升复杂控制学习效率。
课程学习是一种强化学习中的训练机制，通过逐步提高任务难度来促进复杂策略的学习。然而，为特定任务设计有效课程通常需要大量领域知识和人工干预，限制了其在不同领域的应用。本文提出CurricuLLM，利用大语言模型（LLMs）在多样化语言数据上的训练基础及其对世界知识的掌握能力，高效分解任务并跨机器人环境拆解技能。同时，LLMs将自然语言转换为可执行代码的成功经验进一步支持其在课程生成中的作用。CurricuLLM包括三个步骤：(1) 以自然语言形式生成有助于目标任务学习的子任务序列；(2) 将子任务描述转化为可执行的任务代码，包括奖励代码与目标分布代码；(3) 基于轨迹回放和子任务描述评估训练后的策略。我们在多种机器人仿真环境（包括操作、导航、运动控制）中评估了CurricuLLM，证明其能有效辅助复杂机器人控制任务的学习。此外，我们还在真实世界中验证了通过CurricuLLM学习的人形机器人运动策略。项目网站：https://iconlab.negarmehr.com/CurricuLLM/
原文摘要 · Abstract (English)
Curriculum learning is a training mechanism in reinforcement learning (RL) that facilitates the achievement of complex policies by progressively increasing the task difficulty during training. However, designing effective curricula for a specific task often requires extensive domain knowledge and human intervention, which limits its applicability across various domains. Our core idea is that large language models (LLMs), with their extensive training on diverse language data and ability to encapsulate world knowledge, present significant potential for efficiently breaking down tasks and decomposing skills across various robotics environments. Additionally, the demonstrated success of LLMs in translating natural language into executable code for RL agents strengthens their role in generating task curricula. In this work, we propose CurricuLLM, which leverages the high-level planning and programming capabilities of LLMs for curriculum design, thereby enhancing the efficient learning of complex target tasks. CurricuLLM consists of: (Step 1) Generating sequence of subtasks that aid target task learning in natural language form, (Step 2) Translating natural language description of subtasks in executable task code, including the reward code and goal distribution code, and (Step 3) Evaluating trained policies based on trajectory rollout and subtask description. We evaluate CurricuLLM in various robotics simulation environments, ranging from manipulation, navigation, and locomotion, to show that CurricuLLM can aid learning complex robot control tasks. In addition, we validate humanoid locomotion policy learned through CurricuLLM in real-world. Project website is https://iconlab.negarmehr.com/CurricuLLM/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。