让大模型自己生成任务规划并自我纠错,提升机器人规划能力。
Self-CriTeach: LLM Self-Teaching and Self-Critiquing for Improving Robotic Planning via Automated Domain Generation
- 大模型自动生成符号化规划域,兼具训练数据与奖励信号双重作用。
- 在多个任务上实现90%以上成功率,推理成本降低40%。
- 适合研究机器人自主规划、少样本学习与强化学习的学者。
大型语言模型(LLMs)在机器人任务规划中展现出巨大潜力,尤其体现在自动规划领域生成方面。然而,现有方法大多将生成的规划领域视为单纯的任务工具,其在逻辑状态不完整或感知噪声下表现脆弱,忽略了作为可扩展推理监督和结构化奖励信号的潜力。同时,推理型大模型依赖昂贵的思维链(CoT)标注,而强化学习面临奖励设计难题。本文提出Self-CriTeach框架,使大模型能够自主生成符号化规划域,实现双重功能:(1) 大规模生成机器人规划问题-计划对;(2) 提供结构化奖励函数。首先,自生成的领域支持大规模符号化任务计划生成,并自动转换为扩展的思维链轨迹用于监督微调;其次,同一领域被复用作结构化奖励函数,无需人工设计即可为强化学习提供密集反馈。该统一训练流程显著提升了规划成功率、跨任务泛化能力、推理效率,并增强对不完整逻辑状态的鲁棒性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have recently shown strong promise for robotic task planning, particularly through automatic planning domain generation. However, prior approaches largely treat generated planning domains as planning utilities, which are brittle under imperfect logical states and perception noise, overlooking their potential as scalable sources of reasoning supervision and structured reward signals. At the same time, reasoning LLMs depend on chain-of-thought (CoT) supervision that is expensive to collect for robotic tasks, and reinforcement learning (RL) faces challenges in reward engineering. We propose Self-CriTeach, an LLM self-teaching and self-critiquing framework in which an LLM autonomously generates symbolic planning domains that serve a dual role: (1) enabling large-scale generation of robotic planning problem-plan pairs, and (2) providing structured reward functions. First, the self-written domains enable large-scale generation of symbolic task plans, which are automatically transformed into extended CoT trajectories for supervised fine-tuning. Second, the self-written domains are reused as structured reward functions, providing dense feedback for reinforcement learning without manual reward engineering. This unified training pipeline yields a planning-enhanced LLM with higher planning success rates, stronger cross-task generalization, reduced inference cost, and resistance to imperfect logical states. GitHub Page: https://markli1hoshipu.github.io/Plan_LLM/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。