用大模型自动分解复杂任务，少样本也能高效学习
SEAL: SEmantic-Augmented Imitation Learning via Language Model
- 用大模型理解任务语义，自动生成子目标空间
- 仅需少量专家演示（<100条）就达到顶尖性能
- 适合数据稀缺的长时序决策场景，如机器人操作
层次化模仿学习（HIL）是解决长时序决策任务的有前景方法。然而，由于缺乏子目标学习的详细监督标签，且依赖数百至数千条专家示范，实际应用受限。本文提出SEAL框架，利用大语言模型（LLM）强大的语义与世界知识，无需预先了解任务层级即可指定子目标空间，并对状态进行语义化的子目标标注。SEAL采用双编码器结构，结合监督式LLM引导的子目标学习与无监督向量量化（VQ），生成更鲁棒的子目标表示。此外，引入增强型低层规划器以提升对子目标转移的适应能力。实验表明，SEAL在小规模专家数据集（<100条）和复杂长时序任务中均优于现有最先进HIL方法及基于LLM的规划方法。
原文摘要 · Abstract (English)
Hierarchical Imitation Learning (HIL) is a promising approach for tackling long-horizon decision-making tasks. While it is a challenging task due to the lack of detailed supervisory labels for sub-goal learning, and reliance on hundreds to thousands of expert demonstrations. In this work, we introduce SEAL, a novel framework that leverages Large Language Models (LLMs)'s powerful semantic and world knowledge for both specifying sub-goal space and pre-labeling states to semantically meaningful sub-goal representations without prior knowledge of task hierarchies. SEAL employs a dual-encoder structure, combining supervised LLM-guided sub-goal learning with unsupervised Vector Quantization (VQ) for more robust sub-goal representations. Additionally, SEAL incorporates a transition-augmented low-level planner for improved adaptation to sub-goal transitions. Our experiments demonstrate that SEAL outperforms state-of-the-art HIL methods and LLM-based planning approaches, particularly in settings with small expert datasets and complex long-horizon tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。