无需演示或微调，大模型代理直接控制机器人完成复杂操作。
Demonstration-Free Robotic Control via LLM Agents
- 用通用大模型代理直接处理机器人任务规划，不需额外训练。
- 在多个基准上成功率达84.9%至96%，接近有少量演示的视觉语言模型。
- 适合需要自主探索新场景的机器人系统开发与数据增强。
机器人操作日益采用视觉-语言-动作（VLA）模型，虽性能强但通常需特定任务演示和微调，且在领域变化下泛化能力差。本文探讨通用大语言模型（LLM）代理框架（原用于软件工程）能否作为实体操作的新控制范式。提出FAEA（前沿代理作为实体代理），直接将未修改的前沿代理（Claude Agent SDK）应用于实体操作。利用其迭代推理能力，使代理能自主推演操作策略。在LIBERO、ManiSkill3和MetaWorld三个基准上评估，当可访问环境状态时，成功率分别为84.9%、85.7%和96%。该表现接近仅用少于100次演示训练的VLA模型，且无需演示或微调。通过一轮人类反馈优化后，LIBERO上性能提升至88.2%。此无演示能力具实际价值：FAEA可自主探索仿真中的新场景，并生成成功轨迹用于数据增强。结果表明，通用代理足以应对以决策规划为主的操纵任务，为机器人系统引入持续维护的代理基础设施并直接受益于前沿模型进展提供可能。代码已公开于https://github.com/robiemusketeer/faea-sim。
原文摘要 · Abstract (English)
Robotic manipulation has increasingly adopted vision-language-action (VLA) models, which achieve strong performance but typically require task-specific demonstrations and fine-tuning, and often generalize poorly under domain shift. We investigate whether general-purpose large language model (LLM) agent frameworks, originally developed for software engineering, can serve as an alternative control paradigm for embodied manipulation. We introduce FAEA (Frontier Agent as Embodied Agent), which applies an LLM agent framework directly to embodied manipulation without modification. Using the same iterative reasoning that enables software agents to debug code, FAEA enables embodied agents to reason through manipulation strategies. We evaluate an unmodified frontier agent, Claude Agent SDK, across the LIBERO, ManiSkill3, and MetaWorld benchmarks. With privileged environment state access, FAEA achieves success rates of 84.9%, 85.7%, and 96%, respectively. This level of task success approaches that of VLA models trained with less than 100 demonstrations per task, without requiring demonstrations or fine-tuning. With one round of human feedback as an optional optimization, performance increases to 88.2% on LIBERO. This demonstration-free capability has immediate practical value: FAEA can autonomously explore novel scenarios in simulation and generate successful trajectories for training data augmentation in embodied learning. Our results indicate that general-purpose agents are sufficient for a class of manipulation tasks dominated by deliberative, task-level planning. This opens a path for robotics systems to leverage actively maintained agent infrastructure and benefit directly from ongoing advances in frontier models. Code is available at https://github.com/robiemusketeer/faea-sim
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。