arXiv:2409.11041cs.CLcs.RO2024-09中稿 · ITL4HRI workshop a…被引 3

用大模型实现人机对话编程,让机器人装配更灵活

Towards No-Code Programming of Cobots: Experiments with Code Synthesis by Large Code Models for Conversational Programming

  • 通过自然语言对话生成机器人装配指令序列
  • 大模型能准确生成基础指令,但难以抽象出函数或循环
  • 适合希望零代码控制协作机器人的工业场景

尽管家庭环境中的机器人研究众多,目前多数机器人仍用于工厂车间,人机交互也集中于此。传统协作机器人(cobots)需专家编程,修改困难;手动引导则表达能力受限。为此,我们探索利用大语言模型(LLMs)的上下文学习能力,实现对话式代码生成。作为第一步,我们定义了RATS(重复装配任务),一种2D积木搭建任务,用于模拟工业装配场景。在此任务中,'程序员'通过自然语言指导机器人构建特定结构,即以自然语言诱导程序。我们构建了一个数据集,包含目标结构与多种示例指令(人工编写、模板生成、模型生成)及对应代码。在仿真环境中系统评估了当前主流LLMs基于上下文示例合成代码的能力。结果显示,LLMs可生成准确的'一阶代码'(指令序列),但在生成'高阶代码'(如函数抽象、循环使用)方面表现不佳。

原文摘要 · Abstract (English)

While there has been a lot of research recently on robots in household environments, at the present time, most robots in existence can be found on shop floors, and most interactions between humans and robots happen there. ``Collaborative robots'' (cobots) designed to work alongside humans on assembly lines traditionally require expert programming, limiting ability to make changes, or manual guidance, limiting expressivity of the resulting programs. To address these limitations, we explore using Large Language Models (LLMs), and in particular, their abilities of doing in-context learning, for conversational code generation. As a first step, we define RATS, the ``Repetitive Assembly Task'', a 2D building task designed to lay the foundation for simulating industry assembly scenarios. In this task, a `programmer' instructs a cobot, using natural language, on how a certain assembly is to be built; that is, the programmer induces a program, through natural language. We create a dataset that pairs target structures with various example instructions (human-authored, template-based, and model-generated) and example code. With this, we systematically evaluate the capabilities of state-of-the-art LLMs for synthesising this kind of code, given in-context examples. Evaluating in a simulated environment, we find that LLMs are capable of generating accurate `first order code' (instruction sequences), but have problems producing `higher-order code' (abstractions such as functions, or use of loops).

对话编程协作机器人大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。