arXiv:2608.19281cs.ROcs.HC2026-08中稿 · publication in Pro…

让普通人通过对话和可视化块编程控制机器人,还能复用成功指令。

APPROVE: Visual End-User-in-the-Loop Robot Programming with LLMs

  • 结合自然语言与可视化编程块,用户可直观确认或修改生成的程序。
  • 支持用户反馈确认,确保程序符合真实意图,提升可信度。
  • 生成的正确代码存入库,方便后续重复使用,降低编程门槛。

非专业人士编程机器人仍面临挑战,传统方法需专业知识,即使采用积木式界面也缺乏灵活性。近期研究利用大语言模型(LLMs)从自然语言自动生成机器人程序,但这些系统存在透明性不足、难以保证与用户意图对齐、复用性差等问题。本文提出APPROVE(AI驱动的带视觉用户反馈的机器人编程),一个基于LLM的多模态用户编程框架,融合自然语言输入、积木式界面(Blockly)和显式的用户确认步骤。生成的程序以积木形式可视化展示,用户可在执行前确认、修改或拒绝。确认后的功能模块被存入库中供复用,逐步构建可靠的程序组件集。该方法强调以用户为中心的设计,注重用户信任、意图对齐与可复用性。

原文摘要 · Abstract (English)

Programming robots remains challenging for non-experts, as traditional methods require expert knowledge and even block-based interfaces often lack flexibility. Recent work has explored Large Language Models (LLMs) to automatically generate robot programs from natural language, but these systems remain limited by a lack of transparency, missing mechanisms to ensure alignment with user intent, and little support for reuse. We present APPROVE (AI-Powered Programming for Robots with Visual End-User Feedback), an LLM-based multi-modal end-user programming framework that integrates natural language input with a block-based interface and an explicit user confirmation step. Generated programs are visualized using a block-based interface in Blockly, allowing users to confirm, modify, or reject them before execution. Confirmed functions are stored in a library for reuse, gradually building a set of reliable program components. Our approach contributes a human-centered design for LLM-based robot programming that emphasizes user trust, intent alignment, and reusability.

机器人编程大模型应用人机交互

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。