arXiv:2410.06472cs.ROcs.AI2024-10中稿 · IEEE Aerospace Con…被引 28

让非专家也能用自然语言操控机器人,ROS Agent实现智能交互

Enabling Novel Mission Operations and Interactions with ROSA: The Robot Operating System Agent

  • 用大模型将自然语言转为机器人指令,对接ROS系统
  • 支持语音与视觉多模态,通过参数校验保障操作安全
  • 适合科研、教育及工业场景中需要简化人机交互的用户

机器人技术虽已广泛应用于多个领域,但其操作通常需专业知识,限制了非专家用户的使用。本文提出ROSA(Robot Operating System Agent),一个基于AI的智能代理,连接机器人操作系统(ROS)与自然语言接口。通过先进语言模型与开源框架集成,ROSA使操作者能以自然语言下达指令,自动转化为可执行动作,并通过标准化工具与ROS通信。ROSA设计模块化且可扩展,兼容ROS1与ROS2,内置参数验证与约束机制,确保操作安全可靠。尽管初始面向ROS,也可适配其他机器人中间件,提升任务通用性。系统支持语音与视觉感知等多模态能力,显著提升人机交互体验。伦理方面遵循阿西莫夫机器人三定律等原则,强调安全、透明、隐私与责任。在喷气推进实验室的火星沙地模拟场、实验室及仿真环境中,以三类机器人完成初步原型测试。核心ROSA库已开源,推动机器人技术更易用、更负责任地发展。

原文摘要 · Abstract (English)

The advancement of robotic systems has revolutionized numerous industries, yet their operation often demands specialized technical knowledge, limiting accessibility for non-expert users. This paper introduces ROSA (Robot Operating System Agent), an AI-powered agent that bridges the gap between the Robot Operating System (ROS) and natural language interfaces. By leveraging state-of-the-art language models and integrating open-source frameworks, ROSA enables operators to interact with robots using natural language, translating commands into actions and interfacing with ROS through well-defined tools. ROSA's design is modular and extensible, offering seamless integration with both ROS1 and ROS2, along with safety mechanisms like parameter validation and constraint enforcement to ensure secure, reliable operations. While ROSA is originally designed for ROS, it can be extended to work with other robotics middle-wares to maximize compatibility across missions. ROSA enhances human-robot interaction by democratizing access to complex robotic systems, empowering users of all expertise levels with multi-modal capabilities such as speech integration and visual perception. Ethical considerations are thoroughly addressed, guided by foundational principles like Asimov's Three Laws of Robotics, ensuring that AI integration promotes safety, transparency, privacy, and accountability. By making robotic technology more user-friendly and accessible, ROSA not only improves operational efficiency but also sets a new standard for responsible AI use in robotics and potentially future mission operations. This paper introduces ROSA's architecture and showcases initial mock-up operations in JPL's Mars Yard, a laboratory, and a simulation using three different robots. The core ROSA library is available as open-source.

人机交互自然语言控制机器人系统AI代理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。