基于ROS与多模态AI，打造可语音控制的双臂机器人交互平台
Development of a Human-Robot Interaction Platform for Dual-Arm Robots Based on ROS and Multimodal Artificial Intelligence
- 融合视觉与语音的多模态AI，实现人机自然交互
- 双臂机器人定位精度<2cm，物体识别准确率超90%
- 支持远程语音操控，适用于工业与服务场景
本文提出一种基于机器人操作系统（ROS）和多模态人工智能模型的人机交互平台，用于双臂机器人系统。该平台包含双臂机器人硬件系统及软件部分，集成3D相机与嵌入式计算，实现图像处理与自然语言处理功能。首先，设计并开发了定位精度低于2 cm的双臂机器人系统，可在ROS环境中独立运行，并完成工业与服务任务的模拟与建模。其次，集成图像处理人工智能模型，实现物体抓取与分类，准确率超过90%。最后，开发基于语音命令的远程控制软件，通过自然语言处理模型实现人机交互。实验结果表明，该多模态人工智能模型具有高准确性，双臂机器人系统在人机交互环境中表现出强灵活性。
原文摘要 · Abstract (English)
In this paper, we propose the development of an interactive platform between humans and a dual-arm robotic system based on the Robot Operating System (ROS) and a multimodal artificial intelligence model. Our proposed platform consists of two main components: a dual-arm robotic hardware system and software that includes image processing tasks and natural language processing using a 3D camera and embedded computing. First, we designed and developed a dual-arm robotic system with a positional accuracy of less than 2 cm, capable of operating independently, performing industrial and service tasks while simultaneously simulating and modeling the robot in the ROS environment. Second, artificial intelligence models for image processing are integrated to execute object picking and classification tasks with an accuracy of over 90%. Finally, we developed remote control software using voice commands through a natural language processing model. Experimental results demonstrate the accuracy of the multimodal artificial intelligence model and the flexibility of the dual-arm robotic system in interactive human environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。