用手势同时控制机器人移动和操作，低功耗实时响应。
A Bimanual Gesture Interface for ROS-Based Mobile Manipulators Using TinyML and Sensor Fusion
- 左右手分工：左手控移动，右手做手势识别
- 用微型神经网络在嵌入式设备上实现手势识别
- 适合工业、助老、危险环境中的低成本人机交互
基于手势的移动机械臂控制长期面临可靠性、效率和直观性不足的问题。本文提出一种双手手势接口，结合TinyML、频谱分析与多传感器融合，在ROS框架内实现改进。左手指挥动与弯曲由加速度计和柔性传感器捕捉，用于移动底盘导航；右手机体姿态信号通过频谱分析后，由轻量级神经网络分类，实现对7自由度Kinova Gen3机械臂的控制。该系统支持导航与操作并行执行，相比串行方法提升效率与协同性。主要贡献包括双臂控制架构、实时低功耗手势识别、鲁棒多模态传感器融合，以及可扩展的ROS实现。本方法推动了工业自动化、辅助机器人及危险场景中的人机交互发展，提供了一种低成本、开源的实用解决方案，具备实际部署与优化潜力。
原文摘要 · Abstract (English)
Gesture-based control for mobile manipulators faces persistent challenges in reliability, efficiency, and intuitiveness. This paper presents a dual-hand gesture interface that integrates TinyML, spectral analysis, and sensor fusion within a ROS framework to address these limitations. The system uses left-hand tilt and finger flexion, captured using accelerometer and flex sensors, for mobile base navigation, while right-hand IMU signals are processed through spectral analysis and classified by a lightweight neural network. This pipeline enables TinyML-based gesture recognition to control a 7-DOF Kinova Gen3 manipulator. By supporting simultaneous navigation and manipulation, the framework improves efficiency and coordination compared to sequential methods. Key contributions include a bimanual control architecture, real-time low-power gesture recognition, robust multimodal sensor fusion, and a scalable ROS-based implementation. The proposed approach advances Human-Robot Interaction (HRI) for industrial automation, assistive robotics, and hazardous environments, offering a cost-effective, open-source solution with strong potential for real-world deployment and further optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。