可穿戴设备同时解码手部动作、手势和抓握力，精准稳定。
Wearable Multimodal Human-Machine Interface for Integrated Hand Intentions Decoding in Dynamic Teleoperation
- 融合肌电与惯性传感器，实现多模态信号同步解码。
- 手势识别准确率超97%，抓握力预测决定系数达0.95。
- 适合复杂动态环境下远程操控，尤其适合医疗或工业场景。
在光学条件受限的普遍远程操作环境中，兼具可穿戴性与高精度手部意图解码（手姿、手势、抓握力）的接口至关重要。现有方案常因多意图解码多样性或可穿戴性不足而受限。为此，我们开发了一种新型多意图解码人机接口（MI-DHMI），集成高通量表面肌电（sEMG）传感器与手部及前臂安装的惯性测量单元（IMUs）。该接口基于统一框架，实现多意图同步解码。通过多模态深度学习与低噪声硬件设计，解码框架能聚焦真实关联手指运动的sEMG成分，有效降低非约束上肢运动中信号变异带来的误差，显著提升鲁棒性。即使在自由腕部与前臂运动下，系统仍实现手势识别准确率超过97%，抓握力估计决定系数 $R^2 = 0.95$，手姿解码与实际姿态一致，优于基线设备与算法。消融实验进一步验证了框架有效性。两个在线实验（倒水任务与物体抓取）验证了设备在高稳定性任务中的优越性能。本系统为全可穿戴、多意图解码提供了新方案，有力支持泛在远程操作，推动人机交互研究发展。
原文摘要 · Abstract (English)
Under ubiquitous teleoperation environments with optically challenging conditions, an interface for tele-operated grasping that combines wearability with precise decoding of hand intentions (hand pose, gestures, and grasping force) is essential. Yet, existing interfaces often fall short in meeting these demands, compromising either the diversity of multiple intentions decoding or wearability. To address this, we developed a novel Multiple Intentions Decoding Human-Machine Interface (MI-DHMI) that integrates high-throughput surface electromyography (sEMG) sensors with hand-mounted and forearm-mounted inertial measurement units (IMUs). The developed interface is supported by a unified framework for simultaneous multiple intentions decoding. By employing multimodal deep learning and hardware design with a low noise floor, the decoding framework selectively focuses on the sEMG components that are genuinely associated with finger movements. This effectively reduces decoding errors caused by sEMG variability during unconstrained upper-limb motions, thereby significantly enhancing robustness. Even under unconstrained wrist and forearm motion, the interface achieves a gesture recognition accuracy exceeding 97%, grasping force estimation with $R^2 = 0.95$, and hand pose decoding consistent with the actual hand pose, outperforming baseline devices and algorithms. Ablation studies further validate the effectiveness of the proposed decoding framework. Finally, two online experiments were conducted to validate the device, demonstrating its superior performance in high-stability tasks, including a pouring task and object grasping. The developed interface provides a new solution of a fully wearable, multiple intentions decoding system, offering effective support for ubiquitous teleoperation and contributing to the advancement of human-machine interaction research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。