用智能纺织手套实时精准捕捉手部动作与物体交互,精度媲美昂贵相机。
Capturing complex hand movements and object interactions using machine learning-powered stretchable smart textile gloves
- 嵌入螺旋传感纱线和惯性单元的可拉伸手套,支持洗护使用。
- 跨受试者验证平均关节角度误差仅1.45度,接近高成本动捕系统。
- 适用于手势识别、虚拟键盘输入等场景,适合人机交互研究者。
精确实时追踪灵巧手部动作及与物体的交互,在人机交互、元宇宙、机器人和远程医疗中有广泛应用。由于手部关节众多、自由度高,真实手部动作捕捉极具挑战。本文报告了一种基于嵌有螺旋传感器纱线和惯性测量单元的可拉伸、可洗涤智能手套,实现对手部与手指动作的高精度动态追踪。传感器纱线具有宽动态范围(0.005%至155%应变),且经多次使用和洗涤后仍保持稳定。我们采用多阶段机器学习方法,在受试者内和跨受试者交叉验证中,平均关节角度估计均方根误差分别为1.21°和1.45°,精度媲美昂贵的动捕摄像头,且无遮挡或视场限制。我们提出一种数据增强技术,提升对传感器噪声和个体差异的鲁棒性。实验展示了在物体交互中对灵巧手部动作的准确追踪,实现了模拟纸面键盘的精准打字、基于美国手语的复杂动态与静态手势识别,以及物体识别,为新应用开辟了道路。
原文摘要 · Abstract (English)
Accurate real-time tracking of dexterous hand movements and interactions has numerous applications in human-computer interaction, metaverse, robotics, and tele-health. Capturing realistic hand movements is challenging because of the large number of articulations and degrees of freedom. Here, we report accurate and dynamic tracking of articulated hand and finger movements using stretchable, washable smart gloves with embedded helical sensor yarns and inertial measurement units. The sensor yarns have a high dynamic range, responding to low 0.005 % to high 155 % strains, and show stability during extensive use and washing cycles. We use multi-stage machine learning to report average joint angle estimation root mean square errors of 1.21 and 1.45 degrees for intra- and inter-subjects cross-validation, respectively, matching accuracy of costly motion capture cameras without occlusion or field of view limitations. We report a data augmentation technique that enhances robustness to noise and variations of sensors. We demonstrate accurate tracking of dexterous hand movements during object interactions, opening new avenues of applications including accurate typing on a mock paper keyboard, recognition of complex dynamic and static gestures adapted from American Sign Language and object identification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。