arXiv:2606.17427cs.CVcs.HC2026-06

评估手部损伤与遮挡对AR手姿估计精度的影响,发现设备仍可准确识别残障人群手势。

Impact of Hand Impairment and Occlusions on Hand Pose Estimation Accuracy in Augmented Reality Applications

论文配图:Impact of Hand Impairment and Occlusions on Hand Pose Estimation Accuracy in Augmented Reality Applications
图 1 · 摘自论文原文
  • 对比了霍洛伦斯2头显与四种先进算法在真实物体交互中的表现
  • 残障组与正常组误差无显著差异,透明物体略优于不透明物体(0.1毫米)
  • WiLoR和HaMeR算法精度高于头显(差2毫米),适合康复应用优化

混合现实应用可用于手部康复训练。增强现实(AR)头戴显示设备(HMD)能提供生态有效的任务,使用户在观察真实环境的同时与实物互动,并接收显示在头显上的额外提示。尽管这些应用依赖于精确的手部姿态估计,但关于手部损伤或真实物体遮挡对手姿估计准确性的影响仍缺乏研究。此外,尚未建立AR HMD预测与最先进姿态估计方法之间的比较基准。本研究评估了霍洛伦斯2头显及四种先进姿态估计算法(WiLoR、HaMeR、WildHands、MediaPipe)在13名颈脊髓损伤患者(神经损伤水平C3-C6;ASIA损伤分级A-D)与15名健康对照组与透明及不透明物体交互时的表现。通过多相机系统三角测量生成3D关节位置的真值数据。结果表明,损伤组与健康组之间的姿态估计误差无显著差异,说明霍洛伦斯2与各算法对残障人群具有良好的泛化能力。透明物体相比不透明物体带来0.1毫米的精度优势,且WiLoR与HaMeR的预测精度略高于霍洛伦斯2(误差差2毫米)。总体结果表明,霍洛伦斯2可用于手部康复应用,所生成的数据集也可用于改进针对残障人群的姿态估计方法。

原文摘要 · Abstract (English)

Mixed reality applications can be designed for hand rehabilitation. Augmented reality (AR) head mounted displays (HMDs) specifically allow for ecologically valid tasks because individuals can see their real environment and interact with real objects while receiving additional cues on the HMD. While these applications rely on accurate hand pose estimation, there is a gap in investigating the influence of hand impairment or occlusion from real-object interactions on pose estimation accuracy. Further, comparisons between AR HMD predictions and state-of-the-art pose estimation methods have not been established. The current study assessed pose estimation accuracy of the HoloLens 2 HMD and state-of-the-art pose estimation algorithms (WiLoR, HaMeR, WildHands, and MediaPipe) while individuals with cervical spinal cord injury (cSCI; n = 13, Neurological Level of Injury: C3-C6; American Spinal Injury Association Impairment Scale: A-D) and 15 uninjured controls interacted with clear and opaque objects. Ground truth estimates of 3D joint positions were generated via triangulation from a multi-camera setup. Pose estimation accuracy did not differ between the cSCI and uninjured control groups suggesting that 3D joint predictions from the HoloLens 2 and pose estimation algorithms can generalize to populations with hand impairment. Further, clear objects provided a small accuracy advantage over opaque objects (0.1 mm) and predictions from both WiLoR and HaMeR were slightly more accurate than the HoloLens 2 (2 mm). Overall, these results suggest that the HoloLens 2 may be viable for hand rehabilitation applications and the dataset generated can be used to refine pose estimation methods for hand-impaired populations.

手部姿态估计康复应用增强现实残障适配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。