用手势控制增强现实，让行动不便者也能高效操作。
Accessible Gesture-Driven Augmented Reality Interaction System
- 融合视觉与传感器数据，用深度学习识别手部和身体动作
- 对行动障碍用户提升20%任务效率，满意度提高25%
- 支持隐私保护训练，适合残障人士及无障碍设计研究者
增强现实（AR）提供沉浸式交互体验，但因依赖高精度输入方式，对运动障碍或精细动作受限的用户仍不友好。本文提出一种基于手势的AR交互系统，利用深度学习从可穿戴传感器和摄像头中识别手部与身体姿态，并根据用户能力动态调整界面。系统采用视觉变压器（ViTs）、时序卷积网络（TCNs）和图注意力网络（GATs）处理手势信号，通过联邦学习实现跨用户隐私保护的模型训练，并使用强化学习优化菜单布局与交互模式。实验表明，相较于基线系统，该方法使运动障碍用户的任务完成效率提升20%，用户满意度提高25%。该方案显著提升了AR系统的可访问性与可扩展性。
原文摘要 · Abstract (English)
Augmented reality (AR) offers immersive interaction but remains inaccessible for users with motor impairments or limited dexterity due to reliance on precise input methods. This study proposes a gesture-based interaction system for AR environments, leveraging deep learning to recognize hand and body gestures from wearable sensors and cameras, adapting interfaces to user capabilities. The system employs vision transformers (ViTs), temporal convolutional networks (TCNs), and graph attention networks (GATs) for gesture processing, with federated learning ensuring privacy-preserving model training across diverse users. Reinforcement learning optimizes interface elements like menu layouts and interaction modes. Experiments demonstrate a 20% improvement in task completion efficiency and a 25% increase in user satisfaction for motor-impaired users compared to baseline AR systems. This approach enhances AR accessibility and scalability. Keywords: Deep learning, Federated learning, Gesture recognition, Augmented reality, Accessibility, Human-computer interaction
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。