用戒指感应手势打字，结合深度学习提升输入速度与准确率。
RingGesture: A Ring-Based Mid-Air Gesture Typing System Powered by a Deep-Learning Word Prediction Framework
- 通过电极和惯性传感器捕捉手势轨迹，实现空中打字。
- 达到27.3 WPM平均输入速度，峰值达47.9 WPM。
- 新提出的融合评分框架使错误率降低28.2%，速度提升55.2%。
文本输入是现代计算体验的核心能力，轻量级增强现实（AR）眼镜也不例外。由于全天佩戴需求，轻量级AR眼镜难以集成多摄像头以实现广视野手部追踪，这限制了输入方式。为此，我们提出一种基于戒指的空中手势打字系统RingGesture，利用电极标记手势轨迹的起止点，结合惯性测量单元（IMU）传感器进行手部追踪。该方法提供类似VR中射线式空中打字的直观体验，可将手部动作无缝转换为光标导航。为提升准确率与输入速度，我们设计了一种新型深度学习词预测框架Score Fusion，包含三部分：词-手势解码模型、空间拼写纠错模型与轻量级上下文语言模型。该框架通过融合三者得分，实现更高精度的词语预测。对比与纵向研究表明：第一，RingGesture整体有效，平均输入速率达27.3 WPM，峰值达47.9 WPM；第二，Score Fusion相比传统框架Naive Correction，在未纠正字符错误率上降低28.2%，使输入速度提升55.2%。此外，系统可用性评分为83，表明其具备优异可用性。
原文摘要 · Abstract (English)
Text entry is a critical capability for any modern computing experience, with lightweight augmented reality (AR) glasses being no exception. Designed for all-day wearability, a limitation of lightweight AR glass is the restriction to the inclusion of multiple cameras for extensive field of view in hand tracking. This constraint underscores the need for an additional input device. We propose a system to address this gap: a ring-based mid-air gesture typing technique, RingGesture, utilizing electrodes to mark the start and end of gesture trajectories and inertial measurement units (IMU) sensors for hand tracking. This method offers an intuitive experience similar to raycast-based mid-air gesture typing found in VR headsets, allowing for a seamless translation of hand movements into cursor navigation. To enhance both accuracy and input speed, we propose a novel deep-learning word prediction framework, Score Fusion, comprised of three key components: a) a word-gesture decoding model, b) a spatial spelling correction model, and c) a lightweight contextual language model. In contrast, this framework fuses the scores from the three models to predict the most likely words with higher precision. We conduct comparative and longitudinal studies to demonstrate two key findings: firstly, the overall effectiveness of RingGesture, which achieves an average text entry speed of 27.3 words per minute (WPM) and a peak performance of 47.9 WPM. Secondly, we highlight the superior performance of the Score Fusion framework, which offers a 28.2% improvement in uncorrected Character Error Rate over a conventional word prediction framework, Naive Correction, leading to a 55.2% improvement in text entry speed for RingGesture. Additionally, RingGesture received a System Usability Score of 83 signifying its excellent usability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。