arXiv:2601.03098cs.LGcs.NE2026-01被引 1

用肌电信号实现无键盘打字,分步解析手指动作并生成文本。

From Muscle to Text with MyoText: sEMG to Text via Finger Classification and Transformer-Based Decoding

  • 先识别手指动作,再结合打字规律和Transformer模型生成句子。
  • 在30人数据集上达5.4%字符错误率,比基线提升显著。
  • 适合可穿戴设备、元宇宙等无需物理键盘的输入场景。

表面肌电(sEMG)为解码肌肉活动提供了直接神经接口,是实现可穿戴与混合现实系统中无键盘文本输入的有前景基础。以往研究主要从sEMG信号直接识别字母,迈出了关键一步。在此基础上,我们提出MyoText,一种分层解码框架,通过生理上合理的中间阶段将sEMG信号转化为文本。MyoText首先使用CNN-BiLSTM-Attention模型从多通道sEMG中分类手指激活,结合人体工学打字先验推断字母,并通过微调的T5 Transformer重建完整句子。该模块化设计模拟自然打字过程,将肌肉意图与语言输出关联,有效缩小解码搜索空间。在emg2qwerty数据集上对30名用户评估显示,MyoText达到85.4%手指分类准确率、5.4%字符错误率(CER)和6.5%词错误率(WER)。除准确率提升外,该方法建立了一条从神经肌肉信号到文本的合理路径,为虚拟与增强现实中的无键盘输入界面提供蓝图。通过融合人体工学结构与Transformer语言推理,MyoText推动了未来普适计算环境中无缝可穿戴神经输入的可行性。

原文摘要 · Abstract (English)

Surface electromyography (sEMG) provides a direct neural interface for decoding muscle activity and offers a promising foundation for keyboard-free text input in wearable and mixed-reality systems. Previous sEMG-to-text studies mainly focused on recognizing letters directly from sEMG signals, forming an important first step toward translating muscle activity into text. Building on this foundation, we present MyoText, a hierarchical framework that decodes sEMG signals to text through physiologically grounded intermediate stages. MyoText first classifies finger activations from multichannel sEMG using a CNN-BiLSTM-Attention model, applies ergonomic typing priors to infer letters, and reconstructs full sentences with a fine-tuned T5 transformer. This modular design mirrors the natural hierarchy of typing, linking muscle intent to language output and reducing the search space for decoding. Evaluated on 30 users from the emg2qwerty dataset, MyoText outperforms baselines by achieving 85.4% finger-classification accuracy, 5.4% character error rate (CER), and 6.5% word error rate (WER). Beyond accuracy gains, this methodology establishes a principled pathway from neuromuscular signals to text, providing a blueprint for virtual and augmented-reality typing interfaces that operate entirely without physical keyboards. By integrating ergonomic structure with transformer-based linguistic reasoning, MyoText advances the feasibility of seamless, wearable neural input for future ubiquitous computing environments.

肌电输入无键盘打字Transformer可穿戴

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。