将Transformer模型扩展至1.1亿参数，实现高精度sEMG解码。
Scaling and Distilling Transformer Models for sEMG
- 在sEMG数据上扩展Transformer至110M参数，显著提升跨用户性能。
- 100M以上参数模型可压缩至50分之一，性能损失小于1.5%。
- 适合实时、高复杂度的肌电人机交互应用。
表面肌电（sEMG）信号为开发创新人机接口提供了可能，能够反映肌肉活动。然而，训练数据量有限以及部署时计算资源受限，限制了对大模型规模在sEMG任务中的探索。本文证明，原始Transformer模型可在sEMG数据上有效扩展至110M参数，显著提升跨用户性能，超越此前研究普遍采用的<10M参数范围。我们进一步表明，超过100M参数的模型可通过知识蒸馏压缩至50倍更小，性能损失低于1.5%绝对值。这一成果实现了高效且表达能力强的模型，适用于真实环境中复杂的实时sEMG任务。
原文摘要 · Abstract (English)
Surface electromyography (sEMG) signals offer a promising avenue for developing innovative human-computer interfaces by providing insights into muscular activity. However, the limited volume of training data and computational constraints during deployment have restricted the investigation of scaling up the model size for solving sEMG tasks. In this paper, we demonstrate that vanilla transformer models can be effectively scaled up on sEMG data and yield improved cross-user performance up to 110M parameters, surpassing the model size regime investigated in other sEMG research (usually <10M parameters). We show that >100M-parameter models can be effectively distilled into models 50x smaller with minimal loss of performance (<1.5% absolute). This results in efficient and expressive models suitable for complex real-time sEMG tasks in real-world environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。