arXiv:2505.11366cs.ROcs.HC2025-05被引 3

用少量非侵入式输入实现高维机械臂精准控制,提升瘫痪患者操作能力。

Learning Multimodal AI Algorithms for Amplifying Limited User Input into High-dimensional Control Space

  • 融合用户低维输入与环境感知,通过强化学习动态理解意图。
  • 零样本仿真到现实迁移后,任务成功率92.88%,响应速度达侵入式技术水平。
  • 适合神经康复、人机交互领域研究者及残障辅助设备开发者。

当前侵入式辅助技术虽能推断高维运动控制信号,但面临公众接受度低、寿命短及商业化困难等问题。非侵入式方案则常依赖易受干扰的信号,需长期训练,难以实现灵巧任务的鲁棒高维控制。本研究提出一种以人为中心的多模态人工智能方法,作为丧失运动功能的智能补偿机制,使严重瘫痪患者仅通过有限且非侵入式输入即可操控高维辅助设备(如灵巧机械臂)。相比现有最先进非侵入式方法,本研究提出的上下文感知多模态共享自主框架,结合深度强化学习算法,融合低维用户输入与实时环境感知,实现复杂灵巧操作(如抓取放置)中人类意图的自适应、动态、智能解析。在50,000次计算机模拟训练后,基于合成用户的ARAS系统首次成功实现闭环人机协同范式,性能超越现有最先进共享自主算法。经零样本仿真到现实迁移,在23名人类受试者上评估显示,系统具备高精度动态意图识别能力与平稳稳定的3D轨迹控制,任务成功率达92.88%,完成时间与最先进侵入式技术相当。

原文摘要 · Abstract (English)

Current invasive assistive technologies are designed to infer high-dimensional motor control signals from severely paralyzed patients. However, they face significant challenges, including public acceptance, limited longevity, and barriers to commercialization. Meanwhile, noninvasive alternatives often rely on artifact-prone signals, require lengthy user training, and struggle to deliver robust high-dimensional control for dexterous tasks. To address these issues, this study introduces a novel human-centered multimodal AI approach as intelligent compensatory mechanisms for lost motor functions that could potentially enable patients with severe paralysis to control high-dimensional assistive devices, such as dexterous robotic arms, using limited and noninvasive inputs. In contrast to the current state-of-the-art (SoTA) noninvasive approaches, our context-aware, multimodal shared-autonomy framework integrates deep reinforcement learning algorithms to blend limited low-dimensional user input with real-time environmental perception, enabling adaptive, dynamic, and intelligent interpretation of human intent for complex dexterous manipulation tasks, such as pick-and-place. The results from our ARAS (Adaptive Reinforcement learning for Amplification of limited inputs in Shared autonomy) trained with synthetic users over 50,000 computer simulation episodes demonstrated the first successful implementation of the proposed closed-loop human-in-the-loop paradigm, outperforming the SoTA shared autonomy algorithms. Following a zero-shot sim-to-real transfer, ARAS was evaluated on 23 human subjects, demonstrating high accuracy in dynamic intent detection and smooth, stable 3D trajectory control for dexterous pick-and-place tasks. ARAS user study achieved a high task success rate of 92.88%, with short completion times comparable to those of SoTA invasive assistive technologies.

人机交互康复机器人强化学习多模态控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。