用眼神控制机械臂抓取物体，提升行动障碍者操作的直观性与效率。
RaycastGrasp: Eye-Gaze Interaction with Wearable Devices for Robotic Manipulation
- 通过可穿戴混合现实头显实现第一视角眼动交互
- 单次操作意图识别准确率超88%，延迟更低
- 适合需要自然交互的辅助机器人应用场景
机器人操作正越来越多地用于帮助行动障碍者取物。然而，主流的摇杆控制界面因精度要求高、参考系不直观而难以使用。尽管人机交互新范式不断涌现，许多方案仍依赖外部屏幕或受限控制方式，影响直观性与可及性。为此，我们提出一种基于可穿戴混合现实（MR）头显的以我为中心、眼动引导的机器人操作接口。系统让用户从第一人称视角通过自然注视完成对真实物体的交互，辅以增强视觉提示确认意图，并结合预训练视觉模型与机械臂实现意图识别与物体操作。实验表明，该方法显著提升操作准确率，降低系统延迟，在多个真实场景中实现超过88%的单次操作意图与物体识别准确率。结果验证了系统在提升直观性与可及性方面的有效性，凸显其在辅助机器人应用中的实际价值。
原文摘要 · Abstract (English)
Robotic manipulators are increasingly used to assist individuals with mobility impairments in object retrieval. However, the predominant joystick-based control interfaces can be challenging due to high precision requirements and unintuitive reference frames. Recent advances in human-robot interaction have explored alternative modalities, yet many solutions still rely on external screens or restrictive control schemes, limiting their intuitiveness and accessibility. To address these challenges, we present an egocentric, gaze-guided robotic manipulation interface that leverages a wearable Mixed Reality (MR) headset. Our system enables users to interact seamlessly with real-world objects using natural gaze fixation from a first-person perspective, while providing augmented visual cues to confirm intent and leveraging a pretrained vision model and robotic arm for intent recognition and object manipulation. Experimental results demonstrate that our approach significantly improves manipulation accuracy, reduces system latency, and achieves single-pass intention and object recognition accuracy greater than 88% across multiple real-world scenarios. These results demonstrate the system's effectiveness in enhancing intuitiveness and accessibility, underscoring its practical significance for assistive robotics applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。