通过肢体姿态与几何模型实现机器人协作中的指指点位精准定位
Evaluating Pointing Gestures for Target Selection in Human-Robot Collaboration
- 基于肩腕延伸的几何模型从RGB-D流中提取指指点位数据
- 在典型任务中实现90%以上的指代目标准确率
- 适合需要多模态交互的工业协作机器人系统开发
指指点位是人机协作中常用的交互方式，广泛应用于目标选择和工业流程引导。本研究提出一种在平面工作区中定位指指点位的方法，利用姿态估计与基于肩腕延伸的简单几何模型，从RGB-D数据流中提取手势信息。研究设计了严谨的评估方法与全面的分析框架，用于评估指指点位在典型机器人任务中的表现。除了工具精度评估外，该系统还集成于一个概念验证型机器人平台，包含物体检测、语音转录与语音合成模块，展示了多模态融合在协作应用中的可行性。最后，讨论了工具的局限性与性能边界，以理解其在多模态机器人系统中的角色。所有代码与资源已开源：https://github.com/NMKsas/gesture_pointer.git。
原文摘要 · Abstract (English)
Pointing gestures are a common interaction method used in Human-Robot Collaboration for various tasks, ranging from selecting targets to guiding industrial processes. This study introduces a method for localizing pointed targets within a planar workspace. The approach employs pose estimation, and a simple geometric model based on shoulder-wrist extension to extract gesturing data from an RGB-D stream. The study proposes a rigorous methodology and comprehensive analysis for evaluating pointing gestures and target selection in typical robotic tasks. In addition to evaluating tool accuracy, the tool is integrated into a proof-of-concept robotic system, which includes object detection, speech transcription, and speech synthesis to demonstrate the integration of multiple modalities in a collaborative application. Finally, a discussion over tool limitations and performance is provided to understand its role in multimodal robotic systems. All developments are available at: https://github.com/NMKsas/gesture_pointer.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。