用增强现实中的虚拟物体指导机器人完成复杂操作
GhostObjects: Instructing Robots by Manipulating Spatially Aligned Virtual Twins in Augmented Reality
- 用户在AR中操作与实物对齐的虚拟物体来发指令
- 支持多物体选择和自动回位，可实现复杂任务
- 适合非技术用户快速训练机器人执行定制化任务
机器人自主能力日益增强，但个性化指令仍需人类参与。我们提出一种新方法：通过在增强现实（AR）中直接操控与真实物体空间对齐的、等尺寸的虚拟孪生体（GhostObjects），向机器人发出指令。用户无需通过示教或遥操作，即可精准设定物理目标和空间参数。系统支持真实世界中的多物体框选及默认位置自动回位，显著扩展了任务范围，不再局限于简单的抓取放置。该方法提升了人机交互效率与操作精度。
原文摘要 · Abstract (English)
Robots are increasingly capable of autonomous operations, yet human interaction remains essential for issuing personalized instructions. Instead of directly controlling robots through Programming by Demonstration (PbD) or teleoperation, we propose giving instructions by interacting with GhostObjects-world-aligned, life-size virtual twins of physical objects-in augmented reality (AR). By direct manipulation of GhostObjects, users can precisely specify physical goals and spatial parameters, with features including real-world lasso selection of multiple objects and snapping back to default positions, enabling tasks beyond simple pick-and-place.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。