用大模型让机器人根据任务选择合适抓法，提升人机协作效率
LLM-Handover:Exploiting LLMs for Task-Oriented Robot-Human Handovers
- 结合大模型与部件分割，智能判断物体哪部分该抓
- 零样本实验成功率达83%，适应各种使用场景
- 用户测试中86%更偏好该方法，操作更自然
高效的人机协作依赖于任务导向的物品交接，即机器人以支持人类后续使用的姿态递送物品。然而，现有方法常忽略交接后的人类动作，依赖假设限制了泛化能力。为此，我们提出LLM-Handover框架，将大语言模型（LLM）推理与部件分割结合，实现上下文感知的抓取选择与执行。给定RGB-D图像和任务描述，系统推断相关物体部件，并选择优化交接后可用性的抓取方式。为支持评估，我们构建了一个包含60个家居物品、12个类别的新数据集，每件物品均有详细部件标注。实验表明，该方法提升了现有先进部件分割方法在人机交接中的表现；进一步验证显示，LLM-Handover在抓取成功率上更优，且能更好适应交接后的任务约束。硬件实验中，在零样本设置下对常规与非常规交接任务均达到83%的成功率。用户研究结果表明，该方法带来的交接更直观、上下文敏感，86%的参与者表示更倾向使用。
原文摘要 · Abstract (English)
Effective human-robot collaboration depends on task-oriented handovers, where robots present objects in ways that support the partners intended use. However, many existing approaches neglect the humans post-handover action, relying on assumptions that limit generalizability. To address this gap, we propose LLM-Handover, a novel framework that integrates large language model (LLM)-based reasoning with part segmentation to enable context-aware grasp selection and execution. Given an RGB-D image and a task description, our system infers relevant object parts and selects grasps that optimize post-handover usability. To support evaluation, we introduce a new dataset of 60 household objects spanning 12 categories, each annotated with detailed part labels. We first demonstrate that our approach improves the performance of the used state-of-the-art part segmentation method, in the context of robot-human handovers. Next, we show that LLM-Handover achieves higher grasp success rates and adapts better to post-handover task constraints. During hardware experiments, we achieve a success rate of 83% in a zero-shot setting over conventional and unconventional post-handover tasks. Finally, our user study underlines that our method enables more intuitive, context-aware handovers, with participants preferring it in 86% of cases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。