用大模型实现无需标注的物体识别与抓取，让服务机器人更智能。
RoboCup@Home 2024 OPL Winner NimbRo: Anthropomorphic Service Robots using Foundation Models for Perception and Planning
- 基于大模型实现开集词汇的物体分割与文本驱动抓取
- 成功在未标注物体上通过文字描述完成分割与抓取任务
- 结合大语言模型提升自然语言理解与任务规划能力
本文介绍尼姆罗团队在2024年荷兰埃因霍温举行的RoboCup@Home开放平台联赛中的方法与贡献。本年度比赛特别强调开集词汇物体分割与抓取技术，以克服传统监督视觉方法对标签数据的依赖。我们成功实现了仅通过文本描述对未标注物体进行分割与抓取。此外，系统广泛采用大语言模型（LLMs）进行自然语言理解和任务规划。整个比赛过程中，方法展现出良好的鲁棒性与泛化能力。相关演示视频可在线获取。
原文摘要 · Abstract (English)
We present the approaches and contributions of the winning team NimbRo@Home at the RoboCup@Home 2024 competition in the Open Platform League held in Eindhoven, NL. Further, we describe our hardware setup and give an overview of the results for the task stages and the final demonstration. For this year's competition, we put a special emphasis on open-vocabulary object segmentation and grasping approaches that overcome the labeling overhead of supervised vision approaches, commonly used in RoboCup@Home. We successfully demonstrated that we can segment and grasp non-labeled objects by text descriptions. Further, we extensively employed LLMs for natural language understanding and task planning. Throughout the competition, our approaches showed robustness and generalization capabilities. A video of our performance can be found online.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。