用大模型+迭代重规划，让机器人自动完成超市货架补货。
Task Planning for Mobile Manipulation in Retail Stores using Foundation Models with Iterative Re-planning
- 用大语言模型和视觉语言模型理解任务指令
- 通过用户提示与反馈实现错误修正的迭代重规划
- 在仿真环境中验证了补货任务的端到端可行性
零售、仓储和物流等行业的自动化可提升效率、降低成本，并缓解劳动力短缺带来的中断。以往自动化主要聚焦于包装、分拣等结构化后端场景。随着移动操作机器人硬件与基础模型的发展，自动化已可应用于更复杂、以人类为中心的零售环境，如超市货架。本文提出一种基于大语言模型（LLMs）和视觉语言模型（VLMs）的任务规划方法，用于解决超市等场景中的补货问题。系统部署于自研全向移动操作平台，支持用户驱动的提示输入与基于反馈的迭代重规划，实现错误纠正。整个端到端系统在PyBullet仿真环境中对拾取-放置任务进行了验证。
原文摘要 · Abstract (English)
Automation in industries such as retail, warehousing and logistics presents opportunities for greater throughput, cost reduction and mitigation of disruptions from labour shortages. Previously, such efforts have focused on back-room operations involving packing and sorting in relatively structured environments. With advances in robotic mobile manipulation hardware and foundation models, automation can now be applied to more variable and human-centric environments such as retail store shelves. In this work, we present a task-planning approach using Large Language Models (LLMs) and Vision-Language Models (VLMs) to address the restocking problem in retail scenarios such as supermarkets. We demonstrate this system on a custom omnidirectional mobile manipulation platform, with user-driven prompts and a feedback-based iterative re-planning approach for error correction. The end-to-end system is validated in a PyBullet simulation environment for pick-and-place tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。