基础模型让机器人实现从语言到动作的全流程迁移。
Are Foundation Models the Route to Full-Stack Transfer in Robotics?
- 用大模型与Transformer统一抽象层间的迁移机制。
- 跨模态模型使语言、视觉与动作技能可共享知识。
- 适合关注机器人通用智能与迁移学习的研究者。
人类和机器人在不同抽象层次上实现迁移学习，从高层语言到底层运动技能。本文综述了基础模型与Transformer网络在这些层次上的影响，推动机器人更接近“全栈迁移”。从机器人迁移学习视角审视大语言模型（LLMs）、视觉语言模型（VLMs）和视觉-语言-动作模型（VLAs），揭示了超越具体实现的通用迁移概念。同时探讨了基础模型时代机器人数据采集与迁移基准的挑战。我们预期基础模型将成为实现全栈迁移的关键技术并持续引领这一方向。
原文摘要 · Abstract (English)
In humans and robots alike, transfer learning occurs at different levels of abstraction, from high-level linguistic transfer to low-level transfer of motor skills. In this article, we provide an overview of the impact that foundation models and transformer networks have had on these different levels, bringing robots closer than ever to "full-stack transfer". Considering LLMs, VLMs and VLAs from a robotic transfer learning perspective allows us to highlight recurring concepts for transfer, beyond specific implementations. We also consider the challenges of data collection and transfer benchmarks for robotics in the age of foundation models. Are foundation models the route to full-stack transfer in robotics? Our expectation is that they will certainly stay on this route as a key technology.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。