基于Transformer的在线强化学习框架，让机器人在真实环境中自主学习与决策。
Online Decision MetaMorphFormer: A Casual Transformer-Based Reinforcement Learning Framework of Universal Embodied Intelligence
- 用统一模型实现自知、环境识别与动作规划，支持持续在线学习。
- 在多个环境和任务中表现强泛化能力，少样本与零样本测试均有效。
- 适合研究通用具身智能的学者，尤其关注实时决策与跨场景迁移者。
运动控制领域的交互式人工智能是一个有趣课题，尤其是当通用知识能适应多种任务与环境时。尽管已有大量研究借助变压器（Transformer）推进强化学习（RL），但多数方法受限于离线训练流程，难以实现探索与泛化能力。为此，我们提出在线决策元变形式框架（ODM），旨在通过统一模型架构实现自我意识、环境识别与动作规划。受认知与行为心理学启发，ODM智能体可从他人经验中学习、基于自身经历实践并识别世界。该框架适用于任意多关节身体的智能体，在不同环境中使用不同类型任务进行训练，并利用大规模预训练数据集快速初始化，目标环境则持续强化通用策略。通过大量在线实验及少样本、零样本环境测试验证了其性能与泛化能力。研究结果有助于推动具身与认知领域通用人工智能的发展。代码、结果与视频示例见：https://rlodm.github.io/odm/
原文摘要 · Abstract (English)
Interactive artificial intelligence in the motion control field is an interesting topic, especially when universal knowledge is adaptive to multiple tasks and universal environments. Despite there being increasing efforts in the field of Reinforcement Learning (RL) with the aid of transformers, most of them might be limited by the offline training pipeline, which prohibits exploration and generalization abilities. To address this limitation, we propose the framework of Online Decision MetaMorphFormer (ODM) which aims to achieve self-awareness, environment recognition, and action planning through a unified model architecture. Motivated by cognitive and behavioral psychology, an ODM agent is able to learn from others, recognize the world, and practice itself based on its own experience. ODM can also be applied to any arbitrary agent with a multi-joint body, located in different environments, and trained with different types of tasks using large-scale pre-trained datasets. Through the use of pre-trained datasets, ODM can quickly warm up and learn the necessary knowledge to perform the desired task, while the target environment continues to reinforce the universal policy. Extensive online experiments as well as few-shot and zero-shot environmental tests are used to verify ODM's performance and generalization ability. The results of our study contribute to the study of general artificial intelligence in embodied and cognitive fields. Code, results, and video examples can be found on the website \url{https://rlodm.github.io/odm/}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。