不用离散化动作空间，直接用连续策略训练自回归模型。
Quantization-Free Autoregressive Action Transformer
- 用无限词汇生成模型直接建模连续动作，避免量化损失
- 在多个仿真机器人任务上达到顶尖性能
- 优化采样策略提升策略回放效果，适合强化学习与模仿学习
当前基于Transformer的模仿学习方法采用离散动作表示，并在潜在代码上训练自回归Transformer解码器。然而，初始量化破坏了动作空间的连续性，限制了生成模型的能力。本文提出一种无量化的方法，利用生成式无限词汇Transformer（GIVT）作为自回归Transformer的直接连续策略参数化方式。该方法简化了模仿学习流程，在多种流行的仿真机器人任务上实现了最先进的性能。通过仔细研究采样算法，进一步提升了策略回放效果。
原文摘要 · Abstract (English)
Current transformer-based imitation learning approaches introduce discrete action representations and train an autoregressive transformer decoder on the resulting latent code. However, the initial quantization breaks the continuous structure of the action space thereby limiting the capabilities of the generative model. We propose a quantization-free method instead that leverages Generative Infinite-Vocabulary Transformers (GIVT) as a direct, continuous policy parametrization for autoregressive transformers. This simplifies the imitation learning pipeline while achieving state-of-the-art performance on a variety of popular simulated robotics tasks. We enhance our policy roll-outs by carefully studying sampling algorithms, further improving the results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。