用分层时空动作标记器提升机器人少样本模仿学习性能
A Hierarchical Spatiotemporal Action Tokenizer for In-Context Imitation Learning in Robotics
- 分两级向量量化，先细粒度聚类再粗粒度聚合动作
- 在多个仿真与真实机器人任务中达到当前最优效果
- 适合需要快速适应新任务的机器人系统研究者
我们提出一种新型分层时空动作标记器，用于上下文内模仿学习。该方法采用两级向量量化：低层将输入动作分配至细粒度子簇，高层进一步将子簇映射到主簇。该分层策略主要通过重构输入动作来利用空间信息，优于非分层方法。进一步地，我们结合时空线索，构建了分层时空动作标记器（HiST-AT），实现多层级聚类，并同时恢复输入动作及其时间戳。在多个仿真与真实机器人操作基准上的广泛评估表明，该方法在上下文内模仿学习中达到了新的最先进水平。
原文摘要 · Abstract (English)
We present a novel hierarchical spatiotemporal action tokenizer for in-context imitation learning. We first propose a hierarchical approach, which consists of two successive levels of vector quantization. In particular, the lower level assigns input actions to fine-grained subclusters, while the higher level further maps fine-grained subclusters to clusters. Our hierarchical approach outperforms the non-hierarchical counterpart, while mainly exploiting spatial information by reconstructing input actions. Furthermore, we extend our approach by utilizing both spatial and temporal cues, forming a hierarchical spatiotemporal action tokenizer, namely HiST-AT. Specifically, our hierarchical spatiotemporal approach conducts multi-level clustering, while simultaneously recovering input actions and their associated timestamps. Finally, extensive evaluations on multiple simulation and real robotic manipulation benchmarks show that our approach establishes a new state-of-the-art performance in in-context imitation learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。