arXiv:2603.00615cs.RO2026-03被引 2

改进机器人模仿学习的数据采样与增强,提升训练效率和泛化能力。

TGM-VLA: Task-Guided Mixup for Sampling-Efficient and Robust Robotic Manipulation

  • 重构关键帧采样策略,内存占用减少80%,训练速度提升5倍。
  • 引入颜色反转投影分支,解决深色物体识别模糊问题。
  • 任务引导混合策略提升抗干扰能力,多目标场景表现更优。

机器人模仿学习的性能受限于数据质量和训练策略。现有RLBench上的采样方法存在关键帧冗余和时间分布不均问题,导致内存使用低效且优化不稳定。此外,将点云投影到多视角图像时采用黑背景,虽比体素方法高效,但常使深色物体难以区分,影响操作。本文提出一种全新综合框架,显著提升模型性能与训练效率:首先优化关键帧采样策略,内存消耗降低80%,训练速度加快5倍;其次增加颜色反转投影分支,有效解决深色物体辨识模糊问题;最后提出任务引导混合技术,根据任务指令动态融合点云与动作热图,大幅增强对干扰物的鲁棒性,并在多目标场景中表现更佳。大量实验表明,本方法在RLBench上达到90.5%成功率,在复杂干扰条件下COLOSSEUM基准上达68.8%。代码与模型权重已开源。

原文摘要 · Abstract (English)

The performance of robotic imitation learning is fundamentally limited by data quality and training strategies. Prevalent sampling strategies on RLBench suffer from severe keyframe redundancy and imbalanced temporal distribution, leading to inefficient memory usage and unstable optimization. Moreover, reprojecting point clouds onto multi-view images with a black background--while more efficient than voxel-based methods--often causes dark objects to be indistinguishable and hard to manipulate. In this work, we propose a novel holistic framework that significantly improves both model performance and training efficiency. First, we redesign and optimize the keyframe sampling strategy, reducing memory consumption by 80% and accelerating training speed by 5x. Second, we augment the model with a color inversion projection branch--a simple yet effective module that resolves the ambiguity of dark objects. Finally, we propose a task-guided mixup technique that dynamically fuses point clouds and action heatmaps according to task instructions, greatly improving robustness to distractors and performance in multi-goal scenarios. Extensive experiments demonstrate that our method achieves state-of-the-art performance with a 90.5% success rate on RLBench and 68.8% on the COLOSSEUM benchmark under challenging interference conditions. Our code and checkpoints are available at https://github.com/PuFanqi23/TGM-VLA.

机器人操控模仿学习数据增强高效训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。