arXiv:2601.16065cs.CVcs.RO2026-01被引 4

通过动态剪除干扰视觉令牌,提升视觉语言动作模型的执行成功率。

DTP: A Simple yet Effective Distracting Token Pruning Framework for Vision-Language Action Models

  • 动态识别并移除任务无关区域的干扰图像令牌
  • 在SIMPLER基准上实现不同VLA模型的相对成功率提升
  • 无需修改架构或增加输入,适合各类视觉语言动作模型

视觉语言动作(VLA)模型通过利用视觉语言模型(VLM)强大的感知能力,在机器人操作任务中表现突出。然而,默认情况下,VLA模型可能过度关注与任务无关的图像区域,这种现象称为‘干扰令牌’。该行为会干扰模型生成期望的动作令牌,影响任务成功率。本文提出一种简单有效的即插即用式干扰令牌剪枝(DTP)框架,可动态检测并剪除这些干扰图像令牌。通过修正模型的视觉注意力模式,旨在提升任务成功率,并探索不改变原始架构或额外输入下的模型性能上限。在SIMPLER基准(Li et al., 2024)上的实验表明,该方法在多种新型VLA模型中均实现了相对成功率的持续提升,展现出对基于Transformer的VLA模型的广泛适用性。进一步分析显示,所有测试模型的任务成功率与任务无关区域注意力量呈负相关,揭示了VLA模型中普遍存在的现象,为未来研究提供指导。代码已公开于:https://anonymous.4open.science/r/CBD3。

原文摘要 · Abstract (English)

Vision-Language Action (VLA) models have shown remarkable progress in robotic manipulation by leveraging the powerful perception abilities of Vision-Language Models (VLMs) to understand environments and directly output actions. However, by default, VLA models may overly attend to image tokens in the task-irrelevant region, which we describe as 'distracting tokens'. This behavior can disturb the model from the generation of the desired action tokens in each step, affecting the success rate of tasks. In this paper, we introduce a simple yet effective plug-and-play Distracting Token Pruning (DTP) framework, which dynamically detects and prunes these distracting image tokens. By correcting the model's visual attention patterns, we aim to improve the task success rate, as well as exploring the performance upper boundaries of the model without altering its original architecture or adding additional inputs. Experiments on the SIMPLER Benchmark (Li et al., 2024) show that our method consistently achieving relative improvements in task success rates across different types of novel VLA models, demonstrating generalizability to transformer-based VLAs. Further analysis reveals a negative correlation between the task success rate and the amount of attentions in the task-irrelevant region for all models tested, highlighting a common phenomenon of VLA models that could guide future research. We also publish our code at: https://anonymous.4open.science/r/CBD3.

视觉语言动作生成注意力机制机器人控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。