不训练模型，用内置提示增强视觉关系，提升机器人操作精度
IVRA: Improving Visual-Token Relations for Robot Action Policy with Training-Free Hint-Based Guidance
- 通过注入视觉编码器的亲和提示，改进视觉标记间空间关系
- 在低数据下2D任务提升4.2%成功率，3D任务达97.1%准确率
- 无需重训练，适配多种模型与真实机器人场景
许多视觉-语言-动作（VLA）模型将图像块展平为一维标记序列，削弱了精确操作所需的二维空间线索。我们提出IVRA，一种轻量级、无需训练的方法，利用模型内置视觉编码器中已有的亲和提示，增强空间理解，无需外部编码器或重新训练。IVRA将这些亲和信号选择性注入语言模型中实例级特征所在的层。该推理时干预使视觉标记交互重新对齐，更好保留几何结构，同时保持所有模型参数固定。我们在多种VLA架构（LLaRA、OpenVLA、FLOWER）上验证了IVRA的通用性，涵盖2D和3D模拟任务（VIMA与LIBERO），以及多种真实机器人任务。在2D VIMA上，IVRA在低数据条件下相较基线LLaRA平均成功率提升4.2%；在3D LIBERO上，对OpenVLA和FLOWER基线均实现稳定提升，包括基线已达96.3%时提升至97.1%。代码与可视化见：jongwoopark7978.github.io/IVRA
原文摘要 · Abstract (English)
Many Vision-Language-Action (VLA) models flatten image patches into a 1D token sequence, weakening the 2D spatial cues needed for precise manipulation. We introduce IVRA, a lightweight, training-free method that improves spatial understanding by exploiting affinity hints already available in the model's built-in vision encoder, without requiring any external encoder or retraining. IVRA selectively injects these affinity signals into a language-model layer in which instance-level features reside. This inference-time intervention realigns visual-token interactions and better preserves geometric structure while keeping all model parameters fixed. We demonstrate the generality of IVRA by applying it to diverse VLA architectures (LLaRA, OpenVLA, and FLOWER) across simulated benchmarks spanning both 2D and 3D manipulation (VIMA and LIBERO) and on various real-robot tasks. On 2D VIMA, IVRA improves average success by +4.2% over the baseline LLaRA in a low-data regime. On 3D LIBERO, it yields consistent gains over the OpenVLA and FLOWER baselines, including improvements when baseline accuracy is near saturation (96.3% -> 97.1). Code and visualizations are available at: jongwoopark7978.github.io/IVRA
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。