通过融合前后帧视觉信息，提升机器人任务的视觉-语言-动作模型性能。
TTF-VLA: Temporal Token Fusion via Pixel-Attention Integration for Vision-Language-Action Models
- 用像素差异与注意力机制双重判断，选择性融合历史与当前视觉特征。
- 在真实机器人任务中提升8.7%成功率，基准上平均提高4.0个百分点。
- 无需训练即可适配多种模型，适合追求高效鲁棒性的机器人系统开发者。
视觉-语言-动作（VLA）模型通常逐帧处理视觉输入，忽视了机器人操作任务中固有的时间连续性，导致对视觉噪声敏感且无法利用相邻帧间的强关联。本文提出无需训练的时序标记融合（TTF）方法，通过结合高效的灰度像素差分分析与基于注意力的语义相关性评估，实现选择性时序标记融合，并采用硬融合策略与关键帧锚定防止误差累积。在LIBERO、SimplerEnv及真实机器人任务上的实验证明，该方法显著提升性能：在LIBERO上平均提升4.0个百分点（72.4%对比68.4%基线），在SimplerEnv跨环境验证中相对提升4.8%，真实机器人任务中相对提升8.7%。TTF具有模型无关性，可适配OpenVLA与VLA-Cache架构。值得注意的是，实验发现选择性重用查询矩阵反而提升性能，揭示了直接复用KQV矩阵在加速计算的同时改善任务成功率的新方向。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models process visual inputs independently at each timestep, discarding valuable temporal information inherent in robotic manipulation tasks. This frame-by-frame processing makes models vulnerable to visual noise while ignoring the substantial coherence between consecutive frames in manipulation sequences. We propose Temporal Token Fusion (TTF), a training-free approach that intelligently integrates historical and current visual representations to enhance VLA inference quality. Our method employs dual-dimension detection combining efficient grayscale pixel difference analysis with attention-based semantic relevance assessment, enabling selective temporal token fusion through hard fusion strategies and keyframe anchoring to prevent error accumulation. Comprehensive experiments across LIBERO, SimplerEnv, and real robot tasks demonstrate consistent improvements: 4.0 percentage points average on LIBERO (72.4\% vs 68.4\% baseline), cross-environment validation on SimplerEnv (4.8\% relative improvement), and 8.7\% relative improvement on real robot tasks. Our approach proves model-agnostic, working across OpenVLA and VLA-Cache architectures. Notably, TTF reveals that selective Query matrix reuse in attention mechanisms enhances rather than compromises performance, suggesting promising directions for direct KQV matrix reuse strategies that achieve computational acceleration while improving task success rates.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。