无需验证器,通过内部分布差异提升视觉语言动作模型精度
Verifier-free Test-Time Sampling for Vision-Language-Action Models
- 用模型自身输出的分布差异做置信度,选出最优动作
- 在多个仿真与真实场景中稳定提升基线模型性能
- 无需额外训练,适合对精度要求高的机器人控制任务
视觉语言动作模型(VLAs)在机器人控制中表现优异,但在高精度任务上受限于单次推理范式。现有测试时扩展方法依赖外部验证器,需额外训练且泛化能力差。本文提出掩码分布引导选择(MG-Select),一种无需额外训练或外部模块的测试时扩展框架。该方法利用参考动作令牌分布与当前输出间的KL散度作为置信度指标,从多个候选动作中选取最优解。参考分布由同一VLA在随机掩码状态和语言条件输入下生成,既反映动作不确定性,又保持与目标任务分布一致。此外,提出联合训练策略,通过在状态和语言条件上施加丢弃(dropout),使模型同时学习条件与无条件分布,进一步优化参考分布质量。实验表明,MG-Select通过任务相关的条件掩码提供可靠参考,在多样化的仿真与真实世界基准上持续提升基线模型性能。
原文摘要 · Abstract (English)
Vision-Language-Action models (VLAs) have demonstrated remarkable performance in robot control. However, they remain fundamentally limited in tasks that require high precision due to their single-inference paradigm. While test-time scaling approaches using external verifiers have shown promise, they require additional training and fail to generalize to unseen conditions. We propose Masking Distribution Guided Selection (MG-Select), a novel test-time scaling framework for VLAs that leverages the model's internal properties without requiring additional training or external modules. Our approach utilizes KL divergence from a reference action token distribution as a confidence metric for selecting the optimal action from multiple candidates. We introduce a reference distribution generated by the same VLA but with randomly masked states and language conditions as inputs, providing action uncertainty while remaining aligned with the target task distribution. Additionally, we propose a joint training strategy that enables the model to learn both conditional and unconditional distributions by applying dropout to state and language conditions, thereby further improving the quality of the reference distribution. Our experiments demonstrate that MG-Select provides a reliable reference for action selection through task-relevant condition masking and consistently improves base models across diverse simulation and real-world benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。