arXiv:2607.03751cs.RO2026-07

让冻结的视觉语言动作模型学会评估动作后果,提升泛化能力。

Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models

论文配图:Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models
图 1 · 摘自论文原文
  • 用蒙特卡洛树搜索探索模型输出分布,生成带回报的轨迹数据
  • 将动作评估能力蒸馏为轻量Q值模型,部署时选最优动作
  • 无需模拟器,可显著提升未见任务成功率,更省钱高效

视觉-语言-动作(VLA)模型通过大规模预训练获得广泛的具身能力,但其泛化能力远不如大语言模型和视觉语言模型。当前主流方法是通过监督微调或强化学习进行后训练,虽能提升特定任务表现,却削弱了预训练带来的通用性。我们发现关键瓶颈在于动作生成之外的动作评估环节。诊断性实验显示,冻结的VLA本身已具备良好行为,整体成功率从pass@1的33%提升至pass@32的92%。受此启发,我们提出SVA框架:先在仿真中用蒙特卡洛树搜索充分探索VLA输出分布,收集多样轨迹并标注实际回报;再将这些知识蒸馏为轻量级Q值模型,预测候选动作的预期后果;部署时,冻结的VLA提出多个动作候选,评估器选择不确定性正则化后Q值最高的动作,无需模拟器访问。通过解耦动作生成与后果评估,SVA在保留VLA骨干泛化能力的同时,大幅提升任务成功率。跨具身基准测试表明,SVA在未见任务上持续提升泛化性能,并表现出强的测试时扩展能力。令人惊讶的是,90亿参数的VLA在使用SVA后,表现超过270亿参数的基线模型7个百分点,且推理延迟降低27%,表明扩大测试时评估比扩大模型规模更高效。

原文摘要 · Abstract (English)

Vision-Language-Action (VLA) models acquire broad embodied capabilities through large-scale pretraining, yet their generalization remains far more fragile than that of LLMs and VLMs. The prevailing remedy, post-training via supervised fine-tuning or reinforcement learning, improves task-specific performance but narrows the generalist capability that makes pretraining valuable. We identify a key bottleneck: VLA failures stem not only from action generation but also from action evaluation. A diagnostic pass@k study confirms that frozen VLAs already contain competent behaviors in their output distribution, with overall success rates rising from 33% at pass@1 to 92% at pass@32. Inspired by this, we propose SVA (Search, Value, and Act), a simple framework that equips frozen VLA policies with long-term consequence awareness. SVA first uses Monte-Carlo tree search in simulation to fully explore the VLA's output distribution and collect diverse trajectories annotated with empirical returns; this knowledge is then distilled into a lightweight Q-value model that predicts the expected consequence of candidate actions; at deployment, the frozen VLA proposes multiple candidates and the evaluator selects the one with the highest uncertainty-regularized Q-value, requiring no simulator access. By decoupling action proposal from consequence evaluation, SVA preserves the generalization capacity of the VLA backbone while substantially improving task success rates. Experiments across embodied benchmarks show that SVA consistently improves generalization on unseen tasks and exhibits strong test-time scaling behavior. Strikingly, SVA enables a 9B VLA to outperform a 27B VLA by 7 points at 27% lower inference latency, suggesting that scaling test-time evaluation is more cost-effective than scaling model size.

VLA动作评估树搜索泛化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。