arXiv:2606.22540cs.CV2026-06中稿 · ECCV被引 1

提升视觉语言动作模型的执行效率,减少冗余步骤和预测失效。

PolicyTrim: Boosting Intrinsic Policy Efficiency of Vision-Language-Action Models

论文配图:PolicyTrim: Boosting Intrinsic Policy Efficiency of Vision-Language-Action Models
图 1 · 摘自论文原文
  • 通过强化学习优化动作块长度与执行步数,提升模型内在效率。
  • 动作块利用率提升3倍,物理执行步数减少51.4%。
  • 适用于追求高效部署的机器人任务系统,无需牺牲成功率。

视觉语言动作(VLA)模型为机器人操作提供了统一范式,但其实际部署常受执行效率制约。现有工作多聚焦计算效率以降低单步推理延迟,而模型的内在策略效率尚未被充分探索。策略效率由可执行动作块的有效长度与完成任务所需的总物理步数共同决定,二者共同影响执行过程中的前向推理调用次数。我们观察到当前VLA策略存在规划不可靠与动作冗余问题,动作块尾部预测严重退化,且倾向于生成不必要的物理步数。为此,提出基于强化学习的后训练框架PolicyTrim,旨在延长可靠动作块长度并减少冗余物理步数。针对可靠块扩展,采用动态探索策略,显式奖励更长可执行长度的成功完成,逐步推进可信预测时域至经验极限;针对步数效率,设计冗余感知奖励,直接鼓励以更少步数完成任务,同时惩罚不可复现的捷径,有效消除冗余物理动作。在三个基准与三类VLA模型上的大量实验表明,PolicyTrim使动作块利用率提升3倍,物理执行步数减少51.4%,最终实现端到端部署速度最高达5.83倍加速,且不降低任务成功率。

原文摘要 · Abstract (English)

Vision-Language-Action (VLA) models provide a unified paradigm for robotic manipulation, yet their real-world deployment is often bottlenecked by execution efficiency. While existing efforts predominantly focus on compute-centric efficiency to reduce per-step inference latency, the intrinsic \textbf{policy efficiency} of these models remains largely unexplored. Policy efficiency is fundamentally affected by two factors, namely the effective executable length of predicted action chunks and the total physical steps required to complete a task. These two factors jointly determine the total number of forward inference calls during execution. We observe that current VLA policies struggle with planning unreliability and action redundancy, suffering from severe prediction degradation at the tail of action chunks and tending to generate unnecessarily redundant physical steps. To address this, we propose \textbf{PolicyTrim}, a reinforcement learning-based post-training framework that extends the reliable action chunk length and reduces redundant physical steps. For reliable chunk extension, we employ a dynamic exploration strategy that explicitly rewards the successful completion of longer executable lengths, progressively pushing the trustworthy prediction horizon to its empirical limit. For step efficiency, we design a redundancy-aware reward that directly favors successful task completions with fewer steps while penalizing unreproducible shortcuts, effectively eliminating redundant physical actions. Extensive experiments across three benchmarks and three VLA models demonstrate that PolicyTrim improves action chunk utilization by 3$\times$ and reduces physical execution steps by 51.4\%. Ultimately, our framework delivers up to a 5.83$\times$ end-to-end deployment speedup without compromising task success rates.

机器人策略优化效率提升

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。