用时序差分法提升机器人视觉-语言-动作模型的置信度校准能力。
Temporal Difference Calibration in Sequential Tasks: Application to Vision-Language-Action Models
- 引入扩展的Brier分数，将置信度校准与强化学习价值函数关联。
- 在模拟和真实机器人数据上，校准后性能优于当前最优方法。
- 校准后的单步动作概率可提供媲美复杂模型的不确定性估计。
近期视觉-语言-动作（VLA）模型在机器人任务中凸显了序列任务中可靠不确定性量化的重要性。然而，在仅观测部分轨迹的情况下，评估与改进校准仍基本未被探索。本文提出面向周期性任务的序列校准框架，任务成功置信度沿任务过程生成，而成功结果在任务结束时才确定。我们引入Brier分数的序列扩展，并证明对于二元结果，其风险最小化器恰好对应VLA策略的价值函数。这一联系将不确定性校准与强化学习相连接，使得时序差分（TD）价值估计成为时间上的合理校准机制。实验表明，使用TD校准的方法在模拟和真实机器人数据上均优于现有最先进方法。有趣的是，当采用TD校准时，VLA模型的单步动作概率可产生具有竞争力的不确定性估计，这与此前采用其他校准技术所得结论形成对比。
原文摘要 · Abstract (English)
Recent advances in vision-language-action (VLA) models for robotics have highlighted the importance of reliable uncertainty quantification in sequential tasks. However, assessing and improving calibration in such settings remains mostly unexplored, especially when only partial trajectories are observed. In this work, we formulate sequential calibration for episodic tasks, where task-success confidence is produced along an episode, while success is determined at the end of it. We introduce a sequential extension of the Brier score and show that, for binary outcomes, its risk minimizer coincides with the VLA policy's value function. This connection bridges uncertainty calibration and reinforcement learning, enabling the use of temporal-difference (TD) value estimation as a principled calibration mechanism over time. We empirically show that TD calibration improves performance relative to the state-of-the-art on simulated and real-robot data. Interestingly, we show that when calibrated using TD, the VLA's single-step action probabilities can yield competitive uncertainty estimates, in contrast to recent findings that employed different calibration techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。