用未来信息做零成本纠错,让当前模型更准
Privileged Foresight Distillation: Zero-Cost Future Correction for World Action Models

- 把未来信息当作可压缩的纠错信号,通过蒸馏注入当前模型
- 在LIBERO和RoboTwin上提升性能,推理时无需生成未来视频
- 适合想用现有模型提升精度又不想改架构的研究者
世界动作模型在训练中同时预测未来视频和动作,但未来预测分支在推理时可移除而性能几乎不变,暗示其可能仅作为正则化。本文提出该分支实际提供了一种基于动作条件的未来修正:当前模型仅部分捕获这种修正。为此,作者将特权远见定义为动作去噪方向上的残差——即基于真实未来与仅基于当前帧的预测之差——并提出特权远见蒸馏(PFD),将训练时教师模型的该残差,通过共享主干但不同注意力掩码的方式,蒸馏到一个轻量适配器中,学生模型仅依赖当前帧。控制实验验证了该增益源于真正的未来条件修正。实证显示,PFD在LIBERO和RoboTwin基准上持续提升性能,且推理延迟几乎不变。这一视角重新定义了未来信息的作用:不是待预测的目标,也不是正则项,而是可蒸馏的修正信号。
原文摘要 · Abstract (English)
World action models jointly predict future video and action during training, raising an open question about what role the future-prediction branch actually plays. A recent finding shows that this branch can be removed at inference with little to no loss on common manipulation benchmarks, suggesting that future information may act merely as a regularizer on the shared visual backbone. We propose instead that joint training induces an action-conditioned correction that privileged future observations impose on action denoising, and that current-only policies capture this correction only partially. Making the account precise, we formulate privileged foresight as a residual in the action-denoising direction -- the difference between what a model predicts given the true future and what it predicts given only the current frame -- and introduce \emph{Privileged Foresight Distillation (PFD)}, which transfers this residual from a training-time teacher into a small adapter on a current-only student. The teacher and student share the same backbone and differ only in the attention mask over video tokens; future video is never generated at inference. Controlled experiments verify that this gain reflects a genuine future-conditioned correction rather than a side effect of capacity or regularization. Empirically, PFD achieves consistent improvements on LIBERO and RoboTwin manipulation benchmarks while preserving the current-only inference interface at negligible added latency. This view reframes the role of future information in world action models: not as a target to predict, nor as a regularizer to absorb, but as a compressible correction to be distilled.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。