提出新方法诊断潜在世界模型的规划能力，发现奖励预测无法反映真实性能差距。
Operator-on-F complements value-equivalence: a planning-time diagnostic for latent world models
- 用模型自身预测器对比潜空间推进结果与环境差异，评估规划误差
- 模型规模从317M跃升时，规划回报骤降至0.9，但奖励误差仍稳定在0.091
- 该诊断能区分不同架构性能，尤其适合评估复杂模型的规划可靠性
基于模型的强化学习中，世界模型评估通常关注预测奖励和价值的能力，但这可能遗漏潜空间滚动中的规划相关误差。本文引入互补诊断方法 operator-on-F，通过模型自身预测器比较模型在可观测子集F上的k步潜空间前推与环境的差异。在cheetah-run任务上，不同规模的TD-MPC2模型，奖励预测误差始终维持在[0.028, 0.091]区间，仅约3倍变化；未归一化的贝尔曼残差与奖励误差与回报的相关性极弱（斯皮尔曼相关系数分别为-0.10和-0.30）。而操作器误差在相同规模下跨度达0.28至2.62。当模型规模达到317M时，操作器误差为2.62，比0.28–0.36集群高一个数量级，此时规划回报骤降至0.9，而奖励误差虽为最高（0.091），仍在原有小范围内。操作器误差与回报损失的秩相关系数为-0.90（锚定自助95%置信区间[-0.90, -0.70]，样本数n=5；剔除任一规模后仍不低于-0.80）。此外，在TD-MPC2与纯自监督学习（pure-SSL）潜空间世界模型的跨架构对比中，该方法也能提供有判别力的信息。该诊断补充而非取代价值等价性检验。
原文摘要 · Abstract (English)
World-model evaluation for model-based reinforcement learning typically asks whether the learned model predicts reward and value well, which can leave planning-relevant errors in the model's latent rollouts unmeasured. We introduce a complementary diagnostic, operator-on-F, that compares a model's k-step latent pushforward to the environment's on an observable subset F, using the model's own predictor. On a TD-MPC2 size sweep over cheetah-run, reward-prediction error stays within [0.028, 0.091] for every model size - only about 3x variation - so an unnormalized reward-fit check has narrow resolution to distinguish them; the (unnormalized) Bellman residual and reward error themselves have weak relationships with return (Spearman -0.10 and -0.30). Operator error spans 0.28 to 2.62 over the same sizes. At 317M the operator error is 2.62 - an order of magnitude above the 0.28-0.36 cluster - and the planning return collapses to 0.9, while reward-prediction error (0.091) is the highest of the five but stays within the same small [0.028, 0.091] range as the rest of the sweep. The rank correlation between operator error and return loss is -0.90 (anchor-bootstrap 95% CI [-0.90, -0.70] at n=5 sizes; leave-one-out removal of any single size leaves it at -0.80 or stronger). The operator also returns informative, architecture-discriminating estimates in a cross-architecture comparison between TD-MPC2 and a pure-SSL latent world model. The operator diagnostic complements value-equivalence rather than replacing it.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。