深度影响世界模型预测效果,有时提升、有时反噬,取决于训练配置。
Adaptive Compute in Latent World Models: When Depth Helps, Hurts, or Doesn't Matter
- 通过可变深度退出机制测试不同任务下深度的作用
- 3种表现:深度助益(6/9任务)、反向损害(2/9任务)、无显著影响
- 深度是否有效可提前预测,关键在训练数据与模型结构
针对世界模型的自适应计算——即在每步推演中采用可变深度的早退或深度混合预测器——我们直接检验了额外深度是否带来更优预测。在九个DeepMind Control任务上,使用单步(K=1)和多步(K=4)训练,每个设置八次随机种子,通过预注册指标浅层惩罚率ρ = 浅层退出误差 / 全深度误差进行评估。结果呈现三种模式:深度助益(内在型,6/9任务,ρ最高达8倍)、深度反向损害(反转型,2/9任务,ρ低至0.87倍),或几乎无影响(平坦型)。反转现象源于训练方式而非系统动态:仅在首轮推演监督早退可消除该效应(Δ=+0.28,n=8,分布不重叠),揭示出一种“可路由性陷阱”:对每步深度的监督虽使早退可行,却也训练其超越完整堆栈。该模式可预测:一个仅依赖维度的冻结分类器在训练前即可准确外样本预测,包括极端外推。在Transformer预测器中同样复现反转,但其表现随度量空间、规划时域、编码器、主干网络及——最显著地——训练数据变化。在两个重新训练的任务上,使用高绩效策略数据后,反转与内在权衡均消失,损失不变。在CEM规划器中,ρ能有效预测深度是否带来规划收益。所有阈值与门控机制均在对应计算前确定,包含对初始假设的预注册否定。因此,更多计算是否提升世界模型性能,并非任务本身属性,而是操作配置的产物,具有稳定、可预测、机制驱动的核心。
原文摘要 · Abstract (English)
Adaptive compute for world models -- early-exit or mixture-of-depths predictors that spend variable depth per rollout step -- presumes that extra depth buys better predictions. In autoregressive rollouts, where planning actually happens, that premise requires depth's per-step precision to survive composition. We test it directly with one pre-registered instrument, the shallow penalty rho = err(shallowest-exit rollout)/err(full-depth rollout), on nine DeepMind Control tasks under matched single-step (K=1) and multi-step (K=4) training, eight seeds each. Three regimes emerge: depth helps (intrinsic, 6/9 tasks, rho up to 8x), depth actively hurts (inversion, 2/9, rho down to 0.87x), or depth barely matters (flat). The inversion is created by training, not the dynamics: supervising early exits only at the first rollout step erases it (Delta=+0.28, n=8, non-overlapping distributions) -- a routability catch-22: the per-step deep supervision that makes exits routable also trains them to out-roll the full stack. The regime is predictable: a frozen dimensionality-only classifier, committed before training, labels held-out tasks correctly out-of-sample, including an extreme extrapolation. The inversion reproduces under a transformer predictor, yet its manifestation is configuration-dependent, shifting with metric space, horizon, encoder, backbone, and -- most strongly -- training data: on the two tasks we retrained, competent-policy data removes both the inversion and the intrinsic tradeoff, loss unchanged. In a CEM planner, rho predicts whether planning benefits from depth. Every threshold and gate was committed before the corresponding compute, including a pre-registered negative for the motivating hypothesis. Whether more compute helps a world model is not a task property; it is a property of the operating configuration, with a stable, predictable, mechanism-backed core.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。