用分位数匹配改进世界模型的隐状态分布，让预测更稳定可靠。
QQWorld: Quantile-Quantile Matching for World Model Regularization
- 用分位数对齐替代原有方法，增强尾部样本的修正梯度
- 在4个控制环境中提升平均规划成功率，尾部分布更薄
- 适合做强化学习中世界模型训练的研究者和工程师
隐空间世界模型通过在紧凑表示空间中预测未来状态实现高效规划，但其性能高度依赖于学习到的隐变量分布质量。LeWorldModel（LeWM）使用Epps-Pulley（EP）目标将隐变量正则化为各向同性高斯分布。我们发现，当存在孤立尾部样本时，EP的校正梯度会迅速消失，导致重尾偏差控制不足。为此，我们提出QQWorld，用分位数-分位数匹配目标替代EP，直接将投影后的隐变量样本与对应排名的高斯分位数对齐，从而在尾部维持有效校正梯度。我们进一步提出跨批处理的交叉分位数（cross-batch QQ），利用前一批次的分离样本扩大排名池，并分析其偏差-方差权衡。在四个控制环境中，QQWorld显著提升了LeWM的平均规划成功率，同时保持更优的高斯对齐效果与更薄的隐变量尾部。
原文摘要 · Abstract (English)
Latent world models enable efficient planning by predicting future states in a compact representation space, but their performance depends critically on the quality of the learned latent distribution. LeWorldModel (LeWM) regularizes its latents toward an isotropic Gaussian using the Epps-Pulley (EP) objective. We show that the corrective gradients of EP rapidly vanish for isolated tail samples, leaving heavy-tailed deviations insufficiently controlled. To address this limitation, we propose QQWorld, which replaces EP with a quantile-quantile matching objective that directly aligns projected latent samples with rank-matched Gaussian quantiles, thereby maintaining effective corrective gradients in the tails. We further develop cross-batch QQ, which enlarges the effective ranking pool using detached samples from previous batches, and characterize its bias-variance trade-off. Across four control environments, QQWorld effectively improves the average planning success rate of LeWM, while consistently yielding better Gaussian alignment and thinner latent tails.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。