无需重建或对比学习,用正则化实现离散世界模型
Discrete World Models via Regularization
- 通过熵、独立性与局部性正则化,约束隐变量分布
- 在两个组合结构基准上,显式表示精度优于重构类方法
- 适合需要符号推理与规划的强化学习场景
世界模型旨在以紧凑的隐空间捕捉环境状态与动态。使用布尔型状态表示对搜索启发式、符号推理与规划尤为有益。现有方法依赖解码器重构或对比/奖励信号保持隐变量信息量。本文提出无重构、无对比的离散世界模型正则化方法(DWMR):引入一种新损失函数,将隐状态预测与专用正则项耦合。该正则项通过方差、相关性和共偏度惩罚,最大化表示比特的熵与独立性,同时施加稀疏动作变化的局部性先验。为提升优化鲁棒性,还设计了新颖训练策略。在两个具有组合结构的基准测试中,DWMR 学习到的表示与转移更准确。此外,可与辅助重构解码器结合,进一步提升性能。
原文摘要 · Abstract (English)
World models aim to capture the states and dynamics of an environment in a compact latent space. Moreover, using Boolean state representations is particularly useful for search heuristics and symbolic reasoning and planning. Existing approaches keep latents informative via decoder-based reconstruction, or instead via contrastive or reward signals. In this work, we introduce Discrete World Models via Regularization (DWMR): a reconstruction-free and contrastive-free method for unsupervised Boolean world-model learning. In particular, we introduce a novel world-modeling loss that couples latent prediction with specialized regularizers. Such regularizers maximize the entropy and independence of the representation bits through variance, correlation, and coskewness penalties, while simultaneously enforcing a locality prior for sparse action changes. To enable effective optimization, we also introduce a novel training scheme improving robustness to discrete roll-outs. Experiments on two benchmarks with underlying combinatorial structure show that DWMR learns more accurate representations and transitions than reconstruction-based alternatives. Finally, DWMR can also be paired with an auxiliary reconstruction decoder, and this combination yields additional gains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。