提出可零样本迁移的神经符号世界模型，让智能体无需新交互即可适应新任务。
Towards Zero-Shot Task Transfer with Neurosymbolic World Models
- 用符号化组件预测奖励，解耦感知重建与奖励预测
- 在相同符号状态空间上实现零样本任务迁移，无需额外环境交互
- 相比纯神经模型，显著提升跨任务泛化能力，适合需要快速适应的新场景
当前基于模型的强化学习方法通过学习神经世界模型，在潜在空间中规划以改进策略，且无需对环境结构做假设。然而这些模型通常任务依赖：学习到的潜在表示不可解释，与训练任务紧密绑定，难以泛化到新任务。本文提出一种新型世界模型，其奖励预测仅依赖于整个潜在状态中的一小部分结构化符号成分。通过解耦观测重建与奖励预测，所学世界模型可实现零样本适应——即无需进一步环境交互，即可适配新的奖励函数（定义在同一符号状态空间中）。我们讨论了此类神经符号世界模型的主要优势与挑战，并在实验中验证了该方法在跨任务泛化方面的强性能，优于纯神经方法。
原文摘要 · Abstract (English)
State-of-the-art model-based reinforcement learning methods learn neural world models that allow policy improvement by planning in a latent space, without assumptions on the structure of the underlying environment. While expressive, these models are generally task-dependent: they learn uninterpretable latent representations that are tied to the training task and thus hard to generalize to new tasks. In this work, we present a novel world model formulation where the reward prediction only depends on a subset of structured, symbolic components of the whole latent state. Decoupling observation reconstruction and reward prediction allows us to learn world models that can adapt zero-shot, i.e. without further environment interactions, to new reward functions defined over the same symbolic state space. We discuss the main advantages and challenges of learning these neurosymbolic world models and demonstrate the strong generalisation properties of our approach over purely neural methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。