通过控制能力最大化，学习只关注关键控制特征的环境表示。
Learning to Perceive the World Through Control: Empowerment-Based Representation Learning
- 用控制能力目标驱动表示学习，自动分离出与控制相关特征。
- 发现正向与反向两种表示，均对无关特征具有不变性。
- 适合研究具身智能与因果表征的学习者。
在许多实际强化学习环境中，观测维度远高于控制所需变量。本文探讨：能否学习仅捕捉环境控制相关特征的表示？通过研究控制能力（empowerment）目标——该目标广泛用于无监督技能学习——我们发现，基于该目标的代理会产生两种不同表示：前向与反向表示，二者分别捕获状态的不同互补方面，且均对控制无关特征保持不变。因此，最大化控制能力促使代理学习到一种隐式的、以控制为中心的世界模型。分析强调了通过交互而非静态数据集学习表示的重要性：旨在最大化控制的交互是获得有用不变性属性的关键，这一观点与因果学习文献高度一致。
原文摘要 · Abstract (English)
In many practical reinforcement learning environments, observations are far higher-dimensional than the variables that matter for control. In this work, we ask: can we learn representations that capture only control-relevant features of the environment? We study this question through the empowerment objective, which maximizes an agent's influence over the environment and is widely used for unsupervised skill learning. We show that empowerment agents induce two distinct representations -- forward and backward -- that capture complementary aspects of the state, and both of which are invariant to control-irrelevant features. Thus, empowerment maximization leads agents to learn an implicit, control-centric model of the world. Our analysis highlights the importance of learning representations through interaction rather than from passive datasets: interaction aimed at maximizing control is essential for learning useful invariance properties, a perspective that aligns closely with the causal learning literature.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。