用稀疏编码让强化学习高效控制自然图像序列，无需深度学习。
Optimal Control with Natural Images: Efficient Reinforcement Learning using Overcomplete Sparse Codes
- 将自然图像转为过完备稀疏码，提升控制效率。
- 任务规模比完整编码大数个数量级，仍可高效求解。
- 证明深度学习非必要，适合视觉控制研究者。
最优控制与序列决策广泛应用于复杂任务中。对自然图像序列的最优控制是理解视觉在控制中作用的第一步。本文将其形式化为强化学习问题，并推导出图像包含足够信息以实现最优策略的一般条件。当自然图像被编码为‘高效’表示时，强化学习可提供计算高效的最优策略求解方法。为此，引入了一个新的强化学习基准，能轻松扩展至大量状态和长时序。特别地，通过将每幅图像表示为过完备稀疏码，我们成功解决了比使用完整编码大数个数量级的最优控制任务。理论分析支持该行为。本工作还表明，深度学习并非实现高效自然图像控制的必要条件。
原文摘要 · Abstract (English)
Optimal control and sequential decision making are widely used in many complex tasks. Optimal control over a sequence of natural images is a first step towards understanding the role of vision in control. Here, we formalize this problem as a reinforcement learning task, and derive general conditions under which an image includes enough information to implement an optimal policy. Reinforcement learning is shown to provide a computationally efficient method for finding optimal policies when natural images are encoded into "efficient" image representations. This is demonstrated by introducing a new reinforcement learning benchmark that easily scales to large numbers of states and long horizons. In particular, by representing each image as an overcomplete sparse code, we are able to efficiently solve an optimal control task that is orders of magnitude larger than those tasks solvable using complete codes. Theoretical justification for this behaviour is provided. This work also demonstrates that deep learning is not necessary for efficient optimal control with natural images.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。