V-Simba通过架构改进，显著提升视觉强化学习的样本效率。
V-Simba: Unleashing the Architectural Potential of RL in Visual Continuous Control
- 基于SAC架构，引入归一化与点卷积减少计算开销
- 在DMC、Adroit、Meta-World上达到或超越当前最优性能
- 适合关注高效视觉控制的机器人与RL研究者
提升样本效率仍是强化学习的核心挑战，尤其在机器人等真实场景中，数据采集成本高。该问题在视觉强化学习中尤为突出，因高维输入常掩盖学习信号。以往工作多聚焦算法优化，如更优的动力学模型或探索策略，但近期基于状态的强化学习表明，仅通过架构设计即可显著提升样本效率。这引发关键问题：这些架构原理能否迁移至视觉强化学习？为此，我们提出V-Simba，一种受状态空间强化学习中的Simba架构启发的简单有效视觉强化学习架构。在软演员-评论家（SAC）基础上结合数据增强，通过添加归一化层稳定训练，并使用点卷积降低计算量。尽管结构简单，V-Simba在DMC、Adroit和Meta-World基准上表现达到或优于现有最优方法，且比DrQ-v2更高效。代码已开源于https://github.com/DAVIAN-Robotics/V-Simba。
原文摘要 · Abstract (English)
Improving sample efficiency remains a core challenge in reinforcement learning (RL), especially in real-world settings like robotics, where data collection is costly. This challenge is pronounced in visual RL, where high-dimensional inputs often obscure learning signals. While prior work in visual RL has focused on algorithmic solutions, such as better dynamics models or exploration strategies, recent advances in state-based RL show that architectural design alone can lead to significant gains in sample efficiency. This raises an important question: Can these architectural principles transfer to visual RL? In response, we introduce V-Simba, a simple yet effective visual RL architecture inspired by the Simba architecture from state-based RL. Built on top of Soft Actor-Critic (SAC) with data augmentation, V-Simba modifies the architecture by adding normalization layers to stabilize training and using pointwise convolutions to reduce computation. Despite its simplicity, V-Simba matches or outperforms the state-of-the-art methods across the DMC, Adroit, and Meta-World benchmarks, while being more computationally efficient than DrQ-v2. We make our code publicly available at https://github.com/DAVIAN-Robotics/V-Simba.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。