用Mamba模型实现视觉驱动的高效机器人行走,训练快、泛化强。
LocoMamba: Vision-Driven Locomotion via End-to-End Deep Reinforcement Learning with Mamba
- 基于Mamba的跨模态强化学习框架,近线性时间处理长序列。
- 在复杂地形上成功率更高,碰撞更少,训练效率提升显著。
- 适合做机器人视觉导航与端到端控制的研究者参考。
我们提出LocoMamba,一种基于选择性状态空间模型(Mamba)的视觉驱动端到端深度强化学习框架,实现近线性时间序列建模,有效捕捉长程依赖,并支持更长序列的高效训练。首先,通过多层感知机嵌入本体感觉状态,用轻量卷积网络对深度图像进行分块处理,生成紧凑的特征令牌以提升状态表示。其次,堆叠的Mamba层通过近线性时间的选择性扫描融合这些令牌,降低延迟和内存开销,对序列长度和图像分辨率保持鲁棒,并提供归纳偏置以缓解过拟合。第三,采用近端策略优化(PPO)在地形与外观随机化及障碍物密度课程下进行端到端策略训练,使用紧凑的状态中心奖励函数平衡前进速度、平滑性与安全性。我们在包含静态与动态障碍物及不平坦地形的复杂仿真环境中评估该方法。相比现有最优基线,本方法在相同计算预算下收敛更快,获得更高的回报与成功率,碰撞更少,且对未见地形和障碍物密度表现出更强泛化能力。
原文摘要 · Abstract (English)
We introduce LocoMamba, a vision-driven cross-modal DRL framework built on selective state-space models, specifically leveraging Mamba, that achieves near-linear-time sequence modeling, effectively captures long-range dependencies, and enables efficient training with longer sequences. First, we embed proprioceptive states with a multilayer perceptron and patchify depth images with a lightweight convolutional neural network, producing compact tokens that improve state representation. Second, stacked Mamba layers fuse these tokens via near-linear-time selective scanning, reducing latency and memory footprint, remaining robust to token length and image resolution, and providing an inductive bias that mitigates overfitting. Third, we train the policy end-to-end with Proximal Policy Optimization under terrain and appearance randomization and an obstacle-density curriculum, using a compact state-centric reward that balances progress, smoothness, and safety. We evaluate our method in challenging simulated environments with static and moving obstacles as well as uneven terrain. Compared with state-of-the-art baselines, our method achieves higher returns and success rates with fewer collisions, exhibits stronger generalization to unseen terrains and obstacle densities, and improves training efficiency by converging in fewer updates under the same compute budget.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。