用随机扰动提升视觉强化学习效率，单卡几小时完成训练
Efficient On-policy Visual-RL via Stochastic Decoupled Policy Gradient
- 通过轨迹扰动估算梯度，大幅减少环境渲染次数
- 在视觉MuJoCo上训练更快、内存更低、奖励更高
- 适合追求高效视觉控制的机器人研究者
我们提出一种轻量级视觉强化学习方法——随机解耦策略梯度（SDPG），可在单张NVIDIA RTX 4080 GPU上数小时内端到端训练出多样化的视觉运动控制策略。SDPG通过随机扰动轨迹采样来估计策略梯度，相比传统方法显著减少批渲染环境数量，大幅降低计算与内存开销。在视觉MuJoCo基准测试中，SDPG在训练时间、内存使用和最终奖励方面均持续优于基线方法。此外，为支持未来研究，我们构建了一套涵盖灵巧操作与复杂运动的现实视觉机器人基准，并在真实硬件上验证了有效的模拟到现实迁移能力。
原文摘要 · Abstract (English)
We present the stochastic decoupled policy gradient (SDPG), a lightweight visual reinforcement learning (RL) method that trains diverse visuomotor control policies end-to-end within a few hours on a single NVIDIA RTX 4080 GPU. SDPG estimates policy gradients via random perturbations of trajectory rollouts, requiring orders of magnitude fewer batch-rendered environments and substantially reducing compute and memory overhead. On visual MuJoCo benchmarks, SDPG consistently outperforms baseline methods in training time, memory usage, and rewards. Finally, to support future research, we introduce a suite of realistic visual robotics benchmarks spanning dexterous manipulation, challenging locomotion, and demonstrate effective sim-to-real transfer on physical hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。