用强化学习收缩行为分布，让机器人从预训练模型快速变高手。
From Prior to Pro: Efficient Skill Mastery via Distribution Contractive RL Finetuning
- 通过分布收缩机制，用在线反馈放大高成功率动作。
- 在仿真和真实机器人上均实现复杂长程操作技能的高效掌握。
- 适合需要快速适配新任务的机器人系统开发者。
我们提出分布收缩强化学习（DICE-RL），将强化学习作为“分布收缩”算子，用于优化预训练的生成式机器人策略。DICE-RL 通过在线反馈放大高成功率行为，将预训练的行为先验转化为高性能的“专业级”策略。先使用基于扩散或流模型的策略进行广泛行为覆盖预训练，再采用稳定、样本高效的残差离线策略强化学习框架进行微调，结合选择性行为正则化与价值引导的动作选择。大量实验与分析表明，DICE-RL 在性能提升上具有强稳定性与高样本效率。该方法可直接从高维像素输入中实现复杂长时序操作技能的掌握，适用于仿真环境与真实机器人。项目网站：https://zhanyisun.github.io/dice.rl.2026/
原文摘要 · Abstract (English)
We introduce Distribution Contractive Reinforcement Learning (DICE-RL), a framework that uses reinforcement learning (RL) as a "distribution contraction" operator to refine pretrained generative robot policies. DICE-RL turns a pretrained behavior prior into a high-performing "pro" policy by amplifying high-success behaviors from online feedback. We pretrain a diffusion- or flow-based policy for broad behavioral coverage, then finetune it with a stable, sample-efficient residual off-policy RL framework that combines selective behavior regularization with value-guided action selection. Extensive experiments and analyses show that DICE-RL reliably improves performance with strong stability and sample efficiency. It enables mastery of complex long-horizon manipulation skills directly from high-dimensional pixel inputs, both in simulation and on a real robot. Project website: https://zhanyisun.github.io/dice.rl.2026/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。