用点云特征划分任务,让通用-专用学习更高效。
GSL-PCD: Improving Generalist-Specialist Learning with Point Cloud Feature-based Task Partitioning
- 基于物体点云特征聚类,智能分配环境变化到不同专家。
- 相同专家数量下性能提升9.4%,计算与采样成本降低50%。
- 适合机器人抓取等需要泛化的强化学习场景。
深度强化学习在未见环境变化下的泛化能力通常依赖于多样场景的训练。现有算法在处理大量变化时效率低下。通用-专用学习(GSL)框架通过先在所有变化上训练通用模型,再从中生成专注特定子集的专家模型,并由通用模型借助专家进行优化。然而,随机的任务划分会导致相似变化被分配至同一专家,常致每个专家仅针对单一变化,增加计算开销。为此,我们提出基于点云特征的任务划分方法(GSL-PCD),利用从物体点云中提取的特征进行聚类,并采用贪心算法实现均衡划分,使相似环境变化集中于同一专家。在ManiSkill基准的机器人抓取任务上验证表明,该方法在固定专家数下性能比原始划分提升9.4%,同时将计算与样本需求降低50%即可达到相近表现。
原文摘要 · Abstract (English)
Generalization in Deep Reinforcement Learning (DRL) across unseen environment variations often requires training over a diverse set of scenarios. Many existing DRL algorithms struggle with efficiency when handling numerous variations. The Generalist-Specialist Learning (GSL) framework addresses this by first training a generalist model on all variations, then creating specialists from the generalist's weights, each focusing on a subset of variations. The generalist then refines its learning with assistance from the specialists. However, random task partitioning in GSL can impede performance by assigning vastly different variations to the same specialist, often resulting in each specialist focusing on only one variation, which raises computational costs. To improve this, we propose Generalist-Specialist Learning with Point Cloud Feature-based Task Partitioning (GSL-PCD). Our approach clusters environment variations based on features extracted from object point clouds and uses balanced clustering with a greedy algorithm to assign similar variations to the same specialist. Evaluations on robotic manipulation tasks from the ManiSkill benchmark demonstrate that point cloud feature-based partitioning outperforms vanilla partitioning by 9.4%, with a fixed number of specialists, and reduces computational and sample requirements by 50% to achieve comparable performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。