通过课程式掩码策略，让模型分阶段学习复杂度不同的技能。
Learning Versatile Skills with Curriculum Masking
- 根据学习难度动态调整掩码方式，模仿人类循序渐进的学习过程。
- 在零样本技能提示和目标规划任务上表现更优，微调后在离线强化学习中也具竞争力。
- 适合需要多技能泛化能力的智能体训练，如机器人控制与通用决策系统。
掩码预测因其灵活的掩码机制，已成为离线强化学习中一种有前景的预训练范式，支持用统一模型完成多种下游任务。然而，如何平衡不同复杂度技能的学习仍不明确。为此，我们提出 CurrMask，一种面向序列决策的课程掩码预训练方法。受人类按课程组织知识的启发，CurrMask 在预训练过程中动态调整掩码方案，以逐步学习多样技能。大量实验表明，CurrMask 在技能提示、目标条件规划等零样本任务上表现优异，微调后在离线强化学习任务中也具备竞争力。对训练动态的分析显示，该方法通过动态调整掩码，逐步掌握不同复杂度的技能。
原文摘要 · Abstract (English)
Masked prediction has emerged as a promising pretraining paradigm in offline reinforcement learning (RL) due to its versatile masking schemes, enabling flexible inference across various downstream tasks with a unified model. Despite the versatility of masked prediction, it remains unclear how to balance the learning of skills at different levels of complexity. To address this, we propose CurrMask, a curriculum masking pretraining paradigm for sequential decision making. Motivated by how humans learn by organizing knowledge in a curriculum, CurrMask adjusts its masking scheme during pretraining for learning versatile skills. Through extensive experiments, we show that CurrMask exhibits superior zero-shot performance on skill prompting tasks, goal-conditioned planning tasks, and competitive finetuning performance on offline RL tasks. Additionally, our analysis of training dynamics reveals that CurrMask gradually acquires skills of varying complexity by dynamically adjusting its masking scheme.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。