分离策略学习与演化函数训练，提升强化学习的泛化能力与稳定性。
Decoupled Behavioral Cloning for Scalable Inductive Generalization in RL from Specifications
- 先用标准RL学各任务专属策略，再用行为克隆拟合策略演化函数。
- 在多个任务上实现更稳定的训练和更强的零样本泛化性能。
- 适合需要高效泛化的新任务部署场景，尤其适用于复杂环境。
归纳泛化是强化学习中一种使相关任务实例对应相关策略的范式。先前方法通过直接使用强化学习学习高阶策略演化函数来捕捉该结构，但随着训练任务增加，累积奖励反馈变得嘈杂且冲突，导致训练不稳并削弱泛化能力。本文提出DIBS，一种解耦的行为克隆方法：先通过标准RL为每个任务独立学习教师策略，再利用教师标注的状态-动作对，通过行为克隆拟合演化函数。该方法将噪声奖励聚合替换为密集且稳定的监督信号。DIBS在训练稳定性和零样本泛化方面显著优于现有RL与元强化学习算法。
原文摘要 · Abstract (English)
Inductive generalization is a framework for reinforcement learning (RL) generalization in which inductively related task instances admit inductively related policies. Prior work captures this structure via a higher-order policy-evolution function learned directly with RL, but suffers from poor training scalability: as training tasks grow, aggregated reward feedback becomes noisy and conflicting, destabilizing training and weakening generalization. We propose DIBS, a decoupled behavioral cloning approach that separates learning task-specific policies from learning the evolution function. We first learn individual teacher policies per task via standard RL, then fit the evolution function via behavioral cloning on teacher-labeled state-action pairs. This replaces noisy reward aggregation with dense, stable supervision. DIBS achieves significant improvements in both training stability and zero-shot generalization against existing RL and meta-RL algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。