用轻量层让模型快速适应新任务，还能保持旧知识。
Adaptive Policy Backbone via Shared Network
- 在共享主干前后的轻量线性层实现高效微调。
- 相比标准强化学习，样本效率提升，且能处理分布外任务。
- 适合需要快速迁移、部署新环境的强化学习场景。
强化学习在多个领域取得了显著成果，但学习最优策略通常需要大量交互数据，限制了实际应用。常见方法是利用先验知识，如预收集的数据集或参考策略，但在训练与部署任务不匹配时性能下降。现有工作大多局限于分布内设置。为此，我们提出自适应策略主干（APB），一种元迁移强化学习方法，在共享主干前后插入轻量线性层，实现参数高效微调（PEFT），同时在适应过程中保留先验知识。实验表明，APB在样本效率上优于标准RL，并能在现有元强化学习基线失效的分布外（OOD）任务中成功适应。
原文摘要 · Abstract (English)
Reinforcement learning (RL) has achieved impressive results across domains, yet learning an optimal policy typically requires extensive interaction data, limiting practical deployment. A common remedy is to leverage priors, such as pre-collected datasets or reference policies, but their utility degrades under task mismatch between training and deployment. While prior work has sought to address this mismatch, it has largely been restricted to in-distribution settings. To address this challenge, we propose Adaptive Policy Backbone (APB), a meta-transfer RL method that inserts lightweight linear layers before and after a shared backbone, thereby enabling parameter-efficient fine-tuning (PEFT) while preserving prior knowledge during adaptation. Our results show that APB improves sample efficiency over standard RL and adapts to out-of-distribution (OOD) tasks where existing meta-RL baselines typically fail.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。