用分层动态提示提升决策变压器的少样本泛化能力。
Hierarchical Prompt Decision Transformer: Improving Few-Shot Policy Generalization with Global and Adaptive Guidance
- 分两层软令牌：全局任务提示+动态时间步提示
- 在MuJoCo和MetaWorld上7个任务均超越基线方法
- 适合需要快速适应新任务的强化学习场景
决策变压器将强化学习重新定义为条件序列生成问题，提供了一种简单有效的替代传统价值或策略方法的新路径。近期关键进展是将提示机制引入决策变压器以促进少样本策略泛化。然而，现有方法主要使用静态提示片段引导轨迹，限制了其提供上下文相关指导的能力。为此，我们提出一种由检索增强支持的分层提示方法。该方法学习两层软令牌作为引导提示：(1) 全局令牌封装轨迹的任务级信息；(2) 自适应令牌动态从精选演示段中检索，提供聚焦的时间步特定指令。在MuJoCo和MetaWorld环境中的七个基准任务上的实验表明，所提方法持续优于所有基线方法，表明分层提示是实现决策变压器少样本策略泛化的有效策略。
原文摘要 · Abstract (English)
Decision transformers recast reinforcement learning as a conditional sequence generation problem, offering a simple but effective alternative to traditional value or policy-based methods. A recent key development in this area is the integration of prompting in decision transformers to facilitate few-shot policy generalization. However, current methods mainly use static prompt segments to guide rollouts, limiting their ability to provide context-specific guidance. Addressing this, we introduce a hierarchical prompting approach enabled by retrieval augmentation. Our method learns two layers of soft tokens as guiding prompts: (1) global tokens encapsulating task-level information about trajectories, and (2) adaptive tokens that deliver focused, timestep-specific instructions. The adaptive tokens are dynamically retrieved from a curated set of demonstration segments, ensuring context-aware guidance. Experiments across seven benchmark tasks in the MuJoCo and MetaWorld environments demonstrate the proposed approach consistently outperforms all baseline methods, suggesting that hierarchical prompting for decision transformers is an effective strategy to enable few-shot policy generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。