用短序列因果图动态评分，让强化学习更专注关键信息
AdaCred: Adaptive Causal Decision Transformers with Feature Crediting
- 将轨迹建模为短时动作-奖励-状态因果图，替代长序列记忆
- 仅保留任务相关性强的表示，显著减少所需轨迹长度
- 适合对推理效率要求高、数据稀疏的离线强化学习场景
强化学习可视为序列建模问题，模型基于历史状态-动作-奖励序列预测未来行为。现有方法在离线强化学习中通常依赖长轨迹序列，但易过度依赖长期表征，难以根据任务需求准确评估轨迹与表征的重要性。本文提出AdaCred，将轨迹表示为由短时动作-奖励-状态序列构建的因果图，通过自适应赋权与剪枝机制，仅保留对下游任务最相关的表示。实验表明，基于AdaCred的策略使用更短的轨迹序列，且在离线强化学习与模仿学习环境中均持续优于传统方法。
原文摘要 · Abstract (English)
Reinforcement learning (RL) can be formulated as a sequence modeling problem, where models predict future actions based on historical state-action-reward sequences. Current approaches typically require long trajectory sequences to model the environment in offline RL settings. However, these models tend to over-rely on memorizing long-term representations, which impairs their ability to effectively attribute importance to trajectories and learned representations based on task-specific relevance. In this work, we introduce AdaCred, a novel approach that represents trajectories as causal graphs built from short-term action-reward-state sequences. Our model adaptively learns control policy by crediting and pruning low-importance representations, retaining only those most relevant for the downstream task. Our experiments demonstrate that AdaCred-based policies require shorter trajectory sequences and consistently outperform conventional methods in both offline reinforcement learning and imitation learning environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。