将回报、状态和动作融合为单一令牌，提升决策模型效率与泛化能力。
Unified token representations for sequential decision models
- 用统一令牌合并三类信息，缩短序列长度
- 在多个数据集上达到或超越现有最佳性能
- 适合资源受限场景的高效决策系统设计
Transformer 在离线强化学习中通过将轨迹建模为回报、状态和动作的序列展现出强大潜力。然而，现有方法如 Decision Transformer（DT）及其变体存在冗余分词和二次注意力复杂度问题，限制了其在实时或资源受限环境中的可扩展性。为此，我们提出统一令牌表示（UTR），将回报、状态和动作合并为单一令牌，显著降低序列长度与模型复杂度。理论分析表明，UTR 可带来更紧的 Rademacher 复杂度界，暗示更强泛化能力。我们进一步构建两种变体：基于 Transformer 的 UDT 与基于门控 CNN 的 UDC。两者均在保持显著更低计算量的同时，实现与当前最优方法相当或更优的性能。结果表明，UTR 能跨架构良好泛化，可能为未来大规模决策模型提供高效的可扩展基础。
原文摘要 · Abstract (English)
Transformers have demonstrated strong potential in offline reinforcement learning (RL) by modeling trajectories as sequences of return-to-go, states, and actions. However, existing approaches such as the Decision Transformer(DT) and its variants suffer from redundant tokenization and quadratic attention complexity, limiting their scalability in real-time or resource-constrained settings. To address this, we propose a Unified Token Representation (UTR) that merges return-to-go, state, and action into a single token, substantially reducing sequence length and model complexity. Theoretical analysis shows that UTR leads to a tighter Rademacher complexity bound, suggesting improved generalization. We further develop two variants: UDT and UDC, built upon transformer and gated CNN backbones, respectively. Both achieve comparable or superior performance to state-of-the-art methods with markedly lower computation. These findings demonstrate that UTR generalizes well across architectures and may provide an efficient foundation for scalable control in future large decision models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。