用注意力机制建模复杂组合动作，解决动作空间爆炸问题。
SAINT: Attention-Based Policies for Discrete Combinatorial Action Spaces
- 将多组件动作视为无序集合，用自注意力捕捉依赖关系
- 在1.35×10¹⁸种可能动作的环境中表现优于基线
- 适合动作间存在复杂协同的强化学习任务
许多现实世界中的动作空间具有组合结构，导致可能动作数量呈指数级增长，限制了传统强化学习算法的效果。现有方法对子动作施加因子化或顺序结构，难以捕捉复杂的联合行为。我们提出基于Transformer的子动作交互网络（SAINT），将多组件动作表示为无序集合，并通过全局状态条件下的自注意力建模其依赖关系。SAINT具有置换不变性、采样高效性，且兼容标准策略优化算法。在三个任务领域共18个不同的组合环境测试中，包括拥有1.35×10¹⁸种可能动作的环境，SAINT始终优于强基线。
原文摘要 · Abstract (English)
The combinatorial structure of many real-world action spaces leads to exponential growth in the number of possible actions, limiting the effectiveness of conventional reinforcement learning algorithms. Recent approaches for combinatorial action spaces impose factorized or sequential structures over sub-actions, failing to capture complex joint behavior. We introduce the Sub-Action Interaction Network using Transformers (SAINT), a novel policy architecture that represents multi-component actions as unordered sets and models their dependencies via self-attention conditioned on the global state. SAINT is permutation-invariant, sample-efficient, and compatible with standard policy optimization algorithms. In 18 distinct combinatorial environments across three task domains, including environments with $1.35 \times 10^{18}$ possible actions, SAINT consistently outperforms strong baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。