让智能体自动学习状态与动作抽象，提升参数化动作的强化学习效率。
Context-Sensitive Abstractions for Reinforcement Learning with Parameterized Actions
- 在线自适应构建状态与动作抽象，逐步细化关键区域
- 在多个连续状态参数化动作任务中，样本效率显著优于基线
- 无需领域工程，可自动挖掘动作空间潜在结构
现实世界中的序列决策常涉及需同时决定离散动作和连续参数的参数化动作空间。现有方法存在严重局限：规划方法依赖手工设计的动作模型，标准强化学习算法仅适用于离散或连续动作之一，少数支持参数化动作的方法通常依赖领域特定工程，无法利用动作空间的潜在结构。本文通过使智能体在学习过程中自主在线构建状态与动作抽象，将强化学习算法扩展至长时程、稀疏奖励的参数化动作场景。提出算法可渐进式优化抽象，仅在对性能影响大的状态-动作区域增加细节分辨率。在多个连续状态、参数化动作环境中，该抽象驱动方法使TD(λ)实现显著更高的样本效率，优于当前最先进基线。
原文摘要 · Abstract (English)
Real-world sequential decision-making often involves parameterized action spaces that require both, decisions regarding discrete actions and decisions about continuous action parameters governing how an action is executed. Existing approaches exhibit severe limitations in this setting -- planning methods demand hand-crafted action models, and standard reinforcement learning (RL) algorithms are designed for either discrete or continuous actions but not both, and the few RL methods that handle parameterized actions typically rely on domain-specific engineering and fail to exploit the latent structure of these spaces. This paper extends the scope of RL algorithms to long-horizon, sparse-reward settings with parameterized actions by enabling agents to autonomously learn both state and action abstractions online. We introduce algorithms that progressively refine these abstractions during learning, increasing fine-grained detail in the critical regions of the state-action space where greater resolution improves performance. Across several continuous-state, parameterized-action domains, our abstraction-driven approach enables TD($λ$) to achieve markedly higher sample efficiency than state-of-the-art baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。