提出一种无需先验、计算高效的动作选择方法，提升强化学习在复杂环境中的表现。
Where to Intervene: Action Selection in Deep Reinforcement Learning
- 基于数据驱动的无模型动作选择，通过淘汰冗余动作减少搜索空间。
- 采用敲击采样控制假阳性率，在保证精度的同时提升动作选择可靠性。
- 可直接嵌入在线训练流程，适用于多种强化学习任务，尤其适合无先验知识场景。
近年来，深度强化学习在复杂环境中广泛应用，但面临高维动作空间选择的关键挑战。现有方法通常依赖复杂的先验设计或领域专家经验来消除动作空间冗余，计算开销大，泛化能力受限。本文提出一种通用的数据驱动动作选择方法，具备无模型和计算友好特性。该方法不仅能选出最小充分动作集，还通过敲击采样（knockoff sampling）控制假发现率。更重要的是，我们将其无缝集成到深度强化学习的在线训练过程中。实验验证了理论保证的有效性，结果表明，该方法在动作选择准确率和最终累积奖励方面均优于多种替代技术。
原文摘要 · Abstract (English)
Deep reinforcement learning (RL) has gained widespread adoption in recent years but faces significant challenges, particularly in unknown and complex environments. Among these, high-dimensional action selection stands out as a critical problem. Existing works often require a sophisticated prior design to eliminate redundancy in the action space, relying heavily on domain expert experience or involving high computational complexity, which limits their generalizability across different RL tasks. In this paper, we address these challenges by proposing a general data-driven action selection approach with model-free and computationally friendly properties. Our method not only selects minimal sufficient actions but also controls the false discovery rate via knockoff sampling. More importantly, we seamlessly integrate the action selection into deep RL methods during online training. Empirical experiments validate the established theoretical guarantees, demonstrating that our method surpasses various alternative techniques in terms of both performance in variable selection and overall achieved rewards.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。