让AI根据环境自动调整动作执行时长，提升决策灵活性与效率
Adaptive Action Duration with Contextual Bandits for Deep Reinforcement Learning in Dynamic Environments
- 用上下文相关博弈模型动态选择动作重复频率
- 在Atari游戏上性能优于固定时长的基线方法
- 适合实时性要求高的游戏与机器人应用
深度强化学习（DRL）在复杂序列决策任务中表现卓越，如玩Atari 2600游戏和掌握棋类游戏。然而，动作执行的时间尺度这一关键却未被充分探索。本文提出一种新范式，将上下文相关博弈与DRL结合，实现动作持续时间的自适应选择，增强策略灵活性与计算效率。该方法在Deep Q-Network（DQN）基础上引入上下文相关博弈模块，根据状态上下文学习最优动作重复率。在Atari 2600游戏上的实验表明，其性能显著优于静态持续时间基线，验证了自适应时间抽象在DRL中的有效性。该范式为游戏与机器人等对动态动作时长敏感的实时应用提供了可扩展解决方案。
原文摘要 · Abstract (English)
Deep Reinforcement Learning (DRL) has achieved remarkable success in complex sequential decision-making tasks, such as playing Atari 2600 games and mastering board games. A critical yet underexplored aspect of DRL is the temporal scale of action execution. We propose a novel paradigm that integrates contextual bandits with DRL to adaptively select action durations, enhancing policy flexibility and computational efficiency. Our approach augments a Deep Q-Network (DQN) with a contextual bandit module that learns to choose optimal action repetition rates based on state contexts. Experiments on Atari 2600 games demonstrate significant performance improvements over static duration baselines, highlighting the efficacy of adaptive temporal abstractions in DRL. This paradigm offers a scalable solution for real-time applications like gaming and robotics, where dynamic action durations are critical.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。