用粒子群动态调节探索，提升强化学习在复杂环境下的表现
ARISE: Adaptive Reinforcement Integrated with Swarm Exploration
- 引入轻量级粒子群探索层，融合策略动作与候选轨迹
- 在LunarLander-v3上提升46%，非平稳奖励下比PPO高75分
- 无需改动核心算法，适合追求稳定探索的RL研究者
有效探索仍是强化学习中的关键挑战，尤其在奖励非平稳或策略维度较高时。本文提出ARISE，一种轻量级框架，通过在标准策略梯度方法中加入紧凑的基于粒子群的探索层来增强强化学习。ARISE将策略动作与粒子驱动的建议相结合，每个粒子代表动作空间中采样的候选策略轨迹，并利用奖励方差线索自适应调节探索。在简单任务上仅略有提升（如CartPole-v1 +0.7%），但在更复杂任务上表现显著：LunarLander-v3提升46%，Hopper-v4提升22%，同时在Walker2d和Ant上保持稳定性。在奖励非平稳变化时，ARISE展现出明显鲁棒性优势，在CartPole上较PPO高出75分，对LunarLander也有相应改善。消融实验表明，粒子群组件与自适应机制均对性能有贡献。总体而言，ARISE提供了一种无需改变核心算法结构、即可实现更高效且稳健探索的通用路径。
原文摘要 · Abstract (English)
Effective exploration remains a key challenge in RL, especially with non-stationary rewards or high-dimensional policies. We introduce ARISE, a lightweight framework that enhances reinforcement learning by augmenting standard policy-gradient methods with a compact swarm-based exploration layer. ARISE blends policy actions with particle-driven proposals, where each particle represents a candidate policy trajectory sampled in the action space, and modulates exploration adaptively using reward-variance cues. While easy benchmarks exhibit only slight improvements (e.g., +0.7% on CartPole-v1), ARISE yields substantial gains on more challenging tasks, including +46% on LunarLander-v3 and +22% on Hopper-v4, while preserving stability on Walker2d and Ant. Under non-stationary reward shifts, ARISE provides marked robustness advantages, outperforming PPO by +75 points on CartPole and improving LunarLander accordingly. Ablation studies confirm that both the swarm component and the adaptive mechanism contribute to the performance. Overall, ARISE offers a simple, architecture-agnostic route to more exploratory and resilient RL agents without altering core algorithmic structures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。