智能体通过强化学习动态决定何时切换导航策略，提升复杂迷宫求解效率。
Learning When to Switch: Adaptive Policy Selection via Reinforcement Learning
- 用Q-learning根据覆盖度和距离目标的远近自动调整策略切换阈值
- 相比固定阈值，完成时间缩短23%-55%，最差情况性能提升71%
- 无需预先知道墙的位置或最优阈值，适合复杂环境下的自主决策
自主智能体在解决复杂任务时常需多种策略，但何时切换策略仍具挑战。本文提出一种基于强化学习的方法，用于学习两个正交导航策略间的切换阈值。以迷宫导航为例，展示智能体如何在系统性探索（覆盖率）与目标导向寻路（收敛性）间动态切换，从而提升任务表现。与固定阈值方法不同，该智能体利用Q-learning，依据覆盖率和距目标距离自适应调整行为，仅需基本领域知识：迷宫尺寸与目标位置。无需事先知晓墙壁布局、最优阈值或手工设计启发式规则；而是在每轮运行中动态发现有效切换策略。智能体将状态空间离散化为覆盖率与距离桶，根据观测进展信号自适应选择20%-60%的覆盖阈值。在240个测试配置（4种迷宫尺寸16×16至128×128，10个独特迷宫，6种智能体变体）上的实验表明，自适应阈值学习优于单一策略智能体和固定40%阈值基线。结果显示，完成时间减少23%-55%，运行时间方差降低83%，最差情形性能提升71%。所学切换行为在同尺寸类别内可泛化至未见过的墙体结构。性能增益随问题复杂度增加：16×16迷宫提升23%，32×32提升34%，64×64提升55%，表明随着迷宫结构可能性增大，自适应策略相较固定启发式的优势呈比例增长。
原文摘要 · Abstract (English)
Autonomous agents often require multiple strategies to solve complex tasks, but determining when to switch between strategies remains challenging. This research introduces a reinforcement learning technique to learn switching thresholds between two orthogonal navigation policies. Using maze navigation as a case study, this work demonstrates how an agent can dynamically transition between systematic exploration (coverage) and goal-directed pathfinding (convergence) to improve task performance. Unlike fixed-threshold approaches, the agent uses Q-learning to adapt switching behavior based on coverage percentage and distance to goal, requiring only minimal domain knowledge: maze dimensions and target location. The agent does not require prior knowledge of wall positions, optimal threshold values, or hand-crafted heuristics; instead, it discovers effective switching strategies dynamically during each run. The agent discretizes its state space into coverage and distance buckets, then adapts which coverage threshold (20-60\%) to apply based on observed progress signals. Experiments across 240 test configurations (4 maze sizes from 16$\times$16 to 128$\times$128 $\times$ 10 unique mazes $\times$ 6 agent variants) demonstrate that adaptive threshold learning outperforms both single-strategy agents and fixed 40\% threshold baselines. Results show 23-55\% improvements in completion time, 83\% reduction in runtime variance, and 71\% improvement in worst-case scenarios. The learned switching behavior generalizes within each size class to unseen wall configurations. Performance gains scale with problem complexity: 23\% improvement for 16$\times$16 mazes, 34\% for 32$\times$32, and 55\% for 64$\times$64, demonstrating that as the space of possible maze structures grows, the value of adaptive policy selection over fixed heuristics increases proportionally.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。