用强化学习自动学会何时重置搜索,提升效率。
Learning to reset in target search problems
- 用强化学习训练智能体自主决定重置时机。
- 在布朗运动搜索中逼近最优重置策略,效率显著提升。
- 可适配复杂环境,适合研究搜索算法与机器人导航的人参考。
目标搜索问题广泛存在于生物觅食到优化算法等多个领域。近期研究表明,适时重置能显著提升搜索效率,但最优重置策略依赖于具体问题特性,难以人工确定。本文提出一种基于强化学习(RL)的框架,训练智能体在环境中自主学习重置策略以优化搜索效率。首先在经典的布朗运动重置搜索基准上验证:RL智能体始终能恢复出接近最优的尖锐重置分布。随后扩展框架,使智能体不仅能决定重置时机,还能通过转向动作控制空间行为。在此更复杂设置中,智能体发现同时适应重置与转向的策略,优于现有基准。结果表明,强化学习不仅是优化工具,还可揭示可解释的新搜索策略,适用于随机重置搜索过程的研究。
原文摘要 · Abstract (English)
Target search problems are central to a wide range of fields, from biological foraging to the optimization algorithms. Recently, the ability to reset the search has been shown to significantly improve the searcher's efficiency. However, the optimal resetting strategy depends on the specific properties of the search problem and can often be challenging to determine. In this work, we propose a reinforcement learning (RL)-based framework to train agents capable of optimizing their search efficiency in environments by learning how to reset. First, we validate the approach in a well-established benchmark: the Brownian search with resetting. There, RL agents consistently recover strategies closely resembling the sharp resetting distribution, known to be optimal in this scenario. We then extend the framework by allowing agents to control not only when to reset, but also their spatial dynamics through turning actions. In this more complex setting, the agents discover strategies that adapt both resetting and turning to the properties of the environment, outperforming the proposed benchmarks. These results demonstrate how reinforcement learning can serve both as an optimization tool and a mechanism for uncovering new, interpretable strategies in stochastic search processes with resetting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。