arXiv:2505.23865cs.LGcs.AI2025-05

用深度模型估算信息增益,让多个智能体高效探索农田。

Combining Deep Architectures for Information Gain estimation and Reinforcement Learning for multiagent field exploration

  • 用LSTM构建信念模型,结合熵与视角遮蔽图,实时计算信息增益。
  • 双卷积DQN在20x20网格上比单模型快27%完成探索,且减少重复访问。
  • 适合需要低资源、高鲁棒性的农业机器人自主巡检场景。

精准农业需要高效的自主系统监测作物,智能体需在大尺度环境中探索,同时最小化资源消耗。本文将问题建模为一个网格环境中的主动探索任务,每个格子可能包含目标(如受损作物),可从9个预设视角观测。智能体需基于部分、顺序的观测推断每格的目标数量。提出两阶段深度学习框架:预训练的LSTM作为信念模型,更新环境的概率图及其关联熵,定义预期信息增益(IG),使智能体优先探索高信息区域。关键贡献是将视角可见性掩码加入输入,保持部分可观测下的马尔可夫性质,避免重复访问已探视角。比较三种智能体架构:无预训练的基于信息增益的智能体,使用局部3×3输入的CNN-DQN,以及具有更宽空间上下文的双卷积DQN。20×20地图模拟显示,未预训练的智能体表现良好;加入视角掩码后,DQN智能体性能相当;而双卷积智能体在更大环境中持续实现更高探索效率。结果表明,融合不确定性感知、信念状态与可见性追踪的策略具备鲁棒性和可扩展性。未来工作包括课程学习、多智能体协作共享奖励、基于变换器的模型及内在动机机制,以进一步提升学习效率与策略泛化能力。

原文摘要 · Abstract (English)

Precision agriculture requires efficient autonomous systems for crop monitoring, where agents must explore large-scale environments while minimizing resource consumption. This work addresses the problem as an active exploration task in a grid environment representing an agricultural field. Each cell may contain targets (e.g., damaged crops) observable from nine predefined points of view (POVs). Agents must infer the number of targets per cell using partial, sequential observations. We propose a two-stage deep learning framework. A pre-trained LSTM serves as a belief model, updating a probabilistic map of the environment and its associated entropy, which defines the expected information gain (IG). This allows agents to prioritize informative regions. A key contribution is the inclusion of a POV visibility mask in the input, preserving the Markov property under partial observability and avoiding revisits to already explored views. Three agent architectures were compared: an untrained IG-based agent selecting actions to maximize entropy reduction; a DQN agent using CNNs over local 3x3 inputs with belief, entropy, and POV mask; and a Double-CNN DQN agent with wider spatial context. Simulations on 20x20 maps showed that the untrained agent performs well despite its simplicity. The DQN agent matches this performance when the POV mask is included, while the Double-CNN agent consistently achieves superior exploration efficiency, especially in larger environments. Results show that uncertainty-aware policies leveraging entropy, belief states, and visibility tracking lead to robust and scalable exploration. Future work includes curriculum learning, multi-agent cooperation with shared rewards, transformer-based models, and intrinsic motivation mechanisms to further enhance learning efficiency and policy generalization.

多智能体信息增益农业机器人强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。