arXiv:2409.03167cs.AIcs.LG2024-09被引 4

构建可扩展的基础设施管理强化学习框架,支持复杂决策模拟。

InfraLib: Enabling Reinforcement Learning and Decision-Making for Large-Scale Infrastructure Management

  • 提出分层随机退化模型与真实约束建模方法
  • 支持预算周期性、部件不可用等现实约束条件
  • 适用于道路网络等大规模系统,兼具效率与可扩展性

高效管理基础设施对经济稳定、可持续性和公共安全至关重要。然而,由于系统规模庞大、部件随机退化、观测不完全以及资源受限,基础设施维护面临挑战。仅依赖人工判断的决策策略在大尺度和长周期下常导致次优结果。尽管数据驱动方法如强化学习具有潜力,但其应用受限于缺乏合适的仿真环境。本文提出 InfraLib,一个开源的模块化、可扩展框架,将带资源约束的基础设施管理问题建模为序列决策问题。该框架实现分层随机退化模型,支持真实场景下的部分可观测性,并处理周期性预算、部件不可用等实际约束。InfraLib 提供标准化环境用于评估决策方法,配套专家数据收集与策略评估工具。通过合成基准和真实道路网络案例研究,验证了其在多样化场景中的建模能力与大规模计算效率。

原文摘要 · Abstract (English)

Efficient management of infrastructure systems is crucial for economic stability, sustainability, and public safety. However, infrastructure sustainment is challenging due to the vast scale of systems, stochastic deterioration of components, partial observability, and resource constraints. Decision-making strategies that rely solely on human judgment often result in suboptimal decisions over large scales and long horizons. While data-driven approaches like reinforcement learning offer promising solutions, their application has been limited by the lack of suitable simulation environments. We present InfraLib, an open-source modular and extensible framework that enables modeling and analyzing infrastructure management problems with resource constraints as sequential decision-making problems. The framework implements hierarchical, stochastic deterioration models, supports realistic partial observability, and handles practical constraints including cyclical budgets and component unavailability. InfraLib provides standardized environments for benchmarking decision-making approaches, along with tools for expert data collection and policy evaluation. Through case studies on both synthetic benchmarks and real-world road networks, we demonstrate InfraLib's ability to model diverse infrastructure management scenarios while maintaining computational efficiency at scale.

强化学习基础设施决策优化仿真框架

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。