arXiv:2510.03481cs.ROcs.SY2025-10被引 1

将区间MDP的鲁棒灵活合成转化为可求解的整数规划问题

Optimization-Based Robust Permissive Synthesis for Interval MDPs

  • 基于混合整数线性规划直接建模鲁棒贝尔曼约束
  • 在百万级状态空间上保持高效求解,支持灵活策略生成
  • 适合需兼顾安全与决策自由度的机器人控制系统设计

针对机器人在转移不确定性下的决策问题,本文提出一种基于优化的鲁棒灵活合成框架,适用于区间马尔可夫决策过程(IMDP)。由于模型误差和感知噪声,实际系统中转移概率常为区间值。传统鲁棒合成仅输出单一策略,而灵活合成又依赖精确模型,本文首次将鲁棒灵活合成问题建模为全局混合整数线性规划(MILP),直接编码鲁棒贝尔曼约束,最大化可启用的状态-动作对数量,同时保证所有合规策略在所有可接受的转移实现下均满足概率可达性或期望奖励规范。为克服顶点表示带来的指数复杂度,提出基于对偶化的编码方法,无需显式枚举顶点,复杂度随后继状态数线性增长。在四个典型机器人基准任务上的实验表明,该框架可扩展至包含数十万状态的IMDP,为机器人系统提供了兼具不确定性感知与灵活性保留的控制器设计基础。

原文摘要 · Abstract (English)

We present an optimization-based framework for robust permissive synthesis for Interval Markov Decision Processes (IMDPs), motivated by robotic decision-making under transition uncertainty. In many robotic systems, model inaccuracies and sensing noise lead to interval-valued transition probabilities. While robust IMDP synthesis typically yields a single policy and permissive synthesis assumes exact models, we show that robust permissive synthesis under interval uncertainty can be cast as a global mixed-integer linear program (MILP) that directly encodes robust Bellman constraints. The formulation maximizes a quantitative permissiveness metric (the number of enabled state-action pairs), while guaranteeing that every compliant strategy satisfies probabilistic reachability or expected reward specifications under all admissible transition realizations. To address the exponential complexity of vertex-based uncertainty representations, we derive a dualization-based encoding that eliminates explicit vertex enumeration and scales linearly with the number of successors. Experimental evaluation on four representative robotic benchmark domains demonstrates scalability to IMDPs with hundreds of thousands of states. The proposed framework provides a practical and general foundation for uncertainty-aware, flexibility-preserving controller synthesis in robotic systems.

强化学习鲁棒控制规划优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。