arXiv:2412.07686cs.ROcs.AI2024-12中稿 · ICAART conference …

优化传感器冗余配置,在控成本下提升强化学习决策效果。

Optimizing Sensor Redundancy in Sequential Decision-Making Problems

  • 用二次逼近预期收益,结合成本惩罚构建优化模型
  • 在8个Gym环境和机器人抓取任务中验证了配置有效性
  • 适合部署在传感器易故障的真实系统中

强化学习策略依赖传感器观测来做出决策,但实际应用中传感器可能因硬件故障或环境因素(如镜头积尘)失效。为应对这一问题,常采用备用传感器,但会增加成本。本文提出一种优化方法,在成本不超过阈值C的前提下,通过二阶近似预期回报并引入超支惩罚,构建二次规划问题,并使用禁忌搜索算法求解。在8个OpenAI Gym环境及一个自定义的Unity机器人抓取环境(RobotArmGrasping)上评估,结果表明该方法能有效逼近真实预期回报,准确识别最优传感器配置。

原文摘要 · Abstract (English)

Reinforcement Learning (RL) policies are designed to predict actions based on current observations to maximize cumulative future rewards. In real-world applications (i.e., non-simulated environments), sensors are essential for measuring the current state and providing the observations on which RL policies rely to make decisions. A significant challenge in deploying RL policies in real-world scenarios is handling sensor dropouts, which can result from hardware malfunctions, physical damage, or environmental factors like dust on a camera lens. A common strategy to mitigate this issue is the use of backup sensors, though this comes with added costs. This paper explores the optimization of backup sensor configurations to maximize expected returns while keeping costs below a specified threshold, C. Our approach uses a second-order approximation of expected returns and includes penalties for exceeding cost constraints. We then optimize this quadratic program using Tabu Search, a meta-heuristic algorithm. The approach is evaluated across eight OpenAI Gym environments and a custom Unity-based robotic environment (RobotArmGrasping). Empirical results demonstrate that our quadratic program effectively approximates real expected returns, facilitating the identification of optimal sensor configurations.

强化学习传感器冗余优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。