用课程学习+推理时碰撞模型，解决垃圾分拣中容器管理的高危低效问题
Curriculum RL meets Monte Carlo Planning: Optimization of a Real World Container Management Problem
- 分阶段训练强化学习代理应对延迟奖励和类别不平衡
- 推理时引入碰撞模型，减少安全违规90%以上，吞吐量保持高位
- 适合有容量限制的真实工业场景，尤其适用于容器与处理单元比例多变的情况
本文通过在强化学习中引入推理时的碰撞模型，提升有限处理能力的垃圾分拣设施中容器管理的安全性与效率。每个容器存在两个最优清空体积，在提高吞吐量与降低溢出风险间权衡。传统强化学习方法在延迟奖励、稀疏关键事件和高维不确定性下表现不佳，难以持续平衡高体积清空与安全限值违规风险。为此，提出混合方法：(1) 基于课程学习的PPO训练流程，逐步增强代理对延迟奖励和类别不平衡的适应能力；(2) 离线构建的成对碰撞模型，在推理阶段以极低成本主动规避碰撞。实验表明，该方法显著提升碰撞避免率，安全限值违规减少超90%，同时维持高吞吐量，并在不同容器与处理单元（PU）比例下具有良好可扩展性。研究成果为真实设施中的安全高效容器管理系统设计提供实用指导。
原文摘要 · Abstract (English)
In this work, we augment reinforcement learning with an inference-time collision model to ensure safe and efficient container management in a waste-sorting facility with limited processing capacity. Each container has two optimal emptying volumes that trade off higher throughput against overflow risk. Conventional reinforcement learning (RL) approaches struggle under delayed rewards, sparse critical events, and high-dimensional uncertainty -- failing to consistently balance higher-volume empties with the risk of safety-limit violations. To address these challenges, we propose a hybrid method comprising: (1) a curriculum-learning pipeline that incrementally trains a PPO agent to handle delayed rewards and class imbalance, and (2) an offline pairwise collision model used at inference time to proactively avert collisions with minimal online cost. Experimental results show that our targeted inference-time collision checks significantly improve collision avoidance, reduce safety-limit violations, maintain high throughput, and scale effectively across varying container-to-PU ratios. These findings offer actionable guidelines for designing safe and efficient container-management systems in real-world facilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。