用强化学习约束束搜索优化纸张干燥参数，支持灵活调整约束条件。
Reinforcement Learning Constrained Beam Search for Parameter Optimization of Paper Drying Under Flexible Constraints
- 推理时通过束搜索动态调整约束，支持排除无效动作、强制包含目标动作。
- 在复杂约束下优于NSGA-II，速度提升2.58倍以上。
- 适合非实时的强化学习规划与优化任务，如工业过程参数设计。
现有强化学习应用中施加设计约束的方法通常依赖训练时的奖励惩罚或训练/推理时的无效动作屏蔽，但这些方法要么无法在训练后修改，要么对可实现的约束类型有限制。为解决此问题，我们提出强化学习约束束搜索（RLCBS），用于组合优化问题的推理时优化。该方法支持灵活的推理时约束，可排除无效动作并强制包含期望动作，并利用束搜索最大化序列概率，实现更合理的约束融合。RLCBS可扩展至无需实时求解的强化学习规划与优化问题，我们将其应用于新型模块化纸张干燥测试平台的过程参数优化。一个强化学习代理通过生成最优干燥模块与空气供应温度配置，在不同机器速度水平下最小化能耗。结果表明，相较于NSGA-II，在复杂干燥模块配置约束下，RLCBS在推理时表现更优，且速度提升2.58倍或更高。
原文摘要 · Abstract (English)
Existing approaches to enforcing design constraints in Reinforcement Learning (RL) applications often rely on training-time penalties in the reward function or training/inference-time invalid action masking, but these methods either cannot be modified after training, or are limited in the types of constraints that can be implemented. To address this limitation, we propose Reinforcement Learning Constrained Beam Search (RLCBS) for inference-time refinement in combinatorial optimization problems. This method respects flexible, inference-time constraints that support exclusion of invalid actions and forced inclusion of desired actions, and employs beam search to maximize sequence probability for more sensible constraint incorporation. RLCBS is extensible to RL-based planning and optimization problems that do not require real-time solution, and we apply the method to optimize process parameters for a novel modular testbed for paper drying. An RL agent is trained to minimize energy consumption across varying machine speed levels by generating optimal dryer module and air supply temperature configurations. Our results demonstrate that RLCBS outperforms NSGA-II under complex design constraints on drying module configurations at inference-time, while providing a 2.58-fold or higher speed improvement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。