用Q值函数替代状态值函数，提升规划策略的效率与鲁棒性。
Per-Domain Generalizing Policies: On Learning Efficient and Robust Q-Value Functions (Extended Version with Technical Appendix)
- 改用可高效评估的Q值函数，仅需处理当前状态
- 引入正则化项区分教师动作与非动作，显著提升性能
- 在10个领域中超越状态值策略，媲美LAMA-first规划器
在规划学习中，为每个领域学习泛化策略是一项关键挑战。传统方法使用图神经网络通过监督学习，基于教师规划器生成的最优计划来学习状态值函数。本文主张学习Q值函数而非状态值函数，因其对给定状态的评估成本更低，只需处理当前状态而非所有后继状态。令人意外的是，直接监督学习Q值表现不佳，因无法区分教师采取的动作与未采取的动作。为此，我们引入正则化项以强制该区分，所获Q值策略在10个不同领域中均持续优于状态值策略，且性能媲美规划器LAMA-first。
原文摘要 · Abstract (English)
Learning per-domain generalizing policies is a key challenge in learning for planning. Standard approaches learn state-value functions represented as graph neural networks using supervised learning on optimal plans generated by a teacher planner. In this work, we advocate for learning Q-value functions instead. Such policies are drastically cheaper to evaluate for a given state, as they need to process only the current state rather than every successor. Surprisingly, vanilla supervised learning of Q-values performs poorly as it does not learn to distinguish between the actions taken and those not taken by the teacher. We address this by using regularization terms that enforce this distinction, resulting in Q-value policies that consistently outperform state-value policies across a range of 10 domains and are competitive with the planner LAMA-first.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。