通过参数与性能的局部线性关系，实现多目标强化学习的可解释性与高效解搜索。
Interpretability by Design for Efficient Multi-Objective Reinforcement Learning
- 基于参数空间与性能空间的局部线性映射，设计可解释的策略表示。
- 在多个连续控制任务中，帕累托前沿质量与生成效率均优于现有方法。
- 适合需要快速生成高质量多目标解的工程应用，如机器人控制。
多目标强化学习（MORL）旨在优化多个常冲突的目标，以提升强化学习在实际任务中的灵活性与可靠性。通常通过寻找一组多样且非支配的策略构成性能空间中的帕累托前沿来实现。本文提出LLE-MORL，通过基于参数空间与性能空间局部关系的训练机制，实现可解释性设计。利用两者间的局部线性映射，该方法可将策略参数解释为对各目标的影响，并提供结构化表示，从而在连续解域内高效搜索，无需大量重训练即可快速生成高质量解。在多个连续控制任务上的实验表明，LLE-MORL始终优于现有先进方法，在帕累托前沿质量与效率方面表现更优。
原文摘要 · Abstract (English)
Multi-objective reinforcement learning (MORL) aims at optimising several, often conflicting goals to improve the flexibility and reliability of RL in practical tasks. This is typically achieved by finding a set of diverse, non-dominated policies that form a Pareto front in the performance space. We introduce LLE-MORL, an approach that achieves interpretability by design by utilising a training scheme based on the local relationship between the parameter space and the performance space. By exploiting a locally linear map between these spaces, our method provides an interpretation of policy parameters in terms of the objectives, and this structured representation enables an efficient search within contiguous solution domains, allowing for the rapid generation of high-quality solutions without extensive retraining. Experiments across diverse continuous control domains demonstrate that LLE-MORL consistently achieves higher Pareto front quality and efficiency than state-of-the-art approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。