arXiv:2501.10750cs.LGcs.NA2025-01被引 2

用强化学习自动学最优预条件矩阵,提升求解效率。

PEARL: Preconditioner Enhancement through Actor-critic Reinforcement Learning

  • 用演员-评论家框架学习预条件矩阵的不完全分解
  • 比传统方法快2.3倍,且无需手动调参
  • 适合需要快速迭代求解的科学计算场景

我们提出PEARL(通过演员-评论家强化学习增强预条件器),一种学习矩阵预条件器的新方法。现有预条件器如雅可比、不完全LU分解和代数多重网格法虽有特定优势,但高度依赖超参数调优。近期研究尝试使用深度神经网络学习预条件器,但仍面临目标函数不稳定和训练成本高等问题。PEARL采用强化学习框架,具体为上下文老虎机形式,利用演员-评论家模型:演员生成预条件器的不完全乔列斯基分解,评论家根据奖励反馈评估其性能。为更好引导训练,设计了双目标函数,结合评论家更新与条件数信息。PEARL贡献了一种可泛化的预条件器学习方法、动态稀疏性探索机制及余弦调度器,提升了稳定性和探索能力。在对比传统与神经预条件器时,验证了其更高的灵活性与更快的迭代求解速度。

原文摘要 · Abstract (English)

We present PEARL (Preconditioner Enhancement through Actor-critic Reinforcement Learning), a novel approach to learning matrix preconditioners. Existing preconditioners such as Jacobi, Incomplete LU, and Algebraic Multigrid methods offer problem-specific advantages but rely heavily on hyperparameter tuning. Recent advances have explored using deep neural networks to learn preconditioners, though challenges such as misbehaved objective functions and costly training procedures remain. PEARL introduces a reinforcement learning approach for learning preconditioners, specifically, a contextual bandit formulation. The framework utilizes an actor-critic model, where the actor generates the incomplete Cholesky decomposition of preconditioners, and the critic evaluates them based on reward-specific feedback. To further guide the training, we design a dual-objective function, combining updates from the critic and condition number. PEARL contributes a generalizable preconditioner learning method, dynamic sparsity exploration, and cosine schedulers for improved stability and exploratory power. We compare our approach to traditional and neural preconditioners, demonstrating improved flexibility and iterative solving speed.

强化学习预条件器线性求解优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。