首个支持梯度法的多目标优化库，助力大规模模型同时优化多个目标。
LibMOON: A Gradient-based MultiObjective OptimizatioN Library in PyTorch
- 基于梯度的多目标优化方法，利用目标函数的高阶信息。
- 支持千/百万参数模型，突破传统进化算法的规模限制。
- 开源提供公平基准，适用于多任务学习与公平性优化场景。
多目标优化问题（MOPs）在机器学习中广泛存在，应用于多任务学习、公平性或鲁棒性约束下的学习等场景。与将多个目标合并为单一目标不同，MOPs旨在寻找帕累托最优或帕累托集，即同时优化多个目标函数，针对具有数千至数百万参数的模型。现有MOP基准库主要聚焦于进化算法，大多为零阶或启发式方法，未能有效利用目标函数的高阶信息，难以扩展到大规模模型。为此，本文提出LibMOON，首个支持前沿梯度法的多目标优化库，提供公平基准，并向社区开源。
原文摘要 · Abstract (English)
Multiobjective optimization problems (MOPs) are prevalent in machine learning, with applications in multi-task learning, learning under fairness or robustness constraints, etc. Instead of reducing multiple objective functions into a scalar objective, MOPs aim to optimize for the so-called Pareto optimality or Pareto set learning, which involves optimizing more than one objective function simultaneously, over models with thousands / millions of parameters. Existing benchmark libraries for MOPs mainly focus on evolutionary algorithms, most of which are zeroth-order / meta-heuristic methods that do not effectively utilize higher-order information from objectives and cannot scale to large-scale models with thousands / millions of parameters. In light of the above gap, this paper introduces LibMOON, the first multiobjective optimization library that supports state-of-the-art gradient-based methods, provides a fair benchmark, and is open-sourced for the community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。