arXiv:2606.03904cs.LGcs.CV2026-06

改进Adam优化器在多目标学习中的表现,解决权重与几何失配问题。

MAdam: Metric-Aware Multi-Objective Adam

论文配图:MAdam: Metric-Aware Multi-Objective Adam
图 1 · 摘自论文原文
  • 提出MAdam,通过偏好感知的曲率预处理来修正优化方向。
  • 在多任务学习等场景中,显著提升各类多目标求解器性能。
  • 无需修改现有框架,可直接替换Adam使用,适合多目标学习研究者。

多目标优化(MOO)是许多机器学习问题的核心,但当前主流的损失平衡、梯度平衡和基于帕累托的方法几乎都依赖Adam优化器。我们发现这种耦合导致两个系统性偏差:一是权重不匹配——Adam的二阶矩分母将时变偏好向量与梯度统计混杂,使偏好被历史平均化,不同帕累托权衡趋于近似均匀混合;二是几何不匹配——Adam的自适应度量扭曲了MOO求解器假设的欧氏几何,使原本对齐的目标产生表观冲突。为此,我们提出MAdam(Metric-Aware Multi-Objective Adam),一个无需修改求解器和优化器的即插即用封装。MAdam通过偏好条件化的标量目标曲率对协调方向进行预处理,使在该白化输入下Adam的二阶矩退化为单位矩阵,从而实现由偏好条件度量驱动的真实更新。在多任务学习、帕累托前沿恢复、物理信息神经网络及医学影像等多个任务中,MAdam在所有求解器家族上均一致优于标准Adam。

原文摘要 · Abstract (English)

Multi-objective optimization (MOO) underlies many machine learning problems, yet MOO solvers across the loss-balancing, gradient-balancing, and Pareto-based families almost universally hand their reconciled directions to Adam~\cite{kingma2015adam}. We show this coupling introduces two systematic gaps between the solver's intent and the optimizer's execution. The first is a \emph{weighting mismatch}: Adam's second-moment denominator entangles the time-varying preference vector with gradient statistics, marginalizing the preference into a history average and collapsing distinct Pareto trade-offs toward a near-uniform mixture. The second is a \emph{geometric mismatch}: Adam's adaptive metric distorts the Euclidean geometry MOO solvers assume, turning aligned objectives into apparent conflicts. To resolve both jointly, we introduce \textbf{MAdam} (Metric-Aware Multi-Objective Adam), a drop-in wrapper that leaves both solver and optimizer unchanged. MAdam preconditions the reconciled direction by the preference-conditioned curvature of the scalarized objective; on this whitened input, Adam's second moment collapses to identity, so the realized update is governed by the preference-conditioned metric. Across multi-task learning, Pareto-front recovery, physics-informed neural networks, and medical imaging, MAdam consistently improves over Adam for every solver family.

多目标优化Adam改进机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。