arXiv:2605.04087math.OCcs.LG2026-05

无需梯度信息,可在复杂条件下高效优化正交矩阵问题。

BOOOM: Loss-Function-Agnostic Black-Box Optimization over Orthonormal Manifolds for Machine Learning and Statistical Inference

论文配图:BOOOM: Loss-Function-Agnostic Black-Box Optimization over Orthonormal Manifolds for Machine Learning and Statistical Inference
图 1 · 摘自论文原文
  • 用角度空间重构正交流形,实现无约束全局搜索。
  • 在非光滑、多峰场景下优于现有方法,支持并行计算。
  • 适合无梯度、高复杂度的机器学习与统计推断任务。

在统计学、机器学习和科学计算中,对 Stiefel 流形 $ ext{St}(p,d)$(即 $p \times d$ 列正交矩阵集合)的优化至关重要,但面对非凸、非光滑或黑盒目标函数时仍具挑战。现有方法多依赖凸松弛或基于梯度的黎曼优化,难以适用于无导数和高度多模态场景。本文提出 extsc{BOOOM}(Black-box Optimization Over Orthonormal Manifolds),一种损失函数无关的通用框架,用于 $ ext{St}(p,d)$ 上的黑盒优化。核心思想是通过全局 Givens 旋转参数化,将流形精确映射至无约束欧氏角度空间。在此表示基础上, extsc{BOOOM} 采用结构化、可并行的无导数搜索算法——递归改进型模式搜索,通过逐平面旋转实现系统性探索,无需梯度信息,且有助于跳出劣质局部极值。我们建立了统一理论框架,证明角度空间与流形优化等价、平稳性可传递,并在温和条件下实现概率意义上的全局收敛。在多种任务上验证性能:包括异质二次优化、低秩稀疏矩阵分解、独立成分分析及正交联合对角化等广泛研究问题。实验表明,尤其在非光滑和高度多模态场景下,其表现显著优于当前最优方法。此外,通过将监督 PCA 新范式应用于结直肠癌代谢组学数据,展示了其实际应用价值。

原文摘要 · Abstract (English)

Optimization over the Stiefel manifold $\mathrm{St}(p,d)$, the set of $p \times d$ column-orthonormal matrices, is fundamental in statistics, machine learning, and scientific computing, yet remains challenging in the presence of non-convex, non-smooth, or black-box objectives. Existing methods largely rely on either convex relaxations or gradient-based Riemannian optimization, limiting applicability in derivative-free and highly multimodal settings. We propose \textsc{BOOOM} (Black-box Optimization Over Orthonormal Manifolds), a general-purpose framework for loss-function-agnostic optimization on $\mathrm{St}(p,d)$. The key idea is a global Givens rotation-based parametrization that maps the manifold to an unconstrained Euclidean angle space while preserving feasibility exactly. Building on this representation, BOOOM employs a structured, parallelizable, derivative-free search based on Recursive Modified Pattern Search, enabling systematic exploration through plane-wise rotations without requiring gradient information and facilitating escape from poor local optima. We establish a unified theoretical framework showing equivalence between angle-space and manifold optimization, transfer of stationarity, and global convergence in probability under mild conditions. Empirical results across diverse problems, including heterogeneous quadratic optimization, low-rank and sparse matrix decomposition, independent component analysis, and orthogonal joint diagonalization, among other widely studied settings, demonstrate strong performance relative to state-of-the-art methods, particularly in non-smooth and highly multimodal regimes. We further illustrate its practical utility through a novel supervised PCA formulation applied to metabolomics data in colorectal cancer.

优化算法正交流形无梯度优化机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。