用归并排序思想设计新核函数,让高维排列优化更快更准。
From Sorting Algorithms to Scalable Kernels: Bayesian Optimization in High-Dimensional Permutation Spaces
- 基于归并排序构造紧凑排列表示,复杂度仅Θ(n log n)
- 在高维场景下比现有方法快数倍,且优化效果更优
- 适合大规模特征排序和神经架构搜索等复杂任务
贝叶斯优化在黑箱优化中表现优异,但其在高维排列空间的应用受限于可扩展表示的定义难题。当前最优方法依赖Ω(n²)的成对比较,生成稠密表示,难以应对大规模排列。为此,本文提出一种新框架,通过排序算法导出核函数生成高效排列表示。其中,Mallows核可视为枚举排序的特例。进一步提出 extbf{Merge Kernel},利用归并排序的分治结构,实现Θ(n log n)的紧凑表示,无信息损失且有效捕捉排列结构。核心观点是:在低维时,Merge Kernel与Mallows核性能相当;随着维度n增加,其在优化性能和计算效率上均显著超越后者。多种排列优化基准测试验证了该假设,证明Merge Kernel为高维排列空间贝叶斯优化提供了可扩展且更有效的解决方案,从而打开了求解此前不可行的大规模特征排序与组合神经架构搜索等难题的新路径。
原文摘要 · Abstract (English)
Bayesian Optimization (BO) is a powerful tool for black-box optimization, but its application to high-dimensional permutation spaces is severely limited by the challenge of defining scalable representations. The current state-of-the-art BO approach for permutation spaces relies on an exhaustive $Ω(n^2)$ pairwise comparison, inducing a dense representation that is impractical for large-scale permutations. To break this barrier, we introduce a novel framework for generating efficient permutation representations via kernel functions derived from sorting algorithms. Within this framework, the Mallows kernel can be viewed as a special instance derived from enumeration sort. Further, we introduce the \textbf{Merge Kernel} , which leverages the divide-and-conquer structure of merge sort to produce a compact, $Θ(n\log n)$ to achieve the lowest possible complexity with no information loss and effectively capture permutation structure. Our central thesis is that the Merge Kernel performs competitively with the Mallows kernel in low-dimensional settings, but significantly outperforms it in both optimization performance and computational efficiency as the dimension $n$ grows. Extensive evaluations on various permutation optimization benchmarks confirm our hypothesis, demonstrating that the Merge Kernel provides a scalable and more effective solution for Bayesian optimization in high-dimensional permutation spaces, thereby unlocking the potential for tackling previously intractable problems such as large-scale feature ordering and combinatorial neural architecture search.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。