arXiv:2606.02909stat.MLcs.LG2026-06

提出高效梯度高斯过程方法TERA,突破高维优化计算瓶颈。

Scalable Derivative Gaussian Processes via Exact Gradient Reduction

论文配图:Scalable Derivative Gaussian Processes via Exact Gradient Reduction
图 1 · 摘自论文原文
  • 基于目标特定梯度降维,将高维梯度信息压缩为低维方向导数
  • 在10维以上场景下推理速度比传统方法快100倍以上,内存几乎不随维度增长
  • 适合高维黑箱优化、科学模拟等需梯度信息的场景

梯度观测能显著提升高维场景中昂贵函数评估的高斯过程代理模型性能。然而,包含 $n$ 个函数值和 $n$ 个完整梯度的精确推断在 $d$ 维空间中计算复杂度达 $/mathcal{O}(n^3 d^3)$,存在难以处理的计算瓶颈。我们提出 TERA,一种基于目标特定精确梯度降维的可扩展导数高斯过程方法。证明对于平稳核,与目标点及条件点连线方向正交的梯度分量在给定条件下与目标函数值独立;因此,精确后验密度仅由最多 $m^2$ 个方向导数完全刻画。通过将这些降维、无维度依赖的条件分布作为局部因子引入 Vecchia 近似,TERA 有效解耦了 $n$ 与 $d$ 对密集矩阵求逆的影响。每目标评估时间降至 $/mathcal{O}(dm^2 + m^6)$,内存为 $/mathcal{O}(dm^2 + m^4)$,保持原始导数高斯过程的数学形式不变。实验表明,TERA 在保持最先进预测精度的同时,推理速度比标准导数高斯过程快数个数量级。关键的是,计算时间和峰值 GPU 内存基本不随 $d$ 增长,实现了高维空间中的高度可扩展推断。

原文摘要 · Abstract (English)

Gradient observations can substantially improve Gaussian process (GP) surrogates, particularly in high-dimensional settings where function evaluations are expensive. However, exact inference with $n$ function values and $n$ full gradients in $d$ dimensions scales cubically in the joint state size, imposing an intractable $\mathcal{O}(n^3 d^3)$ computational bottleneck. We introduce TERA, a highly scalable derivative GP method based on target-specific exact gradient reduction. We prove that for stationary kernels, the gradient components orthogonal to the directions connecting the target and conditioning points are conditionally independent of the target function value; consequently, the exact conditional density is fully characterized by at most $m^2$ directional derivatives once a conditioning set of size $m$ is specified. By using these reduced, dimension-free conditionals as local factors in a Vecchia approximation, TERA effectively decouples $n$ and $d$ from the dense matrix inversion. This reduces the per-target evaluation cost to $\mathcal{O}(dm^2 + m^6)$ time and $\mathcal{O}(dm^2 + m^4)$ memory, leaving the underlying derivative GP model mathematically unchanged. Empirical evaluations demonstrate that TERA achieves state-of-the-art predictive accuracy while operating orders of magnitude faster than standard derivative GPs. Crucially, both computation time and peak GPU memory remain essentially flat with respect to $d$, enabling highly scalable inference in high-dimensional spaces.

高斯过程梯度信息高维优化可扩展性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。