通过动态调整平滑半径,提升零阶非凸优化的探索与精调平衡。
Power Homotopy for Zeroth-Order Non-Convex Optimizations
- 采用递减平滑半径策略,早期大半径保梯度信号,后期小半径精调局部结构。
- 在ImageNet攻击任务中(维度15万+),性能显著优于固定半径方法。
- 适合高维、无梯度的非凸优化场景,如对抗样本生成与黑箱攻击。
现有方法GS-PowerOpt通过最大化高斯平滑代理函数 $F_{N,σ}(\boldsymbolμ) = \mathbb{E}_{\boldsymbol{x}\sim\mathcal{N}(\boldsymbolμ,σ^2 I_d)}[e^{N f(\boldsymbol{x})}]$ 来求解形如 $\max_{\boldsymbol{x} \in \mathbb{R}^d} f(\boldsymbol{x})$ 的非凸优化问题。我们分析了平滑半径 $σ>0$ 的作用,发现固定 $σ$ 设计存在固有缺陷:较大 $σ$ 有利于全局探索和有限时间优化,但可能扭曲代理函数最大值位置;较小 $σ$ 更好保持局部结构,但会削弱远离高价值区域的梯度信号。为此,提出GS-PowerHP——一种具有递减 $σ$ 调度的幂平滑同伦方法。该机制在早期使用较大平滑半径以在远离最优区域时维持有效梯度信号,随后逐步减小 $σ$ 以增强靠近最大值处的局部精调能力。理论分析表明,该递减调度可改进固定 $σ$ 幂平滑的探索-精调权衡。实验表明,GS-PowerHP持续优于固定 $σ$ 基线,在多个优化任务中表现稳健,尤其在对ImageNet的对抗攻击($d=150{,}528$)中显著超越其他基于平滑的零阶方法。
原文摘要 · Abstract (English)
The existing method of GS-PowerOpt solves the non-convex optimization problem of the form $\max_{\boldsymbol{x} \in \mathbb{R}^d} f(\boldsymbol{x})$ through maximizing a Gaussian-smoothed surrogate $F_{N,σ}(\boldsymbolμ) = \mathbb{E}_{\boldsymbol{x}\sim\mathcal{N}(\boldsymbolμ,σ^2 I_d)}[e^{N f(\boldsymbol{x})}]$. We analyze the role of the smoothing radius $σ>0$ and identify a limitation of the fixed-$σ$ design used in GS-PowerOpt. Specifically, $σ$ induces an inherent exploration--refinement tradeoff: a larger $σ$ improves global exploration and finite-time surrogate optimization, but may distort the location of the surrogate maximizer; in contrast, a smaller $σ$ better preserves local structure but can weaken gradient signals away from high-value regions. To address this limitation, we propose GS-PowerHP, a power-smoothed homotopy method with an incrementally decaying $σ$ schedule. The proposed mechanism uses larger smoothing radii in early iterations to maintain informative gradient signals when the iterate is far from high-value regions, and gradually decreases $σ$ to improve local refinement near the maximizer. We provide theoretical results showing that this decaying schedule improves the exploration--refinement tradeoff of fixed-$σ$ power smoothing. Empirically, GS-PowerHP consistently outperforms the fixed-$σ$ baseline and exhibits robust performance across different optimization tasks, including adversarial attacks on ImageNet ($d=150{,}528$), where it substantially improves over other smoothing-based zeroth-order methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。