用零膨胀高斯分布让进化算法自动发现稀疏解,无需手动设计规则。
Zero-Inflated Gaussian Distributions Enable Parameter-Space Sparsity in Estimation-of-Distribution Algorithms

- 引入零膨胀高斯分布,联合优化稀疏模式与活跃参数值。
- 在月球着陆器任务中收敛更快,最终奖励更高,仅激活少量参数。
- 适合需要稀疏结构的黑箱优化场景,如神经网络剪枝或特征选择。
估计分布算法(EDAs)是一类强大的黑箱优化进化方法,尤其适用于对目标函数结构知之甚少的情况。与依赖人工设计变异和交叉算子、易引入偏见的经典进化算法不同,EDAs 完全避开算子设计:通过拟合优质个体的概率分布,并从中采样生成下一代。尽管在连续参数空间上已有广泛应用,但现有方法尚未推广至稀疏参数空间——即优质解中多数系数恰好为零的情形。因此,现有稀疏黑箱优化器不得不重新引入本应避免的手工稀疏操作符、双层支持集与活跃值交替优化、置零阈值等假设。本文提出多变量零膨胀高斯(ZIG)分布作为EDAs的采样策略。该模型采用潜在高斯结构,分离指示维度与取值维度,可同时建模稀疏模式、活跃参数间的相关性以及两者间的交互,实现无层级的联合优化。我们证明该模型的潜在参数可从观测样本中唯一识别,不同于相关研究中的缺失数据设定,并提出了实用的基于渐近反演的估计器。实验表明,该估计器能准确恢复潜在相关结构;在Lunar Lander基准测试中,所提出的ZIG-EDA相比密集高斯EDAs、手工稀疏进化算法及非正式稀疏EDAs,收敛更快、最终回报更高,且控制器仅激活极小比例参数。
原文摘要 · Abstract (English)
Estimation-of-distribution algorithms (EDAs) are a powerful class of evolutionary methods for black-box optimization, especially when little is known about the structure of the objective. Whereas classical evolutionary algorithms rely on hand-designed mutation and crossover operators, hard to devise for unknown problem structures, and a source of bias, EDAs sidestep operator design entirely: they fit a probability distribution to the best individuals and sample the next generation from it. EDAs are well established on continuous parameter spaces, but they have not previously been generalized to sparse ones, in which most coefficients of a good solution are exactly zero. Existing sparse black-box optimizers therefore reintroduce exactly what EDAs were designed to avoid: hand-crafted sparsity operators, bi-level schemes alternating between support set and active values, zeroing thresholds, and other baked-in assumptions. We close this gap by proposing multivariate zero-inflated Gaussian (ZIG) distributions as EDA sampling laws. A latent Gaussian model with separate indicator and value dimensions represents sparsity patterns, correlations among active parameters, and the interactions between the two, so sparsity patterns and active values are optimized jointly, hierarchy-free. We show that the latent parameters of this model are identifiable from observed samples, unlike in the missing-data settings where related constructions originate, and introduce practical amortized inversion-based estimators for them. The estimators accurately recover latent correlation structures, and on the Lunar Lander benchmark the resulting ZIG-EDA converges faster and reaches higher final returns than a dense Gaussian EDA, a hand-crafted sparse evolutionary algorithm, and an ad-hoc sparse EDA, while finding controllers with only a small fraction of parameters active.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。