扩散模型本质是进化算法,可高效寻找多组最优解。
Diffusion Models are Evolutionary Algorithms
- 用迭代去噪模拟进化中的选择、变异与隔离
- 在参数空间中找到多个最优解,性能超越主流算法
- 支持高维复杂空间求解,计算步骤大幅减少
本文揭示扩散模型本质上是进化算法。将进化视为去噪过程,逆向进化即为扩散,数学上证明扩散模型天然具备选择、突变和生殖隔离机制。基于此,提出扩散进化方法:利用扩散模型中的迭代去噪策略,在参数空间中启发式优化解。相比传统方法,该方法能高效发现多个最优解,性能更优。进一步结合扩散模型的潜在空间扩散与加速采样技术,提出潜在空间扩散进化,可在高维复杂参数空间中快速求解,显著减少计算步数。这一扩散与进化的类比不仅连接了机器学习与生物学,也为双向提升提供新路径,引发关于开放演化问题的思考,并可能在非高斯或离散扩散模型中拓展应用。
原文摘要 · Abstract (English)
In a convergence of machine learning and biology, we reveal that diffusion models are evolutionary algorithms. By considering evolution as a denoising process and reversed evolution as diffusion, we mathematically demonstrate that diffusion models inherently perform evolutionary algorithms, naturally encompassing selection, mutation, and reproductive isolation. Building on this equivalence, we propose the Diffusion Evolution method: an evolutionary algorithm utilizing iterative denoising -- as originally introduced in the context of diffusion models -- to heuristically refine solutions in parameter spaces. Unlike traditional approaches, Diffusion Evolution efficiently identifies multiple optimal solutions and outperforms prominent mainstream evolutionary algorithms. Furthermore, leveraging advanced concepts from diffusion models, namely latent space diffusion and accelerated sampling, we introduce Latent Space Diffusion Evolution, which finds solutions for evolutionary tasks in high-dimensional complex parameter space while significantly reducing computational steps. This parallel between diffusion and evolution not only bridges two different fields but also opens new avenues for mutual enhancement, raising questions about open-ended evolution and potentially utilizing non-Gaussian or discrete diffusion models in the context of Diffusion Evolution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。