用非平衡活性噪声提升扩散模型的记忆能力,让生成更稳定。
Non-equilibrium active noise enhances generative memory in diffusion models
- 引入具有时间相关性的活性噪声,打破系统平衡态
- 信息衰减速率显著降低,高阶语义信息得以保留
- 适合研究复杂结构生成与能量景观恢复的场景
生成式扩散模型虽能高效采样高维分布,但通常依赖白高斯噪声和固定噪声调度来破坏并重建信息。本文证明,通过使用具有时间相关性的活性噪声驱动生成过程偏离平衡态,可从根本上改变系统的信息热力学。我们发现,将数据耦合至非马尔可夫活性环境后,高阶语义信息(如类别身份或分子亚稳态)会存储于辅助自由度的时间相关性中。基于费舍尔信息分析,该机制显著减缓了信息衰减速率,相比被动布朗运动有明显优势。关键的是,这一记忆效应在逆向生成过程中促进更早、更稳健的对称性破缺(物种分化),使系统能解析多尺度结构,类似分子构型中的亚稳态——这些特征在传统去噪过程中常被抹除。结果表明,受活性物质物理启发的非平衡协议,为利用生成扩散模型恢复高维能量景观提供了一条热力学上不同的潜在优势路径。
原文摘要 · Abstract (English)
Generative diffusion models have emerged as powerful tools for sampling high-dimensional distributions, yet they typically rely on white gaussian noise and noise schedules to destroy and reconstruct information. Here, we demonstrate that driving the generative process out of equilibrium using active, temporally correlated noise sources fundamentally alters the information thermodynamics of the system. We show that coupling the data to an active non-Markovian bath creates a `memory effect' where high-level semantic information (such as class identity or molecular metastability) is stored in the temporal correlations of auxiliary degrees of freedom. Using Fisher information analysis, we prove that this active mechanism significantly retards the rate of information decay compared to passive Brownian motion. Crucially, this memory effect facilitates an earlier and more robust symmetry breaking (speciation) during the reverse generative process, allowing the system to resolve multi-scale structures, reminiscent of metastable states in molecular configurations that are washed out in the typical noising processes. Our results suggest that non-equilibrium protocols, inspired by active matter physics, offer a thermodynamically distinct and potentially advantageous pathway for recovering high-dimensional energy landscapes using generative diffusion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。