用特征函数+图优化,让生成模型更稳定高效。
CF-GO-Net: A Universal Distribution Learner via Characteristic Function Networks with Graph Optimizers
- 用特征函数替代概率密度函数,避免分布假设限制。
- 在特征函数空间直接计算距离,训练无约束更稳定。
- 可适配预训练模型,直接在特征空间生成数据。
生成模型旨在学习数据集(如图像)的分布,从而生成统计上与真实数据相似的样本。然而,学习底层概率分布往往极具挑战且难以处理。为此,本文提出一种基于特征函数(CF)的方法,该函数是分布的直接概率描述,不仅始终存在,还提供额外自由度,增强分布学习的灵活性,摆脱了传统方法对概率密度函数(pdf)的依赖。尽管已有研究尝试使用特征函数进行生成建模,但通常施加了强约束。本文方法在特征函数域中计算查询点间的距离,形成一个无约束且定义明确的问题。此外,为解决采样策略这一关键问题,提出基于图神经网络(GNN)的采样优化器,能识别特征函数差异最大的区域。本方法还可利用预训练模型(如训练良好的自编码器),直接在其特征空间中学习,无需修改其参数。这提供了一种灵活稳健的生成建模方式,不仅适用范围更广、性能更优,还能赋予任何潜在空间以生成能力。
原文摘要 · Abstract (English)
Generative models aim to learn the distribution of datasets, such as images, so as to be able to generate samples that statistically resemble real data. However, learning the underlying probability distribution can be very challenging and intractable. To this end, we introduce an approach which employs the characteristic function (CF), a probabilistic descriptor that directly corresponds to the distribution. However, unlike the probability density function (pdf), the characteristic function not only always exists, but also provides an additional degree of freedom, hence enhances flexibility in learning distributions. This removes the critical dependence on pdf-based assumptions, which limit the applicability of traditional methods. While several works have attempted to use CF in generative modeling, they often impose strong constraints on the training process. In contrast, our approach calculates the distance between query points in the CF domain, which is an unconstrained and well defined problem. Next, to deal with the sampling strategy, which is crucial to model performance, we propose a graph neural network (GNN)-based optimizer for the sampling process, which identifies regions where the difference between CFs is most significant. In addition, our method allows the use of a pre-trained model, such as a well-trained autoencoder, and is capable of learning directly in its feature space, without modifying its parameters. This offers a flexible and robust approach to generative modeling, not only provides broader applicability and improved performance, but also equips any latent space world with the ability to become a generative model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。