提出新指标评估生成模型桥接结构缺陷,可提前预测训练效果。
Bridge Graphical Models: Coupling, Projection, and Current-Preserving Dynamics for Generative Modeling
- 定义马尔可夫化差距衡量生成过程信息损失
- 实验证明该指标预判训练损失与图像质量排序一致
- 适用于扩散模型、流匹配等多种生成架构
连续时间生成模型常基于端点条件桥梁构建,但采样需非前瞻性的马尔可夫解码器,仅依赖当前状态与时间。我们识别出这一桥接至解码器的压缩是扩散模型、流匹配、修正流、薛定谔桥及基于场的生成模型共有的结构性瓶颈。引入‘马尔可夫化差距’,即给定马尔可夫状态时桥接速度的条件方差积分,它衡量从采样器可用信息中预测端点运动的最小均方误差,反映训练前不可消除的损失。为使不同模型家族可比,提出‘桥式图模型’(BGM),将端点耦合、桥律、马尔可夫投影和电流保持动力学表示作为独立设计选择。同一形式也描述泊松与静电模型为场线桥核,并对应场线马尔可夫化差距。在CIFAR-10与Fashion-MNIST的合成、潜在空间与像素空间实验中,训练前数分钟内估算的特征空间代理差距,能与下游训练损失及FID保持相同排序方向,在固定架构、桥接、采样器与计算资源下。结果支持马尔可夫化差距作为桥接与耦合设计的预训练诊断工具。
原文摘要 · Abstract (English)
Continuous-time generative models are often built from endpoint-conditioned bridges, but generation requires a different object: a non-anticipative Markov decoder that only observes the current state and time. We identify this bridge-to-decoder compression as a structural bottleneck shared by diffusion models, flow matching, rectified flow, Schrödinger bridges, and field-based generative models. We introduce the \emph{Markovization gap}, the time-integrated conditional variance of the bridge velocity given the Markov state. It is the MMSE of predicting endpoint-conditioned motion from the information available to a sampler, and it measures an irreducible loss incurred before any neural network is trained. To make this bottleneck comparable across model families, we define \emph{Bridge Graphical Models} (BGMs), which separate endpoint coupling, bridge law, Markovian projection, and current-preserving dynamics representation as independent design choices. The same formalism also represents Poisson and electrostatic models as field-line bridge kernels with a corresponding field-line Markovization gap. Across synthetic, latent, and pixel-space pilots on CIFAR-10 and Fashion-MNIST, a feature-space proxy gap estimated in minutes before training ranks design choices in the same direction as downstream training loss and FID under fixed architecture, bridge, sampler, and compute. These results support the Markovization gap as a pre-training diagnostic for bridge and coupling design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。