arXiv:2605.18224cs.LGcs.AI2026-05

提出检测VAE编码器坍缩的简单形验证器与逃逸力机制。

A Simplex Witness Certificate and Escape Force for Constant Collapse in Variational Autoencoders

  • 构建固定教师后验与单纯形验证器,形成可验证的坍缩判据
  • 在所有种子上,原始VAE在CIFAR-10/100均不通过验证,而RST变体通过
  • 揭示坍缩流形上的梯度逃逸路径,适合研究编码器稳定性的研究者

本文研究变分自编码器中编码器均值完全坍缩的问题:即编码器均值与输入无关。先验保持标准高斯分布。训练前,从基于GMM的数据视角选取固定教师后验,并将一个仅依赖潜变量的单纯形验证器附加到编码器均值。该构造产生两个关联对象:一是验证证书——若验证器预测优于对教师后验的最佳常数预测,则编码器均值不可能为输入无关常数;二是局部逃逸方向——在坍缩流形上,教师残差提供一种样本相关的对齐损失下降方向。对于任意全支撑教师后验,该几何结构还给出闭式潜变量代码,使教师-验证器对齐误差为零。其缩放版本构成从常数预测到精确教师代码的边际-能量路径,量化了受保护验证子空间内的非坍缩程度。方法在MNIST、CIFAR-10和CIFAR-100上实现。使用搜索得到的无监督PCA-GMM教师,在五个随机种子下,原始VAE在CIFAR-10和CIFAR-100均未通过教师-验证器证书,而RST变体全部通过。在β_KL∈{2,4,8}的坍缩压力设置下,原始VAE再次失败,而RST-alpha-prefit始终通过证书。自然图像数据集上的逃逸轨迹显示,验证器边界从低边界初始化开始提升,并表现出非零的教师诱导梯度范数。分析局限于编码器均值的精确常数坍缩;生成质量、解码器使用及其他坍缩模式仍为独立问题。

原文摘要 · Abstract (English)

We study exact constant collapse in variational autoencoders: the deterministic encoder mean becomes independent of the input. The prior remains the standard Gaussian. Before VAE training, we select a fixed teacher posterior from a GMM-based view of the data and attach a fixed latent-only simplex witness to the encoder mean. This construction yields two linked objects. The first is a certificate: if the witness prediction improves on the best constant predictor of the teacher, the encoder mean cannot be input-independent constant. The second is a local escape direction: on the collapsed manifold, the teacher residual gives a sample-dependent descent direction for the alignment loss. For any full-support teacher posterior, the same geometry also gives a closed-form latent code with zero teacher-witness alignment error. Its scaled versions trace a margin-energy path from the constant predictor to the exact teacher code, which quantifies non-collapse inside the protected witness subspace. We instantiate the method on MNIST, CIFAR-10, and CIFAR-100. With searched unsupervised PCA-GMM teachers, vanilla VAEs fail the teacher-witness certificate in all five seeds on CIFAR-10 and CIFAR-100, while RST variants pass in all five seeds. Under collapse-stress settings with \(β_{\mathrm{KL}}\in\{2,4,8\}\), vanilla VAE again fails in all seeds, whereas RST-alpha-prefit remains certificate-positive. Escape trajectories on both natural-image datasets increase the witness margin from a low-margin initialization and exhibit nonzero teacher-induced gradient norms. The analysis is confined to exact constant collapse of the encoder mean; generation quality, decoder use, and other collapse modes remain separate questions.

VAE坍缩检测验证器逃逸路径

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。