用在线PCA替代向量量化，实现可微、不崩溃的高效生成模型
PCA-VAE: Differentiable Subspace Quantization without Codebook Collapse
- 以Oja法则训练在线PCA瓶颈，替代传统非可微的向量量化
- 在CelebAHQ上重建质量超越VQ-GAN，且仅需1/10到1/100的隐变量比特
- 隐空间维度自然可解释（如姿态、光照），无需对抗训练或解耦目标
向量量化自编码器虽能生成高质量隐变量，但存在不可微、需直通技巧及易崩溃等固有缺陷。本文从根源解决：用基于Oja规则的在线PCA瓶颈替代向量量化，构建完全可微的模型——PCA-VAE。该模型无需代码本、承诺损失或查表噪声，自动学习正交且按方差排序的隐空间基。尽管结构简单，其在CelebAHQ上的重建质量仍优于VQ-GAN与SimVQ，且隐变量使用比特数仅为后者的1/10至1/100。此外，其隐空间维度自然包含姿态、光照、性别等语义线索，无需对抗正则化或解耦目标。结果表明，PCA是向量量化的一种数学严谨、稳定、高效且语义结构清晰的替代方案，为生成模型提供了新方向。
原文摘要 · Abstract (English)
Vector-quantized autoencoders deliver high-fidelity latents but suffer inherent flaws: the quantizer is non-differentiable, requires straight-through hacks, and is prone to collapse. We address these issues at the root by replacing VQ with a simple, principled, and fully differentiable alternative: an online PCA bottleneck trained via Oja's rule. The resulting model, PCA-VAE, learns an orthogonal, variance-ordered latent basis without codebooks, commitment losses, or lookup noise. Despite its simplicity, PCA-VAE exceeds VQ-GAN and SimVQ in reconstruction quality on CelebAHQ while using 10-100x fewer latent bits. It also produces naturally interpretable dimensions (e.g., pose, lighting, gender cues) without adversarial regularization or disentanglement objectives. These results suggest that PCA is a viable replacement for VQ: mathematically grounded, stable, bit-efficient, and semantically structured, offering a new direction for generative models beyond vector quantization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。