arXiv:2604.01843cs.CVcs.LG2026-04中稿 · ICPR 2026

让图像编码摆脱位置依赖,实现无须先验的直接生成

Investigating Permutation-Invariant Discrete Representation Learning for Spatially Aligned Images

  • 设计无位置信息的向量量化架构,强制编码捕捉全局语义
  • 通过最优匹配量化提升瓶颈容量3.5倍,支持高效图像合成
  • 可直接插值生成新图像,适合需要可控生成的应用场景

向量量化方法(如VQ-VAE、VQ-GAN)学习图像的离散神经表示,但这些表示具有固有的位置依赖性:编码按空间排列且上下文耦合,需依赖自回归或扩散模型先验来建模样本间依赖。本文探讨空间对齐数据的离散表示是否必须包含位置信息。提出一种无位置不变的向量量化自编码器(PI-VQ),约束潜在编码不携带位置信息。实验发现该约束促使编码聚焦于全局语义特征,并支持无需学习先验的直接图像插值。为缓解无位置表示的信息容量下降,引入匹配量化(matching quantization),基于最优二分图匹配,使有效瓶颈容量相较朴素最近邻量化提升3.5倍。学习到的编码具有组合结构,支持基于插值的采样,可在单次前向传播中合成新图像。在CelebA、CelebA-HQ和FFHQ数据集上评估,生成图像在精度、密度和覆盖率方面表现竞争力。讨论了无位置表示的权衡,包括编码的可分离性和可解释性,指明未来研究方向。

原文摘要 · Abstract (English)

Vector quantization approaches (VQ-VAE, VQ-GAN) learn discrete neural representations of images, but these representations are inherently position-dependent: codes are spatially arranged and contextually entangled, requiring autoregressive or diffusion-based priors to model their dependencies at sample time. In this work, we ask whether positional information is necessary for discrete representations of spatially aligned data. We propose the permutation-invariant vector-quantized autoencoder (PI-VQ), in which latent codes are constrained to carry no positional information. We find that this constraint encourages codes to capture global, semantic features, and enables direct interpolation between images without a learned prior. To address the reduced information capacity of permutation-invariant representations, we introduce matching quantization, a vector quantization algorithm based on optimal bipartite matching that increases effective bottleneck capacity by $3.5\times$ relative to naive nearest-neighbour quantization. The compositional structure of the learned codes further enables interpolation-based sampling, allowing synthesis of novel images in a single forward pass. We evaluate PI-VQ on CelebA, CelebA-HQ and FFHQ, obtaining competitive precision, density and coverage metrics for images synthesised with our approach. We discuss the trade-offs inherent to position-free representations, including separability and interpretability of the latent codes, pointing to numerous directions for future work.

向量量化无位置编码图像生成插值采样

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。