用纠错码提升离散VAE的推理精度，改善生成与重构效果
Improved Variational Inference in Discrete VAEs using Error Correcting Codes
- 将离散VAE视为通信系统，引入纠错码增强潜在表示冗余
- 在二值潜在变量上使用重复码，显著降低变分间隙，提升重建质量
- 适合需要高精度推理与不确定性校准的离散表征学习任务
尽管深度概率模型取得进展，学习离散潜在表示仍具挑战。本文提出一种新方法，通过生成视角重构推理问题，将模型视为通信系统，利用纠错码（ECCs）在潜在表示中引入冗余，使变分后验能更准确估计并缩小变分差距。我们以二值潜在变量的离散变分自编码器为基础，采用低复杂度重复码进行概念验证，并扩展至分层结构以解耦全局与局部数据特征。该方法显著提升生成质量、数据重构能力及不确定性校准，即使在使用紧致边界（如重要性加权自编码器目标）训练时，仍优于无编码模型。同时，我们阐述了有效用于改进离散变分推理的纠错码应具备的特性。
原文摘要 · Abstract (English)
Despite advances in deep probabilistic models, learning discrete latent representations remains challenging. This work introduces a novel method to improve inference in discrete Variational Autoencoders by reframing the inference problem through a generative perspective. We conceptualize the model as a communication system, and propose to leverage Error-Correcting Codes (ECCs) to introduce redundancy in latent representations, allowing the variational posterior to produce more accurate estimates and reduce the variational gap. We present a proof-of-concept using a Discrete Variational Autoencoder with binary latent variables and low-complexity repetition codes, extending it to a hierarchical structure for disentangling global and local data features. Our approach significantly improves generation quality, data reconstruction, and uncertainty calibration, outperforming the uncoded models even when trained with tighter bounds such as the Importance Weighted Autoencoder objective. We also outline the properties that ECCs should possess to be effectively utilized for improved discrete variational inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。