arXiv:2603.14536cs.CV2026-03

低分辨率训练的VAE编码器能有效重构高分辨率图像,打破常规认知。

Distilling Latent Manifolds: Resolution Extrapolation by Variational Autoencoders

  • 在低分辨率下蒸馏编码器,通过输入上采样实现高分辨率重建
  • 在512²分辨率上性能显著提升,超越原生训练限制
  • 适合资源受限场景下的高效高分辨率生成应用

变分自编码器(VAE)编码器在现代生成模型中至关重要,但计算开销常促使采用知识蒸馏或量化以获得紧凑替代。现有研究普遍认为模型在接近训练数据分布的样本上表现更优,而本文报告了反直觉现象:仅在低分辨率下蒸馏的VAE编码器,在其原生分辨率下重建效果差,但在更高、未见过的输入分辨率下表现大幅提升。即使从未训练超过256²分辨率,该蒸馏编码器在512²分辨率输入上仍表现出色,部分继承教师模型的分辨率偏好。我们分析不同分辨率下的潜在分布,发现高分辨率输入产生的潜在表示更接近教师模型的流形。在ImageNet-256上的大量实验表明,简单的分辨率重映射(即上采样输入后编码,再下采样重建评估)在PSNR、MSE、SSIM、LPIPS和rFID等多项指标上均带来显著提升。这表明VAE编码器蒸馏学习的是与分辨率一致的潜在流形,而非特定分辨率的像素映射。这意味着高分辨率图像重建能力的蒸馏无需昂贵的高分辨率训练成本,低分辨率数据集也能学到教师模型的高分辨率重建知识。

原文摘要 · Abstract (English)

Variational Autoencoder (VAE) encoders play a critical role in modern generative models, yet their computational cost often motivates the use of knowledge distillation or quantification to obtain compact alternatives. Existing studies typically believe that the model work better on the samples closed to their training data distribution than unseen data distribution. In this work, we report a counter-intuitive phenomenon in VAE encoder distillation: a compact encoder distilled only at low resolutions exhibits poor reconstruction performance at its native resolution, but achieves dramatically improved results when evaluated at higher, unseen input resolutions. Despite never being trained beyond $256^2$ resolution, the distilled encoder generalizes effectively to $512^2$ resolution inputs, partially inheriting the teacher model's resolution preference.We further analyze latent distributions across resolutions and find that higher-resolution inputs produce latent representations more closely aligned with the teacher's manifold. Through extensive experiments on ImageNet-256, we show that simple resolution remapping-upsampling inputs before encoding and downsampling reconstructions for evaluation-leads to substantial gains across PSNR, MSE, SSIM, LPIPS, and rFID metrics. These findings suggest that VAE encoder distillation learns resolution-consistent latent manifolds rather than resolution-specific pixel mappings. This also means that the high training cost on memory, time and high-resolution datasets are not necessary conditions for distilling a VAE with high-resolution image reconstruction capabilities. On low resolution datasets, the distillation model still could learn the detailed knowledge of the teacher model in high-resolution image reconstruction.

VAE蒸馏分辨率外推潜在流形

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。