arXiv:2605.22777cs.CV2026-05

用8个轻量查询提升图像重建与生成质量,不破坏预训练语义空间。

DecQ: Detail-Condensing Queries for Enhanced Reconstruction and Generation in Representation Autoencoders

论文配图:DecQ: Detail-Condensing Queries for Enhanced Reconstruction and Generation in Representation Autoencoders
图 1 · 摘自论文原文
  • 引入轻量细节压缩查询,从视觉模型中间层提取细粒度信息
  • 重建性能提升:PSNR从19.13 dB增至22.76 dB,仅增3.9%计算量
  • 生成更快更准:无需引导时收敛快3.3倍,FID低至1.05

表示自编码器(RAEs)利用冻结的视觉基础模型(VFMs)作为分词编码器,提供鲁棒的高层表征,促进潜在扩散模型快速收敛和高质量生成。然而,冻结VFM会限制其空间重建能力,制约细粒度生成与图像编辑;而通过微调引入重建信号又会破坏预训练语义空间,降低生成保真度。为此,我们提出DecQ,一种简单有效的RAE框架。DecQ引入轻量级细节压缩查询,通过压缩模块从VFM中间特征中提取细粒度信息,并将其融入解码器以支持重建,同时在生成建模中与块令牌共同生成。通过融合浅层与深层信息,DecQ有效缓解了重建-生成权衡,提升两者性能。实验表明:(1) 仅增加8个查询和3.9%计算量,基于DINOv2的冻结RAE重建性能显著提升,PSNR从19.13 dB增至22.76 dB;(2) 在生成建模中,DecQ实现3.3倍于RAE的收敛速度,无需引导时达到FID 1.41,有引导时达1.05。

原文摘要 · Abstract (English)

Representation Autoencoders (RAEs) leverage frozen vision foundation models (VFMs) as tokenizer encoders, providing robust high-level representations that facilitate fast convergence and high-quality generation in latent diffusion models. However, freezing the VFM inherently constrains its spatial reconstruction capacity, limiting fine-grained generation and image editing; in contrast, incorporating reconstruction-oriented signals via fine-tuning disrupts the pretrained semantic space and degrades generative fidelity. To address this trade-off, we propose DecQ, a simple yet effective framework for RAEs. Specifically, DecQ introduces lightweight detail-condensing queries that extract fine-grained information from intermediate VFM features through condenser modules. These queries are incorporated into the decoder to support reconstruction and are jointly generated with patch tokens during generative modeling. By aggregating information from both shallow and deep layers, DecQ effectively mitigates the reconstruction--generation trade-off, improving both reconstruction quality and generative performance. Our experiments demonstrate that: (1) with only 8 additional queries and 3.9% extra computation, DecQ improves reconstruction over the frozen DINOv2-based RAE, increasing PSNR from 19.13 dB to 22.76 dB; and (2) for generative modeling, DecQ achieves 3.3$\times$ faster convergence than RAE, attaining an FID of 1.41 without guidance and 1.05 with guidance.

自编码器图像生成细节增强轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。