arXiv:2605.20808cs.CV2026-05被引 1

通过空间自相似性对齐,实现超分辨率图像生成的结构与细节统一。

Spatial Gram Alignment for Ultra-High-Resolution Image Synthesis

论文配图:Spatial Gram Alignment for Ultra-High-Resolution Image Synthesis
图 1 · 摘自论文原文
  • 用视觉基础模型的内在自相似性约束生成特征,不破坏预训练潜空间
  • 在1024×1024分辨率下实现结构更稳、细节更锐的文本到图像生成
  • 适用于已有扩散模型的无损增强,适合高精度图像生成研究者

现代超高清图像生成严重依赖大规模预训练潜空间扩散模型(LDM)的强大生成能力。尽管近期表示对齐方法通过将基础模型(如SAM或DINO)的视觉先验蒸馏至生成潜特征中已证明有效,但将这些方法扩展到极高分辨率的预训练LDM时,暴露出学习能力与生成保真度之间的关键冲突。具体而言,强制进行逐块特征蒸馏会扰动预训练的潜流形,导致生成质量下降。为此,我们提出空间格拉姆对齐(SGA),一种新框架,在利用视觉基础模型表示先验的同时,保持LDM的原生生成能力。超越受限的直接对齐,SGA通过对齐生成特征与基础先验的内部自相似性,施加非侵入性空间约束。该约束有效建立宏观结构一致性,而原生生成目标则保留了原始LDM固有的微观像素级保真度。值得注意的是,这一灵活策略可无缝集成于预训练LDM中的中间扩散特征与VAE潜空间。大量实验表明,SGA在超高清文本到图像合成任务中达到当前最优性能,成功调和了全局结构完整性和细粒度视觉细节之间的矛盾。代码已开源:https://github.com/zhang0jhon/SGA。

原文摘要 · Abstract (English)

Modern ultra-high-resolution image synthesis relies heavily on the robust generative capacity of large-scale pre-trained Latent Diffusion Models (LDMs). While recent representation alignment methods have proven effective by distilling visual priors from foundation models (e.g., SAM or DINO) into generative latent features, scaling these approaches to pre-trained LDMs at extreme resolutions exposes a critical learnability-fidelity conflict. Specifically, forcing direct patch-wise feature distillation inherently perturbs the pre-trained latent manifold, ultimately leading to generation degradation. To address this bottleneck, we propose Spatial Gram Alignment (SGA), a novel framework that explicitly leverages the representation priors of vision foundation models while preserving the native generative capacity of LDMs. Moving beyond restrictive direct alignment, SGA imposes a non-invasive spatial constraint by aligning the internal self-similarities of the generative features with those of the foundation priors. This spatial constraint effectively establishes macroscopic structural coherence, while the native generative objectives retain the microscopic pixel-level fidelity inherent to the original LDMs. Notably, this versatile strategy integrates seamlessly across both intermediate diffusion features and VAE latents within pre-trained LDMs. Extensive experiments demonstrate that SGA achieves state-of-the-art performance for ultra-high-resolution text-to-image synthesis, yielding an effective reconciliation between global structural integrity and fine-grained visual details. Code is available at https://github.com/zhang0jhon/SGA.

图像生成扩散模型超分辨率特征对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。