让图像生成更像人脑:分层渐进,细节更准。
SSG: Scaled Spatial Guidance for Multi-Scale Visual Autoregressive Generation
- 用频域方法提取高层语义残差,指导生成顺序
- 无需训练,在推理时提升图像质量和多样性
- 适配多种视觉自回归模型,效率高延迟低
视觉自回归(VAR)模型通过逐尺度预测生成图像,天然实现从粗到细的快速高保真合成,贴近人类感知。但实际推理中,因模型容量有限和误差累积,层级结构容易偏离。本文从信息论视角重新审视此问题,发现若确保每级生成包含前级未解释的高频内容,可缓解训练-推理差异。为此提出无训练、推理时使用的缩放空间引导(SSG),通过频率感知的离散空间增强(DSE)方法,精准提取并强调目标高频语义残差,有效引导生成保持预期层次结构,同时维持全局一致性。SSG适用于基于离散视觉标记的各类VAR模型,无论标记化方式或条件模态如何。实验表明,其在保持低延迟的前提下,稳定提升图像保真度与多样性,揭示了分层生成中未被充分挖掘的效率潜力。代码已开源。
原文摘要 · Abstract (English)
Visual autoregressive (VAR) models generate images through next-scale prediction, naturally achieving coarse-to-fine, fast, high-fidelity synthesis mirroring human perception. In practice, this hierarchy can drift at inference time, as limited capacity and accumulated error cause the model to deviate from its coarse-to-fine nature. We revisit this limitation from an information-theoretic perspective and deduce that ensuring each scale contributes high-frequency content not explained by earlier scales mitigates the train-inference discrepancy. With this insight, we propose Scaled Spatial Guidance (SSG), training-free, inference-time guidance that steers generation toward the intended hierarchy while maintaining global coherence. SSG emphasizes target high-frequency signals, defined as the semantic residual, isolated from a coarser prior. To obtain this prior, we leverage a principled frequency-domain procedure, Discrete Spatial Enhancement (DSE), which is devised to sharpen and better isolate the semantic residual through frequency-aware construction. SSG applies broadly across VAR models leveraging discrete visual tokens, regardless of tokenization design or conditioning modality. Experiments demonstrate SSG yields consistent gains in fidelity and diversity while preserving low latency, revealing untapped efficiency in coarse-to-fine image generation. Code is available at https://github.com/Youngwoo-git/SSG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。