解决视觉自回归模型高分辨率生成时的重复与失真问题
ExtraVAR: Stage-Aware RoPE Remapping for Resolution Extrapolation in Visual Autoregressive Models

- 按阶段重新映射旋转位置编码,抑制生成过程中的频率失配
- 在256×256到1024×1024分辨率上均实现更清晰的细节和结构
- 适合需要高分辨率图像生成的研究者与应用开发者
视觉自回归(VAR)模型已成为图像生成的新范式,但其固定训练分辨率限制了高分辨率直接生成。将大语言模型或扩散模型的分辨率外推方法直接迁移至VAR会引发全局重复、局部重复和细节退化三种典型失败模式。我们发现这些问题是由于各生成阶段的主导旋转位置编码(RoPE)频段错配所致。为此,提出阶段感知的RoPE重映射策略,为每个频段分配阶段特异性重映规则,有效抑制三类故障。进一步观察到,随着分辨率提升,注意力呈现系统性分散。现有方法依赖预设缩放因子,无法自适应目标分辨率,也难以准确捕捉分散程度。因此,提出基于熵的自适应注意力校准:通过分辨率无关的归一化熵量化分散程度,导出闭式每头缩放因子,使外推后注意力熵与训练阶段对齐。大量实验表明,本方法在结构一致性和细节保真度上均优于现有外推方法。代码已开源。
原文摘要 · Abstract (English)
Visual Autoregressive (VAR) models have emerged as a strong alternative to diffusion for image synthesis, yet their fixed training resolution prevents direct generation at higher resolutions. Naively transferring training-free extrapolation methods from LLMs or diffusion models to VAR yields three characteristic failure modes: global repetition, local repetition, and detail degradation. We trace them to a unified band-stage mismatch: VAR generates images in a coarse-to-fine, scale-wise process where each stage is driven by a distinct dominant RoPE frequency band, and each failure mode emerges when the dominant band of a particular stage is disrupted. Building on this insight, we propose Stage-Aware RoPE Remapping, a training-free strategy that assigns each frequency band a stage-specific remapping rule, jointly suppressing all three failure modes. We further observe that attention becomes systematically dispersed as the image resolution increases. Existing methods typically depend on predefined attention scaling factors, which are neither adaptive to the target resolution nor capable of faithfully capturing the actual extent of attention dispersion. We therefore propose Entropy-Driven Adaptive Attention Calibration, which quantifies dispersion via a resolution-invariant normalized entropy and yields a closed-form per-head scaling factor that realigns the extrapolated-resolution attention entropy with its training-resolution counterpart. Extensive experiments show that our method consistently outperforms prior resolution-extrapolation methods in both structural coherence and fine-detail fidelity. Our code is available at https://github.com/feihongyan1/ExtraVAR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。