arXiv:2511.14063cs.CV2025-11被引 11

提升自回归模型的语义条件生成能力,让图像生成更贴合指令。

Semantic Context Matters: Improving Conditioning for Autoregressive Models

  • 用压缩语义前缀和语义对齐引导增强生成过程中的条件控制。
  • 在指令编辑与可控生成任务中,视觉质量与语义对齐显著优于现有方法。
  • 适用于多种自回归生成范式,代码开源,适合图像生成与编辑研究者。

自回归(AR)模型在图像生成中展现出强大潜力,相比基于扩散的方法具有更好的可扩展性和多模态系统集成优势。然而,将其推广到通用图像编辑仍面临条件控制弱、效率低的问题,常导致指令遵循差和视觉伪影。为此,我们提出SCAR——一种面向自回归模型的语义上下文驱动方法。SCAR引入两个关键组件:压缩语义预填充,将高层语义编码为紧凑高效的前缀;以及语义对齐引导,在自回归解码过程中对齐最后的视觉隐藏状态与目标语义,以提升指令保真度。与解码阶段注入方法不同,SCAR在向量量化预填充的灵活性基础上,克服了其语义表达局限与高成本问题。该方法可泛化至下一令牌与下一集合两种自回归范式,仅需最小架构改动。在指令编辑与可控生成基准测试中,SCAR在视觉保真度和语义对齐方面均优于现有自回归方法,同时保持可控性。代码将公开于https://github.com/AMAP-ML/SCAR。

原文摘要 · Abstract (English)

Recently, autoregressive (AR) models have shown strong potential in image generation, offering better scalability and easier integration with unified multi-modal systems compared to diffusion-based methods. However, extending AR models to general image editing remains challenging due to weak and inefficient conditioning, often leading to poor instruction adherence and visual artifacts. To address this, we propose SCAR, a Semantic-Context-driven method for Autoregressive models. SCAR introduces two key components: Compressed Semantic Prefilling, which encodes high-level semantics into a compact and efficient prefix, and Semantic Alignment Guidance, which aligns the last visual hidden states with target semantics during autoregressive decoding to enhance instruction fidelity. Unlike decoding-stage injection methods, SCAR builds upon the flexibility and generality of vector-quantized-based prefilling while overcoming its semantic limitations and high cost. It generalizes across both next-token and next-set AR paradigms with minimal architectural changes. SCAR achieves superior visual fidelity and semantic alignment on both instruction editing and controllable generation benchmarks, outperforming prior AR-based methods while maintaining controllability. Code will be released at https://github.com/AMAP-ML/SCAR.

自回归生成图像编辑语义对齐条件控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。