arXiv:2505.12274cs.CV2025-05被引 7

用统一序列生成多条件图像,控制更灵活。

Context-Aware Autoregressive Models for Multi-Conditional Image Generation

  • 将多种条件直接嵌入令牌序列,保持语义清晰。
  • 混合位置编码提升空间对齐与条件区分能力。
  • 无需微调即可支持任意条件组合,适合可控生成场景。

自回归变换器最近在图像生成质量与效率上已达到与顶级扩散模型相当的水平。与扩散架构不同,自回归模型可自然地将任意模态整合到单一、统一的令牌序列中,为多条件图像生成任务提供简洁解决方案。本文提出名为 ContextAR 的灵活高效框架,将多种条件(如 Canny 边缘图、深度图、姿态)直接嵌入令牌序列,保留模态特异性语义。为在保持空间对齐的同时增强不同条件类型的区分性,引入融合旋转位置编码与可学习位置编码的混合位置编码。设计了条件感知注意力机制,在降低计算复杂度的同时保留条件内有效感知。无需任何微调,ContextAR 在推理阶段即可支持任意条件组合。实验表明,该方法具备强大的可控性与通用性,在多种多条件驱动场景下性能媲美现有基于扩散模型的方法,并接近现有自回归基线表现。

原文摘要 · Abstract (English)

Autoregressive transformers have recently shown impressive image generation quality and efficiency on par with state-of-the-art diffusion models. Unlike diffusion architectures, autoregressive models can naturally incorporate arbitrary modalities into a single, unified token sequence--offering a concise solution for multi-conditional image generation tasks. In this work, we propose $\textbf{ContextAR}$, a flexible and effective framework for multi-conditional image generation. ContextAR embeds diverse conditions (e.g., canny edges, depth maps, poses) directly into the token sequence, preserving modality-specific semantics. To maintain spatial alignment while enhancing discrimination among different condition types, we introduce hybrid positional encodings that fuse Rotary Position Embedding with Learnable Positional Embedding. We design Conditional Context-aware Attention to reduces computational complexity while preserving effective intra-condition perception. Without any fine-tuning, ContextAR supports arbitrary combinations of conditions during inference time. Experimental results demonstrate the powerful controllability and versatility of our approach, and show that the competitive perpormance than diffusion-based multi-conditional control approaches the existing autoregressive baseline across diverse multi-condition driven scenarios. Project page: $\href{https://context-ar.github.io/}{https://context-ar.github.io/.}$

图像生成自回归模型多条件控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。