无需训练即可实现自回归图像编辑的结构保持方法
Anchor Token Matching: Implicit Structure Locking for Training-free AR Image Editing
- 通过锚点标记匹配动态对齐注意力模式
- 在不增加训练的情况下保持对象布局与全局一致性
- 适合追求高效、灵活图像编辑的研究者与开发者
文本到图像生成在扩散模型推动下取得突破性进展,支持高保真合成与基于交叉注意力操控的精确编辑。近期自回归(AR)模型重新成为有力替代方案,利用逐词生成逼近扩散模型性能。然而,现有为扩散模型设计的编辑技术难以直接迁移到AR模型,因其结构控制机制存在根本差异。具体表现为注意力图空间贫乏及编辑过程中结构误差的序列累积,导致物体布局失真与整体一致性破坏。本文提出首个无需训练的AR视觉模型编辑策略——隐式结构锁定(ISLock)。该方法不依赖显式注意力操作或微调,而是通过锚点标记匹配(ATM)协议,动态将自注意力模式与参考图像对齐,隐式维持潜在空间中的结构蓝图。实验表明,ISLock在无额外训练条件下实现高质量、结构一致的编辑,性能优于或媲美传统方法。研究为高效、灵活的AR图像编辑开辟新路径,进一步缩小扩散与自回归生成模型间的性能差距。代码将公开于 https://github.com/hutaiHang/ATM。
原文摘要 · Abstract (English)
Text-to-image generation has seen groundbreaking advancements with diffusion models, enabling high-fidelity synthesis and precise image editing through cross-attention manipulation. Recently, autoregressive (AR) models have re-emerged as powerful alternatives, leveraging next-token generation to match diffusion models. However, existing editing techniques designed for diffusion models fail to translate directly to AR models due to fundamental differences in structural control. Specifically, AR models suffer from spatial poverty of attention maps and sequential accumulation of structural errors during image editing, which disrupt object layouts and global consistency. In this work, we introduce Implicit Structure Locking (ISLock), the first training-free editing strategy for AR visual models. Rather than relying on explicit attention manipulation or fine-tuning, ISLock preserves structural blueprints by dynamically aligning self-attention patterns with reference images through the Anchor Token Matching (ATM) protocol. By implicitly enforcing structural consistency in latent space, our method ISLock enables structure-aware editing while maintaining generative autonomy. Extensive experiments demonstrate that ISLock achieves high-quality, structure-consistent edits without additional training and is superior or comparable to conventional editing techniques. Our findings pioneer the way for efficient and flexible AR-based image editing, further bridging the performance gap between diffusion and autoregressive generative models. The code will be publicly available at https://github.com/hutaiHang/ATM
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。