通过提升图像对称性,解决生成模型任务冲突问题。
Equivariant Image Modeling
- 采用列式分块与窗口因果注意力增强平移对称性
- 256x256下性能媲美顶尖自回归模型,资源更少
- 适合追求高效生成与零样本泛化的研究者
当前生成模型如自回归和扩散模型将高维数据分布学习分解为多个子任务,但联合优化时存在内在冲突,现有方法无法在不牺牲效率或可扩展性的前提下解决。本文提出一种新型等变图像建模框架,利用自然视觉信号的平移不变性,从本质上对齐各子任务的优化目标。方法包括:(1) 列式标记化,增强水平方向的平移对称性;(2) 窗口因果注意力,确保位置间上下文关系一致。在256x256分辨率的类别条件ImageNet生成任务上,该方法性能接近最先进自回归模型,同时计算开销更低。系统分析表明,增强等变性显著降低任务间冲突,大幅提升零样本泛化能力,并支持超长图像合成。本工作首次实现生成建模中的任务对齐分解,为高效参数共享与无冲突优化提供新思路。代码与模型已公开于https://github.com/drx-code/EquivariantModeling。
原文摘要 · Abstract (English)
Current generative models, such as autoregressive and diffusion approaches, decompose high-dimensional data distribution learning into a series of simpler subtasks. However, inherent conflicts arise during the joint optimization of these subtasks, and existing solutions fail to resolve such conflicts without sacrificing efficiency or scalability. We propose a novel equivariant image modeling framework that inherently aligns optimization targets across subtasks by leveraging the translation invariance of natural visual signals. Our method introduces (1) column-wise tokenization which enhances translational symmetry along the horizontal axis, and (2) windowed causal attention which enforces consistent contextual relationships across positions. Evaluated on class-conditioned ImageNet generation at 256x256 resolution, our approach achieves performance comparable to state-of-the-art AR models while using fewer computational resources. Systematic analysis demonstrates that enhanced equivariance reduces inter-task conflicts, significantly improving zero-shot generalization and enabling ultra-long image synthesis. This work establishes the first framework for task-aligned decomposition in generative modeling, offering insights into efficient parameter sharing and conflict-free optimization. The code and models are publicly available at https://github.com/drx-code/EquivariantModeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。