用正交算子让扩散模型更懂图像分块的上下文关系。
OPRO: Orthogonal Panel-Relative Operators for Panel-Aware In-Context Image Generation
- 在冻结位置编码上叠加可学习的分块专用正交算子
- 保持特征几何结构与同块内生成一致性
- 适合需要精准分块控制的图像编辑任务
我们提出一种参数高效的适配方法，用于预训练扩散变换器的分块感知上下文图像生成。核心思想是在骨干模型的冻结位置编码上组合可学习的、分块特异的正交算子。该设计具备两个理想属性：(1) 同构性，保持内部特征的几何结构；(2) 同块不变性，维持模型预训练的块内合成行为。通过受控实验，我们证明该方法的有效性不依赖特定位置编码设计，可在多种位置编码范式间泛化。通过实现有效的分块相对条件建模，该方法显著提升各类基于上下文图像的指令编辑流程，包括当前最先进方法。
原文摘要 · Abstract (English)
We introduce a parameter-efficient adaptation method for panel-aware in-context image generation with pre-trained diffusion transformers. The key idea is to compose learnable, panel-specific orthogonal operators onto the backbone's frozen positional encodings. This design provides two desirable properties: (1) isometry, which preserves the geometry of internal features, and (2) same-panel invariance, which maintains the model's pre-trained intra-panel synthesis behavior. Through controlled experiments, we demonstrate that the effectiveness of our adaptation method is not tied to a specific positional encoding design but generalizes across diverse positional encoding regimes. By enabling effective panel-relative conditioning, the proposed method consistently improves in-context image-based instructional editing pipelines, including state-of-the-art approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。