用频域注意力和坐标噪声生成多视角一致图像，提升视觉连贯性。
Multi-view Image Diffusion via Coordinate Noise and Fourier Attention
- 基于傅里叶注意力机制捕捉非重叠区域特征，增强全局对齐。
- 通过坐标与深度图初始化噪声，建立跨视角的噪声相关性。
- 在多个指标上超越当前最佳方法，适合多视角生成任务研究者。
近期，基于扩散模型的文本到图像生成在保真度和泛化能力上相较以往基线取得显著进展。然而，从提示词生成整体一致的多视角图像仍是重要且具挑战性的任务。为此，我们提出一种新的扩散过程，结合新型注意力机制、噪声初始化技术及交叉注意力损失。该傅里叶注意力模块聚焦于生成场景中非重叠区域的特征，以更好对齐整体外观。噪声初始化技术融合了由像素坐标和深度图导出的共享噪声与低空间频率信息，诱导跨视角的噪声相关性。交叉注意力损失进一步对齐场景中具有相同提示词的特征。我们的方法在多个定量指标上优于现有最先进方法，定性结果也更优。
原文摘要 · Abstract (English)
Recently, text-to-image generation with diffusion models has made significant advancements in both higher fidelity and generalization capabilities compared to previous baselines. However, generating holistic multi-view consistent images from prompts still remains an important and challenging task. To address this challenge, we propose a diffusion process that attends to time-dependent spatial frequencies of features with a novel attention mechanism as well as novel noise initialization technique and cross-attention loss. This Fourier-based attention block focuses on features from non-overlapping regions of the generated scene in order to better align the global appearance. Our noise initialization technique incorporates shared noise and low spatial frequency information derived from pixel coordinates and depth maps to induce noise correlations across views. The cross-attention loss further aligns features sharing the same prompt across the scene. Our technique improves SOTA on several quantitative metrics with qualitatively better results when compared to other state-of-the-art approaches for multi-view consistency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。