用随机扩散同步技术，在任意空间生成高质量图像，无需额外训练。
StochSync: Stochastic Diffusion Synchronization for Image Generation in Arbitrary Spaces
- 结合扩散同步与梯度优化思路，实现弱条件下的图像生成
- 360°全景图生成效果超越以往微调方法，细节更丰富
- 适合无图像条件的全景生成或3D纹理任务
我们提出一种零样本方法StochSync，利用预训练图像扩散模型在任意空间（如球面用于360°全景图、网格表面用于纹理）中生成图像。现有方法主要分为两类：扩散同步通过在不同投影空间联合反向扩散并同步目标空间，需充足条件才能生成高质量结果；得分蒸馏采样通过梯度下降逐步更新目标空间数据，虽增强一致性但常损失细节。本文首次揭示二者关联并区分差异，提出新方法StochSync，融合两者优势，可在弱条件情况下仍有效生成。实验表明，该方法在360°全景生成任务（无图像条件）中表现最佳，优于以往微调方法；在3D网格纹理任务（有深度条件）中结果也与当前最优方法相当。
原文摘要 · Abstract (English)
We propose a zero-shot method for generating images in arbitrary spaces (e.g., a sphere for 360° panoramas and a mesh surface for texture) using a pretrained image diffusion model. The zero-shot generation of various visual content using a pretrained image diffusion model has been explored mainly in two directions. First, Diffusion Synchronization-performing reverse diffusion processes jointly across different projected spaces while synchronizing them in the target space-generates high-quality outputs when enough conditioning is provided, but it struggles in its absence. Second, Score Distillation Sampling-gradually updating the target space data through gradient descent-results in better coherence but often lacks detail. In this paper, we reveal for the first time the interconnection between these two methods while highlighting their differences. To this end, we propose StochSync, a novel approach that combines the strengths of both, enabling effective performance with weak conditioning. Our experiments demonstrate that StochSync provides the best performance in 360° panorama generation (where image conditioning is not given), outperforming previous finetuning-based methods, and also delivers comparable results in 3D mesh texturing (where depth conditioning is provided) with previous methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。