让扩散模型生成更好图像，关键在学怎么造高质量初始噪声
Learning High-Quality Initial Noise for Single-View Synthesis with Diffusion Models
- 用离散欧拉反演法把图像语义注入随机噪声，生成成对数据集
- 设计编码器-解码器网络，直接将普通噪声转为高质量噪声
- 可插件式接入多种单视图合成模型，提升多数据集表现
基于扩散模型的单视图新视角合成（NVS）近期受到关注，因其能从单张图像和相机位姿条件生成一系列新视角图像。研究发现，扩散模型中某些高质量初始噪声模式比其他噪声生成效果更好。然而，目前缺乏专门用于学习此类高质量噪声的框架。为从随机高斯噪声中获得高质量噪声，本文提出两项贡献：首先，设计离散欧拉反演方法，将图像语义信息注入随机噪声，构建随机噪声与高质量噪声的成对数据集；其次，提出一种基于编码器-解码器网络（EDN）的学习框架，可直接将随机噪声转换为高质量噪声。实验表明，所提EDN可无缝嵌入多种NVS模型（如SV3D和MV-Adapter），在多个数据集上实现显著性能提升。代码已公开于：https://github.com/zhihao0512/EDN。
原文摘要 · Abstract (English)
Single-view novel view synthesis (NVS) models based on diffusion models have recently attracted increasing attention, as they can generate a series of novel view images from a single image prompt and camera pose information as conditions. It has been observed that in diffusion models, certain high-quality initial noise patterns lead to better generation results than others. However, there remains a lack of dedicated learning frameworks that enable NVS models to learn such high-quality noise. To obtain high-quality initial noise from random Gaussian noise, we make the following contributions. First, we design a discretized Euler inversion method to inject image semantic information into random noise, thereby constructing paired datasets of random and high-quality noise. Second, we propose a learning framework based on an encoder-decoder network (EDN) that directly transforms random noise into high-quality noise. Experiments demonstrate that the proposed EDN can be seamlessly plugged into various NVS models, such as SV3D and MV-Adapter, achieving significant performance improvements across multiple datasets. Code is available at: https://github.com/zhihao0512/EDN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。