用单图生成一致新视角，无需额外模块
WAVE: Warp-Based View Guidance for Consistent Novel View Synthesis Using a Single Image
- 通过视图引导的扭曲实现注意力自适应调整
- 在多个扩散模型上提升视角一致性表现
- 训练免费，适合快速部署到现有模型
从单张图像生成高质量新视角需要保持不同视角间的结构一致性。尽管扩散模型推动了新视角合成的发展，但其仍难以维持跨视角的空间连续性。已有方法将扩散模型与3D模型结合以解决此问题，但因复杂的多步流程而效率低下。本文提出一种无需额外模块的新视角一致性生成方法，利用扩散模型本身，通过视图引导的扭曲实现训练无关的注意力自适应调节和噪声重初始化，确保视角一致性。通过为新视角数据集设计的综合评估框架，实验表明该方法在多种扩散模型上均显著提升了视角一致性，展现了广泛适用性。
原文摘要 · Abstract (English)
Generating high-quality novel views of a scene from a single image requires maintaining structural coherence across different views, referred to as view consistency. While diffusion models have driven advancements in novel view synthesis, they still struggle to preserve spatial continuity across views. Diffusion models have been combined with 3D models to address the issue, but such approaches lack efficiency due to their complex multi-step pipelines. This paper proposes a novel view-consistent image generation method which utilizes diffusion models without additional modules. Our key idea is to enhance diffusion models with a training-free method that enables adaptive attention manipulation and noise reinitialization by leveraging view-guided warping to ensure view consistency. Through our comprehensive metric framework suitable for novel-view datasets, we show that our method improves view consistency across various diffusion models, demonstrating its broader applicability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。