让模型学会关注更相关视角，提升少样本图像生成质量
Pay Attention to Where You Looked
- 根据视角与目标的几何距离和角度动态调整输入视图权重
- 在少样本场景下，生成图像的保真度和细节显著提升
- 适用于各类生成模型，适合做3D视觉重建的研究者
新型视角合成（NVS）借助生成建模实现了逼真图像生成。在少样本NVS中，现有方法通常假设所有输入视角对目标视角的重要性相同，导致效果不佳。本文提出一种相机加权机制，根据源视角与目标视角的相关性动态调整其重要性。设计了两种方案：一种基于欧氏距离和角度差的确定性加权；另一种采用交叉注意力学习视角权重。此外，模型可在此机制下进一步训练，以优化对视角相关性的理解，提升合成质量。该机制具有良好的可适配性，能集成到多种NVS算法中，显著增强生成高保真新视角的能力。实验表明，自适应视角加权能有效提高合成精度与真实感，为改善NVS提供了新方向。
原文摘要 · Abstract (English)
Novel view synthesis (NVS) has advanced with generative modeling, enabling photorealistic image generation. In few-shot NVS, where only a few input views are available, existing methods often assume equal importance for all input views relative to the target, leading to suboptimal results. We address this limitation by introducing a camera-weighting mechanism that adjusts the importance of source views based on their relevance to the target. We propose two approaches: a deterministic weighting scheme leveraging geometric properties like Euclidean distance and angular differences, and a cross-attention-based learning scheme that optimizes view weighting. Additionally, models can be further trained with our camera-weighting scheme to refine their understanding of view relevance and enhance synthesis quality. This mechanism is adaptable and can be integrated into various NVS algorithms, improving their ability to synthesize high-quality novel views. Our results demonstrate that adaptive view weighting enhances accuracy and realism, offering a promising direction for improving NVS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。