用快速回归引导扩散模型，实现高质量且低延迟的新视角生成。
Sphinx: Efficiently Serving Novel View Synthesis using Regression-Guided Selective Refinement
- 用回归模型快速初始化，减少扩散模型的去噪负担。
- 通过自适应噪声调度，仅对不确定区域精细优化，平均提速1.8倍。
- 无需训练即可部署，适合动态变化的实时推理场景。
新视角生成（NVS）旨在从非原始输入视点生成场景的新图像。基于扩散模型的NVS虽能生成高质量、时序一致的图像，但计算开销巨大；而基于回归的NVS虽计算成本低，但生成质量较差，导致高保真、高效推理框架的设计仍是一大挑战。为此，我们提出Sphinx——一种无需训练的混合推理框架，可在显著降低计算量的前提下实现接近扩散模型的生成质量。Sphinx利用回归模型进行快速初始化，引导并减轻扩散模型的去噪任务，并结合选择性精炼与自适应噪声调度，将更多计算资源分配至不确定性高的区域和帧。该设计支持灵活权衡性能与质量，可动态适配不同延迟和保真度需求。评估显示，Sphinx相比纯扩散模型推理平均提速1.8倍，感知退化低于5%，在质量与延迟之间建立了新的帕累托前沿。
原文摘要 · Abstract (English)
Novel View Synthesis (NVS) is the task of generating new images of a scene from viewpoints that were not part of the original input. Diffusion-based NVS can generate high-quality, temporally consistent images, however, remains computationally prohibitive. Conversely, regression-based NVS offers suboptimal generation quality despite requiring significantly lower compute; leaving the design objective of a high-quality, inference-efficient NVS framework an open challenge. To close this critical gap, we present Sphinx, a training-free hybrid inference framework that achieves diffusion-level fidelity at a significantly lower compute. Sphinx proposes to use regression-based fast initialization to guide and reduce the denoising workload for the diffusion model. Additionally, it integrates selective refinement with adaptive noise scheduling, allowing more compute to uncertain regions and frames. This enables Sphinx to provide flexible navigation of the performance-quality trade-off, allowing adaptation to latency and fidelity requirements for dynamically changing inference scenarios. Our evaluation shows that Sphinx achieves an average 1.8x speedup over diffusion model inference with negligible perceptual degradation of less than 5%, establishing a new Pareto frontier between quality and latency in NVS serving.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。