arXiv:2601.20857cs.CV2026-01被引 2

不用微调扩散模型,就能提升3D高斯点云的渲染质量。

FreeFix: Boosting 3D Gaussian Splatting via Fine-Tuning-Free Diffusion Models

  • 用预训练图像扩散模型做2D-3D交替优化,避免微调成本。
  • 在多个数据集上实现与微调方法相当甚至更优的多帧一致性。
  • 通过像素级置信度掩码精准定位需修复区域,适合新视角生成任务。

神经辐射场与3D高斯点云已推动新视角合成发展,但仍依赖密集输入,且在外推视角下性能下降。现有方法利用生成模型(如扩散模型)提供额外监督,但面临泛化性与保真度的权衡:微调扩散模型可提升保真度,但易过拟合;无微调方法保持泛化性,却常导致保真度不足。本文提出FreeFix,一种无需微调的方案,通过预训练图像扩散模型增强外推渲染效果。我们设计了交错式2D-3D优化策略,证明图像扩散模型可在不使用昂贵视频扩散模型的前提下实现一致优化。此外,我们深入分析2D优化的引导信号,提出像素级置信度掩码以识别不确定区域并针对性改进。跨多个数据集的实验表明,FreeFix显著提升多帧一致性,性能达到或超越微调方法,同时保持强泛化能力。

原文摘要 · Abstract (English)

Neural Radiance Fields and 3D Gaussian Splatting have advanced novel view synthesis, yet still rely on dense inputs and often degrade at extrapolated views. Recent approaches leverage generative models, such as diffusion models, to provide additional supervision, but face a trade-off between generalization and fidelity: fine-tuning diffusion models for artifact removal improves fidelity but risks overfitting, while fine-tuning-free methods preserve generalization but often yield lower fidelity. We introduce FreeFix, a fine-tuning-free approach that pushes the boundary of this trade-off by enhancing extrapolated rendering with pretrained image diffusion models. We present an interleaved 2D-3D refinement strategy, showing that image diffusion models can be leveraged for consistent refinement without relying on costly video diffusion models. Furthermore, we take a closer look at the guidance signal for 2D refinement and propose a per-pixel confidence mask to identify uncertain regions for targeted improvement. Experiments across multiple datasets show that FreeFix improves multi-frame consistency and achieves performance comparable to or surpassing fine-tuning-based methods, while retaining strong generalization ability.

3D重建扩散模型渲染优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。