用额外视图信息指导稀疏视角3D高斯溅射修复,提升重建质量。
TRACE-GS: On-Policy Trajectory Distillation with Privileged Geometric Conditioning for Sparse-View 3DGS Restoration

- 通过教师模型在训练时提供带几何先验的轨迹监督
- 在多个数据集上实现稳定性能提升,泛化性好
- 适合需要高质量3D重建的稀疏视角场景应用
我们提出TRACE-GS,一种基于策略轨迹蒸馏的框架,在训练时利用额外视图提供的丰富几何信息作为特权条件,引导扩散先验对稀疏视角3D高斯溅射(3DGS)进行修复。现有方法在独立加噪状态上进行监督,无法覆盖推理过程中的真实状态。在稀疏视角下,初始几何约束不足导致去噪方向偏差,并沿生成路径累积。而TRACE-GS采用基于策略的轨迹蒸馏:教师模型以更多训练视图为条件,为稀疏视角学生模型在自身推理轨迹上的每一步提供目标,使去噪方向和跨视图响应在每个状态保持一致。该方法属于学习使用特权信息(LUPI)设置,部署时仅保留稀疏视角学生模型,其输出作为伪观测用于3DGS优化。据我们所知,这是首个将特权几何信息用于稀疏视角3DGS修复的在线策略监督方法,在多个数据集与稀疏视角设置中均取得一致提升和强泛化能力。
原文摘要 · Abstract (English)
We present TRACE-GS, an on-policy trajectory distillation framework that leverages privileged geometric conditioning at training time, thereby adapting a diffusion prior to sparse-view 3D Gaussian Splatting (3DGS) restoration. Rather than pursuing increasingly sophisticated restoration architectures, we identify a more fundamental limitation shared by existing diffusion-based approaches: supervision at independently noised states does not cover those reached during inference. In sparse-view 3DGS, under-constrained geometry biases denoising from the outset, and the resulting deviations compound along the rollout. TRACE-GS instead performs on-policy trajectory distillation: a teacher conditioned on richer geometry from additional training views supplies targets along the sparse-view student's own rollout, aligning denoising directions and cross-view responses at each visited state. This training-only geometry places TRACE-GS in the learning using privileged information (LUPI) setting. At deployment, only the sparse-view student is retained, and its restored renderings serve as pseudo-observations for 3DGS refinement. To the best of our knowledge, TRACE-GS is the first to derive on-policy supervision from privileged geometry for sparse-view 3DGS restoration, achieving consistent gains and strong generalization across datasets and sparse-view settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。