arXiv:2608.15651cs.CV2026-08

用隐空间预测提升3D高斯点云的鲁棒性与泛化能力

Gaussian-JEPA: Joint-Embedding Predictive Learning for 3D Gaussian Splats

论文配图:Gaussian-JEPA: Joint-Embedding Predictive Learning for 3D Gaussian Splats
图 1 · 摘自论文原文
  • 通过隐空间预测替代属性重建,实现多尺度特征监督
  • 在重采样和部分观测下表现更稳定,保留更多实例信息
  • 适合需要泛化能力的3D表示学习任务,如分割与分类

3D高斯点云(3DGS)用各向异性基元联合编码几何与外观。固定预算编码器处理采样观测,同一物体可能呈现不同基元形态。现有自监督方法主要重建被掩码的高斯属性,依赖输入空间解码器且监督绑定单一采样结果。我们提出Gaussian-JEPA,从可见上下文预测被遮蔽的高斯令牌块表示。在线编码器处理上下文,共享指数移动平均编码器提供带梯度截断的多尺度目标特征。互补目标投影与特征空间锚定实现隐空间监督,无需重建高斯属性。我们在高斯重采样、部分观测及可渲染形状补全任务上评估特征表现,并迁移至部件分割与物体分类。相比匹配的重建预训练,Gaussian-JEPA在重采样输入下更一致,部分观测下保留更多信息,且对高斯补全提供更强冻结特征。结果支持隐空间预测作为可复用3D高斯表示的有效目标。代码见项目页:https://amazingren.github.io/Gaussian-JEPA/

原文摘要 · Abstract (English)

3D Gaussian Splatting (3DGS) represents 3D content with anisotropic primitives that jointly encode geometry and appearance. Fixed-budget encoders consume sampled observations of Gaussian assets, so the same object may be observed through different primitive realizations. Existing self-supervised methods mainly reconstruct masked Gaussian attributes, tying supervision to one sampled realization and requiring an input-space decoder. Latent prediction offers an alternative, but its application to Gaussian tokens requires targets that accommodate coupled attributes and heterogeneous spatial support. We introduce Gaussian-JEPA, which predicts representations of held-out Gaussian token blocks from visible context. An online encoder processes the context, while a shared exponential-moving-average encoder supplies stop-gradient features for multi-scale targets. Complementary target projections and feature-space grounding provide latent supervision without reconstructing Gaussian attributes. We evaluate the features under Gaussian resampling, partial observations, and renderable shape completion, together with transfer to part segmentation and object classification. Compared with matched reconstruction pretraining, Gaussian-JEPA is more consistent across resampled inputs, retains more instance information under partial observations, and provides stronger frozen features for Gaussian completion. These results support latent prediction as an effective objective for reusable 3D Gaussian representations. Code is on the project page (https://amazingren.github.io/Gaussian-JEPA/).

3D高斯自监督特征学习隐空间预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。