用可学习的令牌直接预测3D高斯坐标，提升重建精度与稳定性。
TokenGS: Decoupling 3D Gaussian Prediction from Pixels with Learnable Tokens
- 引入可学习的高斯令牌，解耦像素与3D高斯预测
- 在静态和动态场景上达到当前最优前馈重建效果
- 支持测试时优化，适合需要鲁棒性的3D重建应用
本文重新审视基于Transformer的前馈3D高斯喷溅（3DGS）预测中的若干关键设计。我们认为，将高斯均值作为沿相机射线的深度进行回归是次优的，因此提出仅使用自监督渲染损失直接回归3D均值坐标。该方法使模型从标准编码器结构转向编码器-解码器架构，引入可学习的高斯令牌，从而摆脱预测原型数对输入图像分辨率和视角数量的依赖。所提出的TokenGS方法在姿态噪声和多视角不一致情况下表现更鲁棒，且能在不破坏已学习先验的前提下，实现高效的测试时优化。在静态和动态场景上，TokenGS均取得当前最优的前馈重建性能，生成更规则的几何结构和更均衡的3DGS分布，并自然恢复静态-动态分解与场景运动等新兴属性。
原文摘要 · Abstract (English)
In this work, we revisit several key design choices of modern Transformer-based approaches for feed-forward 3D Gaussian Splatting (3DGS) prediction. We argue that the common practice of regressing Gaussian means as depths along camera rays is suboptimal, and instead propose to directly regress 3D mean coordinates using only a self-supervised rendering loss. This formulation allows us to move from the standard encoder-only design to an encoder-decoder architecture with learnable Gaussian tokens, thereby unbinding the number of predicted primitives from input image resolution and number of views. Our resulting method, TokenGS, demonstrates improved robustness to pose noise and multiview inconsistencies, while naturally supporting efficient test-time optimization in token space without degrading learned priors. TokenGS achieves state-of-the-art feed-forward reconstruction performance on both static and dynamic scenes, producing more regularized geometry and more balanced 3DGS distribution, while seamlessly recovering emergent scene attributes such as static-dynamic decomposition and scene flow.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。