用空间有序的Transformer提升3D高斯点渲染速度与质量
Z-Order Transformer for Feed-Forward Gaussian Splatting
- 通过Z序排列将无序高斯点转为有序序列，实现高效稀疏注意力
- 仅用少量高斯点即可达成高质量新视角合成，比传统方法快数倍
- 适合需要实时渲染的3D重建场景，如AR/VR和机器人视觉
近期3D高斯点绘制（3DGS）在逼真新视角合成方面取得显著进展。然而，传统3DGS依赖缓慢的迭代优化过程，限制了其在实时场景中的应用。为突破此瓶颈，现有前馈方法尝试直接从图像预测高斯属性，但常面临高斯基元冗余和渲染质量不足的问题。本文提出一种专为前馈3DGS设计的Transformer架构。关键洞察在于：通过Z序策略将无结构的高斯点集组织为具有空间连贯性的序列，可有效利用稀疏注意力捕捉高斯点间的空间与语义关系。同时，该策略自适应抑制冗余，保留关键结构细节。使Transformer能在单次前向传播中高效建模上下文、压缩高斯基元并预测属性。大量实验表明，本方法以更少的高斯点实现快速且高质量的新视角合成。
原文摘要 · Abstract (English)
Recent advances in 3D Gaussian Splatting (3DGS) have enabled significant progress in photorealistic novel view synthesis. However, traditional 3DGS relies on a slow, iterative optimization process, which limits its use in scenarios demanding real-time results. To overcome this bottleneck, recent feed-forward methods aim to predict Gaussian attributes directly from images, but they often struggle with the redundancy of Gaussian primitives and rendering quality. In this work, we introduce a transformer-based architecture specifically designed for feed-forward Gaussian Splatting. Our key insight is that spatial and semantic relationships among Gaussians can be effectively captured through a sparse attention mechanism, enabled by a Z-order strategy that organizes the unstructured Gaussian set into a spatially coherent sequence. Furthermore, we incorporate this Z-order strategy to adaptively suppress redundancy while preserving critical structural details. This allows the transformer to efficiently model context, compress Gaussian primitives, and predict Gaussian attributes in a single forward pass. Comprehensive experiments demonstrate that our method achieves fast and high-quality novel view synthesis with fewer Gaussian primitives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。