无需精确相机位姿，用稀疏图像快速生成高质量3D高斯点云
FreeSplatter: Pose-free Gaussian Splatting for Sparse-view 3D Reconstruction
- 用轻量Transformer统一处理多视角图像，直接生成对齐的3D高斯点
- 在挑战性数据集上优于多个依赖位姿的大型重建模型
- 适合需要快速3D建模且无相机参数的场景，如内容创作
稀疏视角三维重建通常依赖精确的相机位姿，但从稀疏图像中获取这些参数仍具挑战。本文提出FreeSplatter，一种可扩展的前馈框架，能从未标定的稀疏视角图像中快速生成高质量3D高斯点云，并在数秒内估计相机参数。该方法采用精简的Transformer架构，通过自注意力模块实现多视图图像令牌间的信息交互，将它们解码为统一参考坐标系下的像素对齐3D高斯原语。这一表示支持高保真三维建模和利用现成求解器高效估计相机参数。我们开发了两种专用变体——面向物体级和场景级重建——并在大规模数据集上进行训练。令人瞩目的是，FreeSplatter在多个挑战性基准测试中显著优于多个依赖位姿的大型重建模型（LRMs），同时在相机位姿估计精度上达到或超越当前最优的无位姿方法MASt3R。除技术性能外，FreeSplatter还简化了文本/图像到3D内容的生成流程，免去相机位姿管理复杂性，实现卓越的视觉保真度。
原文摘要 · Abstract (English)
Sparse-view reconstruction models typically require precise camera poses, yet obtaining these parameters from sparse-view images remains challenging. We introduce FreeSplatter, a scalable feed-forward framework that generates high-quality 3D Gaussians from uncalibrated sparse-view images while estimating camera parameters within seconds. Our approach employs a streamlined transformer architecture where self-attention blocks facilitate information exchange among multi-view image tokens, decoding them into pixel-aligned 3D Gaussian primitives within a unified reference frame. This representation enables both high-fidelity 3D modeling and efficient camera parameter estimation using off-the-shelf solvers. We develop two specialized variants--for object-centric and scene-level reconstruction--trained on comprehensive datasets. Remarkably, FreeSplatter outperforms several pose-dependent Large Reconstruction Models (LRMs) by a notable margin while achieving comparable or even better pose estimation accuracy compared to state-of-the-art pose-free reconstruction approach MASt3R in challenging benchmarks. Beyond technical benchmarks, FreeSplatter streamlines text/image-to-3D content creation pipelines, eliminating the complexity of camera pose management while delivering exceptional visual fidelity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。