用三角形直接生成可模拟的3D场景,一次前传搞定。
TriSplat: Simulation-Ready Feed-Forward 3D Scene Reconstruction

- 用定向三角形表示场景,从图像直接预测点云和三角属性。
- 在RealEstate10K和DL3DV上重建更准确,且表面几何更真实。
- 输出可直接用于物理引擎,适合仿真与交互应用。
稀疏视角3D重建通常采用前馈式点阵网络,直接从图像预测显式几何体。然而现有方法多依赖高斯表示,仅间接暴露表面,下游仿真、物理推理或具身交互仍需昂贵的后处理步骤,违背了前馈设计初衷。该问题在无姿态设定下尤为突出,需联合估计场景结构与相机参数。本文提出TriSplat,一种以定向三角形为渲染基元的前馈重建网络,可从单次前传直接输出可用于仿真的网格场景。输入图像后,网络预测局部3D点图、三角属性、相机位姿及可选内参。不同于将三角朝向作为自由潜变量回归,本方法从预测点图构造几何法向量,经图像条件化法向头优化,并转化为稳定局部坐标系用于三角参数化。引入单法向初始化策略稳定训练初期,同时通过透明度与模糊度调度逐步锐化表面表示,实现直接网格提取。在RealEstate10K和DL3DV上的实验表明,该表示相比高斯前馈基线生成更符合几何结构的重建,同时保持竞争性新视角渲染质量。由于渲染基元本身即为表面三角形,输出可直接被物理引擎、碰撞检测器和标准渲染管线使用,无需转换,是真正实用的前馈式3D场景仿真就绪解决方案。
原文摘要 · Abstract (English)
Sparse-view 3D reconstruction is increasingly addressed with feed-forward splatting networks that predict explicit primitives directly from images. Yet most existing methods remain centered on Gaussian primitives and expose surfaces only indirectly: extracting a usable mesh for downstream simulation, physics reasoning, or embodied interaction still requires expensive post-hoc steps that break the feed-forward promise. This limitation is especially pronounced in pose-free settings, where scene structure and camera parameters must be estimated jointly from sparse observations. We present TriSplat, a feed-forward reconstruction network that represents scenes with oriented triangle primitives and directly exports simulation-ready mesh scenes from a single forward pass. Given input images, the network predicts local 3D point maps, triangle attributes, camera poses, and optional intrinsics. Rather than regressing triangle orientation as an unconstrained latent variable, our approach constructs geometry normals from the predicted point maps, refines them with an image-conditioned normal head, and converts them into stable local frames for triangle parameterization. A mono-normal bootstrap schedule further stabilizes early training, while opacity and blur scheduling progressively sharpens the learned surface representation for direct mesh extraction. Experiments on RealEstate10K and DL3DV show that this representation produces more geometry-faithful reconstructions than Gaussian feed-forward baselines while maintaining competitive novel-view rendering quality. Because the rendering primitives are themselves surface triangles, the output can be directly ingested by physics engines, collision detectors, and standard rendering pipelines without any conversion, making it a practical simulation-ready solution for feed-forward 3D scene reconstruction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。