arXiv:2606.13652cs.CVcs.GR2026-06

提出可生成完整3D几何并保持像素对齐的新方法,突破可见表面限制。

World Tracing: Generative Pixel-Aligned Geometry Beyond the Visible

论文配图:World Tracing: Generative Pixel-Aligned Geometry Beyond the Visible
图 1 · 摘自论文原文
  • 为每个输入像素预测多层3D点,从可见面到被遮挡面逐层延伸
  • 在物体、场景和动态数据上同时提升可见面重建与遮挡部分生成效果
  • 保留2D-3D对应关系,支持文本编辑和视图合成等应用

图像转3D方法常在真实性和完整性间权衡:深度估计算法锚定于输入像素,但仅能预测可见表面;而图像转3D模型虽能生成完整形状,却常与输入像素错位。本文提出「世界追踪」(World Tracing),一种生成式像素对齐的几何表示,既能预测与观测像素对齐的3D点,又能完成可见表面之外的几何结构。对于每个输入像素,该方法预测一个按相机空间排序的3D点堆叠序列,第一层为可见表面,后续层表示从前到后与遮挡表面的交点。我们基于此设计了世界追踪扩散变换器(WT-DiT),将多个几何层视为独立去噪令牌,通过分解注意力与全局注意力耦合。模型采用像素空间流匹配训练,并结合混合噪声调度,平衡可见表面重建与遮挡几何生成。在物体、场景及动态基准测试中,该方法在可见表面重建与完整几何生成上均优于深度预测器与图像转3D生成器。此外,其保留2D到3D的对应关系,支持文本驱动的3D场景编辑、几何条件的新增视角视频合成,并可无需训练直接集成至带纹理网格生成器。

原文摘要 · Abstract (English)

Image-to-3D methods often trade off faithfulness and completeness: depth estimators are anchored to input pixels but stop at the visible surface, while image-to-3D models generate complete shapes that are often misaligned with the input. We introduce World Tracing, a generative pixel-aligned geometry representation that predicts 3D points aligned with observed pixels while completing geometry beyond the visible surface. For each input pixel, World Tracing predicts an ordered stack of camera-space 3D points, where the first layer represents the visible surface and subsequent layers represent front-to-back intersections with occluded surfaces. We instantiate this representation with a world-tracing diffusion transformer, WT-DiT, which treats multiple geometry layers as separate denoising tokens coupled through factorized and global attention. WT-DiT is trained with pixel-space flow matching and a mixed noise schedule that balances visible-surface reconstruction with occluded-geometry generation. World Tracing achieves strong performance on visible-surface reconstruction and complete geometry generation across object, scene, and dynamic benchmarks, outperforming both depth predictors and image-to-3D generators. It also preserves 2D-to-3D correspondence, enabling text-driven 3D scene editing, geometry-conditioned novel-view video synthesis, and training-free integration with textured-mesh generators.

3D生成像素对齐扩散模型几何推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。