arXiv:2606.11314cs.CVcs.GR2026-06

TRON融合高斯射线追踪与神经渲染,实现真实场景的实时可控编辑。

TRON: Tracing Rays to Orchestrate a Neural Renderer for 3D Gaussian Reconstructions

论文配图:TRON: Tracing Rays to Orchestrate a Neural Renderer for 3D Gaussian Reconstructions
图 1 · 摘自论文原文
  • 用学习到的逆渲染先验约束高斯场材质,提升物理准确性
  • 在210万帧合成与真实数据上训练,支持动态光照与物体插入
  • 兼具快速生成与精细编辑能力,适合交互式3D应用

我们提出TRON,一种结合3D高斯射线追踪与神经渲染的框架,可在新光照、动态物体运动、物体插入和材质编辑下实现真实世界3D场景的逼真且可控制渲染。现有基于物理渲染(PBR)的高斯方法因重建几何、材质估计和光传输误差难以实现逼真重光照;而神经渲染通常缺乏显式场景表示,限制了交互编辑能力。TRON融合二者优势:利用学习的逆渲染模型中的内在分解先验来正则化高斯场的材质属性,并将射线追踪器作为辐射度引导而非直接输出像素。该输出构成结构化的3D骨架,驱动轻量级神经渲染器弥合着色模型受限估计与逼真图像之间的域差距。关键洞察在于,显式3D知识结合强材质先验带来速度与可控性,神经渲染则实现照片级图像合成。为支持真实场景,我们在包含210万帧合成与真实世界帧的新数据集上采用多阶段训练策略——大规模预训练后进行针对性微调。TRON在真实性上优于基于高斯的重光照方法,在可编辑性和速度上超越先前神经渲染方法。据我们所知,TRON是首个在真实捕捉3D环境中实现实用交互应用的方法,能应对动态几何、光照与材质变化下的逼真外观表现。

原文摘要 · Abstract (English)

We introduce TRON, a rendering framework that combines 3D Gaussian ray tracing with neural rendering to enable realistic and controllable rendering of real-world 3D scenes under novel lighting, dynamic object motion, object insertion, and material editing. Prior approaches that rely solely on physically based rendering (PBR) of Gaussian representations struggle to achieve realistic relighting due to imperfections in reconstructed geometry, material estimates, and light transport estimation. At the same time, neural rendering methods often lack an explicit scene representation, limiting their ability to support interactive editing with fine-grained manipulation. TRON bridges these two paradigms. We use intrinsic decomposition priors from a learned inverse rendering model to regularize the material properties of a Gaussian field, and repurpose a ray tracer to provide radiometric guidance rather than final pixels. By treating this output as a structured 3D scaffold, we empower a lightweight neural renderer to bridge the domain gap between shading-model constrained estimates and photorealistic output. Our key insight is that the combination of explicit 3D knowledge with robust material priors provides speed and controllability, while neural rendering enables the synthesis of photorealistic images. To support real-world scenarios, we train our neural renderer with a multi-stage strategy consisting of large-scale pretraining and targeted fine-tuning on a newly constructed dataset of 2.1M rendered synthetic and real-world frames from 3D reconstructions. TRON outperforms Gaussian-based relighting methods in realism, and prior neural renderers in editability and speed. To the best of our knowledge, TRON is the first method to enable practical interactive applications in captured 3D environments, offering realistic appearance under dynamic geometric, lighting and material conditions.

3D重建神经渲染高斯建模交互渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。