用Transformer直接渲染三角网格，实现全局光照且无需场景微调。
RenderFormer: Transformer-based Neural Rendering of Triangle Meshes with Global Illumination
- 将三角网格转为序列，通过Transformer分两阶段完成渲染
- 支持复杂形状与光照传输，无需针对每个场景训练
- 适合需要快速渲染真实光照的3D应用开发
我们提出RenderFormer，一种基于Transformer的神经渲染流程，可直接从三角网格表示的场景中生成带完整全局光照效果的图像，且无需针对特定场景进行训练或微调。不同于传统物理驱动的渲染方式，我们将渲染建模为序列到序列的转换：将包含反射属性的三角形序列转换为表示像素小块的输出序列。RenderFormer采用两阶段架构：第一阶段为视图无关阶段，建模三角形之间的光传输；第二阶段为视图相关阶段，将代表射线束的令牌转换为对应像素值，由第一阶段的三角形序列引导。两个阶段均基于Transformer结构，并在最小先验约束下学习。我们在具有不同复杂度形状和光传输特性的场景上验证了RenderFormer的有效性。
原文摘要 · Abstract (English)
We present RenderFormer, a neural rendering pipeline that directly renders an image from a triangle-based representation of a scene with full global illumination effects and that does not require per-scene training or fine-tuning. Instead of taking a physics-centric approach to rendering, we formulate rendering as a sequence-to-sequence transformation where a sequence of tokens representing triangles with reflectance properties is converted to a sequence of output tokens representing small patches of pixels. RenderFormer follows a two stage pipeline: a view-independent stage that models triangle-to-triangle light transport, and a view-dependent stage that transforms a token representing a bundle of rays to the corresponding pixel values guided by the triangle-sequence from the view-independent stage. Both stages are based on the transformer architecture and are learned with minimal prior constraints. We demonstrate and evaluate RenderFormer on scenes with varying complexity in shape and light transport.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。