用可编辑的神经素实现物体级3D场景分解与逼真渲染
MLP Splatting: Object-Centric Neural Fields

- 每个物体用独立小MLP表示,局部化建模辐射与透明度
- 内存降低15倍、渲染快3倍,支持无需分割图的交互编辑
- 适合需要物体级操作的3D内容创作与开放词汇交互
3D表征是场景渲染、理解与交互的基础。现有方法如3D高斯泼溅和神经辐射场虽能实现逼真新视角合成,但难以将场景元素分解为少数基础单元,需额外分割或分组才能进行物体级操作。本文提出MLP-Splatting,通过少量表达力强的光场基元实现场景分解,并保持逼真新视角合成能力。每个基元由独立紧凑的MLP建模,具有局部空间支持,预测辐射与不透明度。相比低层高斯基元或单一全局辐射场,我们的神经基元具备更强表达能力且仍保持空间局部性。渲染通过高效的稀疏体素合成完成,基于射线-基元交互。仅使用RGB监督即可训练出代表局部场景区域的基元,常对应物体或部件,支持通过选择少量基元实现无需分割图的交互式物体编辑。引入可选语义特征蒸馏后,还能实现开放词汇场景交互与开集即时分割。实验表明,相比先进方法,本方法内存降低15倍,渲染速度提升3倍。
原文摘要 · Abstract (English)
3D representations are fundamental to scene rendering, understanding, and interaction. Recent approaches, such as 3D Gaussian Splatting and Neural Radiance Fields, achieve impressive photorealistic novel-view synthesis, but lack the ability to easily decompose scene elements into a few primitives, requiring additional segmentation or grouping for object-level manipulation. We present MLP-Splatting, a method that enables scene decomposition via a few expressive light-field primitives while providing photorealistic novel-view synthesis. MLP-Splatting models each primitive as an independent compact MLP with localized spatial support that predicts radiance and opacity. In contrast to low-level Gaussian primitives or a single global radiance field, our neural primitives provide greater expressive capacity while remaining spatially localized. Rendering is performed through efficient sparse volumetric compositing over ray-primitive interactions. Our primitives are supervised using RGB supervision alone, which yields primitives that represent local scene regions often corresponding to objects or object parts, enabling interactive object-level editing without segmentation masks by selecting a handful of primitives. Our method, augmented with optional semantic feature distillation, enables open-vocabulary scene interaction and open-set instant segmentation. Compared to state-of-the-art methods, we achieve substantially lower memory usage (1/15$\times$) and faster rendering (3$\times$), as we show in our experiments compared to semantic 3DGS methods. Project Page: https://shinjeongkim.com/mlp-splatting
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。