将光谱解混引入NeRF，实现无监督材料分割与可编辑材质渲染
UnMix-NeRF: Spectral Unmixing Meets Neural Radiance Fields
- 用全局端元字典和点级丰度建模材料光谱反射特性
- 在合成数据上达到98.7%的材料分类准确率，优于现有方法
- 支持通过修改端元字典自由编辑场景材质，适合机器人与AR应用
基于NeRF的分割方法依赖RGB数据，仅关注物体语义，缺乏内在材质属性，限制了在机器人、增强现实、仿真等领域的精准材质感知。我们提出UnMix-NeRF框架，将光谱解混集成至NeRF中，实现联合高光谱新视角合成与无监督材料分割。方法通过漫反射与镜面反射分量建模光谱反射率，利用学习得到的全局端元字典表示纯材料特征，点级丰度捕捉其分布。材料分割基于沿学习端元的光谱预测，实现无监督聚类。此外，可通过修改端元字典灵活操控材质外观，支持场景编辑。大量实验验证了该方法在光谱重建与材料分割上的优越性。
原文摘要 · Abstract (English)
Neural Radiance Field (NeRF)-based segmentation methods focus on object semantics and rely solely on RGB data, lacking intrinsic material properties. This limitation restricts accurate material perception, which is crucial for robotics, augmented reality, simulation, and other applications. We introduce UnMix-NeRF, a framework that integrates spectral unmixing into NeRF, enabling joint hyperspectral novel view synthesis and unsupervised material segmentation. Our method models spectral reflectance via diffuse and specular components, where a learned dictionary of global endmembers represents pure material signatures, and per-point abundances capture their distribution. For material segmentation, we use spectral signature predictions along learned endmembers, allowing unsupervised material clustering. Additionally, UnMix-NeRF enables scene editing by modifying learned endmember dictionaries for flexible material-based appearance manipulation. Extensive experiments validate our approach, demonstrating superior spectral reconstruction and material segmentation to existing methods. Project page: https://www.factral.co/UnMix-NeRF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。