arXiv:2501.09259cs.CVphysics.app-ph2025-01

融合干涉与光学显微图像,实现微结构高精度带色三维重建。

OpticFusion: Multi-Modal Neural Implicit 3D Reconstruction of Microstructures by Fusing White Light Interferometry and Optical Microscopy

  • 用多视角干涉图与显微图像联合建模,通过神经隐式表示融合数据。
  • 重建结果包含精细几何与自然颜色,真实微结构纹理还原度高。
  • 无需昂贵硬件改造,适合材料、生物等领域的微观分析应用。

白光干涉测量(WLI)是精确获取微结构三维形貌的光学工具,但无法捕捉样品表面的真实颜色,而颜色信息对众多微尺度研究至关重要。以往方法通过改进硬件或软件来弥补此缺陷,但成本高昂。本文首次从计算机视觉多模态重建角度提出OpticFusion,利用额外数字光学显微镜(OM)采集多视角WLI与OM图像,通过两步数据关联获得两者姿态。基于神经隐式表示融合多模态数据,并采用颜色分解技术提取样本真实颜色。在自建多模态微结构数据集上测试,该方法实现了具有自然颜色纹理的高细节三维重建。所提方法为多个微尺度研究领域提供了实用工具。源码与真实世界数据集已公开于https://github.com/zju3dv/OpticFusion。

原文摘要 · Abstract (English)

White Light Interferometry (WLI) is a precise optical tool for measuring the 3D topography of microstructures. However, conventional WLI cannot capture the natural color of a sample's surface, which is essential for many microscale research applications that require both 3D geometry and color information. Previous methods have attempted to overcome this limitation by modifying WLI hardware and analysis software, but these solutions are often costly. In this work, we address this challenge from a computer vision multi-modal reconstruction perspective for the first time. We introduce OpticFusion, a novel approach that uses an additional digital optical microscope (OM) to achieve 3D reconstruction with natural color textures using multi-view WLI and OM images. Our method employs a two-step data association process to obtain the poses of WLI and OM data. By leveraging the neural implicit representation, we fuse multi-modal data and apply color decomposition technology to extract the sample's natural color. Tested on our multi-modal dataset of various microscale samples, OpticFusion achieves detailed 3D reconstructions with color textures. Our method provides an effective tool for practical applications across numerous microscale research fields. The source code and our real-world dataset are available at https://github.com/zju3dv/OpticFusion.

三维重建多模态融合显微成像神经隐式

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。