用光学先验训练神经网络,提升超微型金属透镜内窥镜成像质量。
MetaScope: Optics-Driven Neural Network for Ultra-Micro Metalens Endoscopy
- 基于物理光学先验设计神经网络,学习金属透镜的点扩散函数与空间畸变。
- 解决金属透镜成像中的亮度衰减和色差问题,提升分割与重建效果。
- 适用于微型内窥镜场景,尤其适合生物医学真实成像任务。
微型内窥镜已推动人体内部精准视觉感知的发展。现有研究仍受限于传统凸透镜相机,其毫米级厚度严重制约微尺度临床应用。随着超材料光学兴起,基于微米级金属透镜的超微型成像备受关注。然而,金属透镜的物理特性差异导致数据采集与算法研究严重滞后。为此,本文旨在填补该空白,推进新型金属透镜内窥镜技术。首先,构建金属透镜内窥镜数据集并开展初步光学仿真,识别出两个受强光学先验约束的物理问题。其次,提出MetaScope——一种面向金属透镜内窥镜的光学驱动神经网络。其核心包括:光强补偿模块(OIA),通过学习光学嵌入纠正强度衰减;色差校正模块(OCC),基于学习到的点扩散函数分布,通过空间形变学习缓解色差。为增强联合学习,引入梯度引导的知识蒸馏机制。大量实验表明,MetaScope在金属透镜图像分割与复原上均优于当前最优方法,并在真实生物医学场景中展现出出色泛化能力。
原文摘要 · Abstract (English)
Miniaturized endoscopy has advanced accurate visual perception within the human body. Prevailing research remains limited to conventional cameras employing convex lenses, where the physical constraints with millimetre-scale thickness impose serious impediments on the micro-level clinical. Recently, with the emergence of meta-optics, ultra-micro imaging based on metalenses (micron-scale) has garnered great attention, serving as a promising solution. However, due to the physical difference of metalens, there is a large gap in data acquisition and algorithm research. In light of this, we aim to bridge this unexplored gap, advancing the novel metalens endoscopy. First, we establish datasets for metalens endoscopy and conduct preliminary optical simulation, identifying two derived optical issues that physically adhere to strong optical priors. Second, we propose MetaScope, a novel optics-driven neural network tailored for metalens endoscopy driven by physical optics. MetaScope comprises two novel designs: Optics-informed Intensity Adjustment (OIA), rectifying intensity decay by learning optical embeddings, and Optics-informed Chromatic Correction (OCC), mitigating chromatic aberration by learning spatial deformations informed by learned Point Spread Function (PSF) distributions. To enhance joint learning, we further deploy a gradient-guided distillation to transfer knowledge from the foundational model adaptively. Extensive experiments demonstrate that MetaScope not only outperforms state-of-the-art methods in both metalens segmentation and restoration but also achieves impressive generalized ability in real biomedical scenes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。