让NeRF同时处理多倍放大图像,精准还原微米级工业细节。
MZEN: Multi-Zoom Enhanced NeRF for 3-D Reconstruction with Unknown Camera Poses
- 引入可学习的缩放参数,扩展针孔相机模型以支持多倍放大。
- 先用广角图定全局结构,再通过缩放一致的裁剪匹配定位放大图。
- 在工业检测场景中显著提升细节还原能力,适合微米级缺陷分析。
NeRF方法在无相机位姿条件下仍能实现高质量3D重建,但在工业检测中仍难以捕捉关键细粒度结构,如生产线上的亚微米级缺陷或扫描电镜(SEM)图像中的芯片细节。此时传感器分辨率固定、计算资源有限,唯一提升细节的方法是增加放大图像,但会破坏多视角一致性,影响无位姿训练。本文提出首个原生支持多倍放大图像集的NeRF框架——MZEN:(i) 在针孔相机模型中加入显式可学习的缩放系数,动态调节焦距;(ii) 提出新位姿策略:先用广角图像建立全局度量空间,再将放大图像通过缩放一致的裁剪与匹配操作,对齐至最近的广角图像后进行联合优化。在八组前向场景(包括合成TCAD模型、真实SEM微结构及BLEFF物体)上,MZEN持续优于无位姿基线模型,甚至超越高分辨率变体,峰值信噪比(PSNR)提升最高达28%,结构相似性(SSIM)提升10%,低层次感知图像相似度(LPIPS)降低最高达222%。因此,MZEN将NeRF拓展至真实工厂环境,在保持全局精度的同时,精准捕获工业检测所需的微米级细节。
原文摘要 · Abstract (English)
Neural Radiance Fields (NeRF) methods excel at 3D reconstruction from multiple 2D images, even those taken with unknown camera poses. However, they still miss the fine-detailed structures that matter in industrial inspection, e.g., detecting sub-micron defects on a production line or analyzing chips with Scanning Electron Microscopy (SEM). In these scenarios, the sensor resolution is fixed and compute budgets are tight, so the only way to expose fine structure is to add zoom-in images; yet, this breaks the multi-view consistency that pose-free NeRF training relies on. We propose Multi-Zoom Enhanced NeRF (MZEN), the first NeRF framework that natively handles multi-zoom image sets. MZEN (i) augments the pin-hole camera model with an explicit, learnable zoom scalar that scales the focal length, and (ii) introduces a novel pose strategy: wide-field images are solved first to establish a global metric frame, and zoom-in images are then pose-primed to the nearest wide-field counterpart via a zoom-consistent crop-and-match procedure before joint refinement. Across eight forward-facing scenes$\unicode{x2013}$synthetic TCAD models, real SEM of micro-structures, and BLEFF objects$\unicode{x2013}$MZEN consistently outperforms pose-free baselines and even high-resolution variants, boosting PSNR by up to $28 \%$, SSIM by $10 \%$, and reducing LPIPS by up to $222 \%$. MZEN, therefore, extends NeRF to real-world factory settings, preserving global accuracy while capturing the micron-level details essential for industrial inspection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。