用物理模型生成显微镜图像数据,解决真实标注数据少的问题。
OSOG: A Differentiable, Physics-Informed Synthetic Data Engine for Micro-Optical Environments

- 基于衍射和相位延迟的物理模型,实现可微分的光学仿真
- 50毫秒内生成4万张波前图像,速度超20帧/秒
- 适合做显微图像检测与逆向参数恢复的研究者
计算显微中的深度学习受限于密集标注数据的稀缺。尽管合成数据在宏观计算机视觉中已有所突破,传统图形引擎依赖几何光线追踪,无法捕捉显微所需的微光学现象;而波光学建模虽存在,却难以在深度学习所需规模下高效渲染。为此,我们提出光学合成对象生成器(OSOG),一个高性能、全可微分的前向建模引擎。基于衍射与相位延迟的物理模型,将连续的光程差(OPD)计算映射至原生PyTorch的结构化数组(SoA)架构。我们在三个维度验证该框架:首先,仅用OSOG生成数据训练的YOLOv11-OBB目标检测模型,在真实高遮挡溶菌酶显微图像上实现鲁棒零样本迁移;其次,引入DiffOSOG,证明其端到端可微性可实现连续光学参数的精确恢复;最后,相比传统串行光线追踪的$/mathcal{O}(N)$瓶颈,OSOG实现亚线性扩展,在50毫秒内完成40,000个复杂波前粒子的合成(>20 FPS)。OSOG通过快速、可扩展且物理可信的张量流水线,支持真正的实时、按需数据生成。
原文摘要 · Abstract (English)
Deep learning in computational microscopy is severely constrained by the scarcity of densely annotated datasets. While synthetic data generation has bridged this gap in macroscopic computer vision, traditional graphics engines rely on geometric ray-tracing, failing to capture the micro-optical phenomena required for microscopy. Conversely, while wave-optics formulations exist, rendering them computationally tractable at the scale required for deep learning remains a massive systems challenge. To address this, we introduce the Optical Synthetic Object Generator (OSOG), a high-performance, fully differentiable forward-modeling engine. Drawing on established physical models of diffraction and phase retardation, OSOG maps continuous Optical Path Difference (OPD) calculations into a highly optimized, PyTorch-native Structure-of-Arrays (SoA) architecture. We validate this computational framework across three axes: First, object detection models (YOLOv11-OBB) trained purely on OSOG-generated data achieve robust zero-shot transfer to real-world highly occluded Lysozyme micrographs. Second, we introduce DiffOSOG, demonstrating that the engine's end-to-end differentiability allows for the exact recovery of continuous optical parameters via curriculum-guided inverse rendering. Finally, OSOG bypasses the $\mathcal{O}(N)$ bottlenecks of sequential ray-tracing, demonstrating sub-linear scaling by synthesizing 40,000 complex wave-optic particles in under 50 milliseconds (\>20 FPS). By providing a fast, scalable, and physically grounded tensor pipeline, OSOG enables true real-time, on-the-fly dataset generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。