用轻量微调让扩散模型高效修复扫描探针显微镜图像中的结构伪影。
From Artefact to Insight: Efficient Low-Rank Adaptation of BrushNet for Scanning Probe Microscopy Image Restoration
- 用低秩微调(LoRA)仅调整0.2%参数,适配预训练扩散模型。
- 在7390对数据上微调后,PSNR提升6.61dB,LPIPS减半,媲美全量训练。
- 适合纳米成像领域研究者,可在单张普通显卡上运行。
扫描探针显微镜(SPM)虽具纳米级分辨率,却常受线扫描缺失、增益噪声、探针卷积和相位跳跃等结构伪影干扰。现有方法多将去伪影视为独立去噪或插值任务,生成式修复尚处空白。本文提出基于扩散模型的修复框架,仅用7390对从739次实验中提取的伪影-清洁图像对,通过小于0.2%权重的低秩微调(LoRA)适配预训练BrushNet。在即将发布的公开基准SPM InpBench上,该方法使峰值信噪比(PSNR)提升6.61 dB,感知图像相似性(LPIPS)减半,性能接近甚至略超全量重训,且仅需单张GPU而非四张高内存卡。模型可跨高度、振幅、相位等多种通道泛化,精准还原细微结构,有效抑制自然图像先验带来的幻觉伪影。该轻量框架为不可复现的SPM图像提供了高效可扩展的恢复方案,推动扩散模型在纳米成像分析中的广泛应用。
原文摘要 · Abstract (English)
Scanning Probe Microscopy or SPM offers nanoscale resolution but is frequently marred by structured artefacts such as line scan dropout, gain induced noise, tip convolution, and phase hops. While most available methods treat SPM artefact removal as isolated denoising or interpolation tasks, the generative inpainting perspective remains largely unexplored. In this work, we introduce a diffusion based inpainting framework tailored to scientific grayscale imagery. By fine tuning less than 0.2 percent of BrushNet weights with rank constrained low rank adaptation (LoRA), we adapt a pretrained diffusion model using only 7390 artefact, clean pairs distilled from 739 experimental scans. On our forthcoming public SPM InpBench benchmark, the LoRA enhanced model lifts the Peak Signal to Noise Ratio or PSNR by 6.61 dB and halves the Learned Perceptual Image Patch Similarity or LPIPS relative to zero-shot inference, while matching or slightly surpassing the accuracy of full retraining, trainable on a single GPU instead of four high-memory cards. The approach generalizes across various SPM image channels including height, amplitude and phase, faithfully restores subtle structural details, and suppresses hallucination artefacts inherited from natural image priors. This lightweight framework enables efficient, scalable recovery of irreplaceable SPM images and paves the way for a broader diffusion model adoption in nanoscopic imaging analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。