研究大体积图像生成中的拼接伪影,发现感知指标无法捕捉影响分割性能的细微瑕疵。
Stitching and dimensionality effects on large artificially generated volume datasets

- 对比三种拼接方法与2D/3D patch维度,评估生成效果
- FID分数忽略细微伪影,3D模型仅微弱提升分割性能
- 2D模型训练更稳定,多方向集成对低质量数据有帮助
通过循环GAN模型在冷冻电镜数据集上训练,研究了三种拼接策略与2D/3Dpatch维度对大规模图像生成的影响。评估内容包括感知质量与下游线粒体分割表现。结果表明:(1) FID分数无法检测显著影响分割任务的细微拼接伪影;(2) 采用无伪影拼接的3D模型在下游任务中表现略优于2D模型,但提升有限,难以抵消计算开销;(3) 2D模型因支持更大批处理而训练更稳定。此外,从三个正交方向集成预测可改善低质量输出,但对高质量数据无效。研究强调,在生物医学成像中,仅依赖感知指标不足以评估领域适配质量,需重视拼接伪影的防控。
原文摘要 · Abstract (English)
Generating large images via deep learning requires patching input data to accommodate hardware memory limitations, then assembling output patches, a process that can introduce stitching artifacts when neighboring patches do not align at borders. While these artifacts are known to affect segmentation tasks, their impact on generative models for style-transfer remains poorly understood. We investigated three stitching approaches and two patch dimensionalities (2D vs 3D) using cycleGAN models trained on cryo-electron microscopy datasets. We evaluated both perceptual quality and performance on downstream mitochondria segmentation. Our key findings reveal that: (1) FID scores fail to detect subtle stitching artifacts that significantly impact downstream segmentation performance, (2) 3D models with artifact-free stitching marginally outperform 2D models on downstream tasks, though the improvement barely justifies the computational cost, and (3) 2D models train more stably due to larger batch sizes. Additionally, we demonstrate that ensembling predictions from three orthogonal directions can improve low-quality volumes but provides no benefit for high-quality outputs. These results demonstrate that maximizing generative model performance on large scientific datasets requires careful consideration and mitigation of stitching artifacts, and that perceptual metrics alone are insufficient for evaluating domain adaptation quality in biomedical imaging.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。