真实低分辨率数据缺失导致3D超分辨模型性能虚高,新数据集揭示此偏差。
VoDaSuRe: A Large-Scale Dataset Revealing Domain Shift in Volumetric Super-Resolution
- 构建包含真实高低分辨率配对扫描的大规模3D数据集VoDaSuRe
- 用真实数据训练的模型保留结构但精度差,用下采样数据训练的更锐利但不准确
- 提醒研究者需用真实复杂数据评估3D超分辨方法,避免过度乐观
近期3D超分辨技术在医学与科学成像中表现优异,基于Transformer和CNN的方法在极端放大倍率下也取得良好效果。然而本文发现,这些性能很大程度上源于在下采样数据上训练,而非真实低分辨率扫描。这一依赖性部分源于缺乏成对的高/低分辨率3D数据集。为此,我们提出VoDaSuRe——一个大规模体积数据集,包含真实配对的高低分辨率扫描。在该数据集上训练模型后,我们发现:在下采样数据上训练的模型生成图像更锐利,而用真实低分辨率数据训练的模型则会平滑细结构;反之,将下采样训练的模型应用于真实扫描时虽能保留更多结构,但预测不准确。结果表明当前超分辨方法被夸大——在真实数据上无法恢复低分辨率扫描中丢失的结构,反而预测出平滑的平均值。我们认为,深度学习驱动的3D超分辨进展需要如VoDaSuRe这样具有高复杂度的真实配对数据集。数据集与代码已公开:https://augusthoeg.github.io/VoDaSuRe/
原文摘要 · Abstract (English)
Recent advances in volumetric super-resolution (SR) have demonstrated strong performance in medical and scientific imaging, with transformer- and CNN-based approaches achieving impressive results even at extreme scaling factors. In this work, we show that much of this performance stems from training on downsampled data rather than real low-resolution scans. This reliance on downsampling is partly driven by the scarcity of paired high- and low-resolution 3D datasets. To address this, we introduce VoDaSuRe, a large-scale volumetric dataset containing paired high- and low-resolution scans. When training models on VoDaSuRe, we reveal a significant discrepancy: SR models trained on downsampled data produce substantially sharper predictions than those trained on real low-resolution scans, which smooth fine structures. Conversely, applying models trained on downsampled data to real scans preserves more structure but is inaccurate. Our findings suggest that current SR methods are overstated - when applied to real data, they do not recover structures lost in low-resolution scans and instead predict a smoothed average. We argue that progress in deep learning-based volumetric SR requires datasets with paired real scans of high complexity, such as VoDaSuRe. Our dataset and code are publicly available through: https://augusthoeg.github.io/VoDaSuRe/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。