将医学图像修复与分割联合建模,提升临床图像处理效果
Vision-Language Controlled Deep Unfolding for Joint Medical Image Restoration and Segmentation
- 将修复与分割统一为优化问题,实现数学耦合的协同优化
- 在多模态数据上提升PSNR 0.92 dB,Dice系数提高9.76%
- 适合需要高精度图像重建与精准分割的医疗影像研究者
我们提出VL-DUN,一种面向联合全融合医学图像恢复与分割(AiOMIRS)的原理性框架,弥合低层信号恢复与高层语义理解之间的鸿沟。传统流程将任务割裂处理,而我们的核心洞察是二者具有根本协同性:修复提供清晰解剖结构以增强分割,语义先验则正则化恢复过程。VL-DUN通过两项创新解决序列处理的次优问题:(1) 将AiOMIRS建模为统一优化问题,推导出可解释的联合展开机制,使修复与分割数学耦合并相互精炼;(2) 引入频域感知的Mamba机制,捕捉长程依赖以实现全局分割,同时保留恢复所需的高频纹理。该机制以线性复杂度实现高效全局上下文建模,有效缓解标准架构的频谱偏差。作为该项任务的开创性工作,VL-DUN在多模态基准上达到新最优性能,提升PSNR 0.92 dB,Dice系数提升9.76%。结果表明,联合协同学习相比孤立处理,为复杂临床流程提供了更优、更鲁棒的解决方案。代码已开源:https://github.com/cipi666/VLDUN。
原文摘要 · Abstract (English)
We propose VL-DUN, a principled framework for joint All-in-One Medical Image Restoration and Segmentation (AiOMIRS) that bridges the gap between low-level signal recovery and high-level semantic understanding. While standard pipelines treat these tasks in isolation, our core insight is that they are fundamentally synergistic: restoration provides clean anatomical structures to improve segmentation, while semantic priors regularize the restoration process. VL-DUN resolves the sub-optimality of sequential processing through two primary innovations. (1) We formulate AiOMIRS as a unified optimization problem, deriving an interpretable joint unfolding mechanism where restoration and segmentation are mathematically coupled for mutual refinement. (2) We introduce a frequency-aware Mamba mechanism to capture long-range dependencies for global segmentation while preserving the high-frequency textures necessary for restoration. This allows for efficient global context modeling with linear complexity, effectively mitigating the spectral bias of standard architectures. As a pioneering work in the AiOMIRS task, VL-DUN establishes a new state-of-the-art across multi-modal benchmarks, improving PSNR by 0.92 dB and the Dice coefficient by 9.76\%. Our results demonstrate that joint collaborative learning offers a superior, more robust solution for complex clinical workflows compared to isolated task processing. The codes are provided in https://github.com/cipi666/VLDUN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。