提出自适应Transformer,解决医学图像修复中多任务干扰与学习不均衡问题。
TAT: Task-Adaptive Transformer for All-in-One Medical Image Restoration
- 通过任务自适应权重生成,动态分配各任务专属参数,缓解梯度冲突。
- 基于任务难度自适应调整损失权重,避免强任务压制弱任务。
- 在PET合成、CT去噪、MRI超分三项任务上均达领先效果,适合多任务医疗图像修复场景。
医学图像修复(MedIR)旨在从低质量图像中恢复高质量图像。近期研究聚焦于能同时处理多种不同医图修复任务的全合一模型。然而,由于模态和退化类型差异显著,使用共享模型需关注两个关键任务间关系:任务干扰(跨任务参数更新方向冲突)与任务失衡(因任务学习难易不一导致优化不均)。为此,我们提出任务自适应Transformer(TAT),通过两项创新动态适配不同任务:首先,引入任务自适应权重生成策略,为每项任务生成专属权重参数,消除共享参数上的潜在梯度冲突;其次,设计任务自适应损失平衡策略,根据任务学习难度动态调整损失权重,防止强任务主导或弱任务欠训练。大量实验表明,TAT在三项医图修复任务——PET合成、CT去噪、MRI超分辨率——中均实现先进性能,且在任务特定与全合一设置下表现优异。代码已开源:https://github.com/Yaziwel/TAT。
原文摘要 · Abstract (English)
Medical image restoration (MedIR) aims to recover high-quality medical images from their low-quality counterparts. Recent advancements in MedIR have focused on All-in-One models capable of simultaneously addressing multiple different MedIR tasks. However, due to significant differences in both modality and degradation types, using a shared model for these diverse tasks requires careful consideration of two critical inter-task relationships: task interference, which occurs when conflicting gradient update directions arise across tasks on the same parameter, and task imbalance, which refers to uneven optimization caused by varying learning difficulties inherent to each task. To address these challenges, we propose a task-adaptive Transformer (TAT), a novel framework that dynamically adapts to different tasks through two key innovations. First, a task-adaptive weight generation strategy is introduced to mitigate task interference by generating task-specific weight parameters for each task, thereby eliminating potential gradient conflicts on shared weight parameters. Second, a task-adaptive loss balancing strategy is introduced to dynamically adjust loss weights based on task-specific learning difficulties, preventing task domination or undertraining. Extensive experiments demonstrate that our proposed TAT achieves state-of-the-art performance in three MedIR tasks--PET synthesis, CT denoising, and MRI super-resolution--both in task-specific and All-in-One settings. Code is available at https://github.com/Yaziwel/TAT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。